Roundup Tracker - Issues

Issue 2551173

classification
Should the ETag Header get sent on POST, PUT and PATCH ?
Type: behavior Severity: normal
Components: API Versions:
process
Status: new
: Make ETag content-encoding aware.
View: 2551175
: : marcus.priesch, rouilj
Priority: :

Created on 2021-11-30 10:26 by marcus.priesch, last changed 2022-07-09 22:24 by rouilj.

Messages
msg7380 Author: [hidden] (marcus.priesch) Date: 2021-11-30 10:26
I am currently developing a web-frontend which uses the REST API (via vuex-rest-api) 

Currently the ETag Header is only sent when i GET a ressource.

Now, when the user changes parts of the ressource and i update it using a PUT 
request (encapsulated in a POST) i get back the changed parts, but no ETag Header.

The same happens when the user creates a new ressource with a POST request i 
get back the id and URL, but also no ETag Header.

Therefore i always need to issue an additional GET request on the same ressource 
afterwards to have the ETag information for further editing (PUT'ing) the ressource.

Which is 1) ressource consuming and 2) not convenient ;)

So the Question is: 

is the current behaviour by design ? 

or can we change it so that the Etag Header gets sent automatically when issuing PUT, POST or PATCH.

regards,
marcus.
msg7381 Author: [hidden] (rouilj) Date: 2021-11-30 18:25
Hi Marcus:

In message <1638268008.03.0.115660569954.issue2551173@roundup.psfhosted.org>,
Marcus Priesch writes:
>I am currently developing a web-frontend which uses the REST API
>(via vuex-rest-api) 

Cool.

>Currently the ETag Header is only sent when i GET a resource.

Yup this is by design as I didn't find any RFC that discussed ETag
generation for anything but GET and HEAD.

>Now, when the user changes parts of the ressource and i update it using
>a PUT request (encapsulated in a POST) i get back the changed parts,
>but no ETag Header.

Correct. The ETag header maps the URL to the full representation of
the URL not a partial representation. ICU PUT responses are not
supposed to be cached by intermediate caches or the browser but ...

>The same happens when the user creates a new ressource with a POST
>request i get back the id and URL, but also no ETag Header.

Same answer as above.

>Therefore i always need to issue an additional GET request on the same
>ressource afterwards to have the ETag information for further editing
>(PUT'ing) the ressource.

That appears to be the case in REST. (IIRC you can issue a HEAD and
get back the ETag without having to transfer the bytes in the
representation. It is the same amount of work as a GET for the server
but does save transmission costs.)

>Which is 1) ressource consuming and 2) not convenient ;)

Yup.

>So the Question is: 
>
>is the current behaviour by design ? 

I am going to say yes.

>or can we change it so that the Etag Header gets sent automatically
>when issuing PUT, POST or PATCH.

https://stackoverflow.com/questions/42246577/why-responses-to-put-requests-must-not-provide-an-etag#42248759

would seem to indicate not as the response doesn't include a the full
representation like a GET.

While https://cloud.google.com/apis/design/standard_methods does
indicate that the resource should be returned on a put/patch it is
possible to have a partial representation which should not have an
ETag header. (We supply a partial representation.)

Since the partial representation is not a valid representation of the
whole (https://.../issue/1) URL, it must not be cached (using the etag
header) at the very least.

The PATCH request has the same partial representation problem IIRC.

For a POST we don't return any representation of the newly created
object, so an ETag header isn't allowed.

So adding the header would not appear to be correct. I wonder if
adding an @etag to the metadata part of the response would work? So for
POST you get:

  {
    "data": {
        "link": "https://.../demo/rest/data/issue/2280",
        "id": "2280",
        "@etag": "xyzzy....rstq"
    }
  }

for PUT:

    {
      "data": {
          "attribute": {
              "title": "This is now my title"
          },
          "type": "issue",
          "link": "https://.../demo/rest/data/issue/23",
          "id": "23",
	  "@etag": "xyzzy....rstq"
      }
  }

and similarly for PATCH:

  {
      "data": {
          "attribute": {
              "nosy": [
                  "3",
                  "4"
              ]
          },
          "type": "issue",
          "link": "https://.../rest/data/issue/23",
          "id": "23",
          "@etag": "xyzzy....rstq"
      }
  }

where the @etag value is what you would get if you were to make a
GET/HEAD request. I am not sure how easy this is to implement though.

While I was looking into this, it turns out that the ETag must be
different for different content-encodings.
https://datatracker.ietf.org/doc/html/rfc7232#section-2.3.3.  We do
explicitly set the vary header to include accept-encoding, but
apparently, that is not enough.

I am not sure how that affects the use of ETag and intermediate
caches. I suspect the easiest way to handle this is to take the ETag
and append the content-encoding. So

  ETag: 584f82231079e349031bbb853747df1c-gzip
  Content-Encoding: gzip

  ETag: 584f82231079e349031bbb853747df1c-br
  Content-Encoding: br

  ETag: 584f82231079e349031bbb853747df1c-zstd
  Content-Encoding: zstd

and then strip the suffix when comparing it server side. So as a
client you don't care about the ETag value, it still remains opaque.
But an intermediate cache will have three different copies that they
can return depending on what the accept-header is.
msg7382 Author: [hidden] (marcus.priesch) Date: 2021-11-30 19:57
Hello John,

i am very sorry, i completely missed what the ETag really is designed
for ! sorry for that !

i just read the wikipedia page - and YES, it should only be present on
GET - and definitely needs to have the encodig part in it !

but for our use case it would definitely make sense to have that
metadata thing ... :) - i will give it a look tomorrow ...

thanks,
marcus.
msg7383 Author: [hidden] (rouilj) Date: 2021-11-30 20:34
Hi Marcus:

In message <f8ca8fd9-78b7-6eb7-f290-358c21a93b83@priesch.co.at>,
Marcus Priesch writes:
>i am very sorry, i completely missed what the ETag really is designed
>for ! sorry for that !

No problem. I did a lot of research on etags and things, but I have
forgotten most of it. It was due for a new look.

>i just read the wikipedia page - and YES, it should only be present on
>GET - and definitely needs to have the encodig part in it !

Yeah. I had hoped that the vary header would do the trick and it did
with varnish. But it looks like I missed the RFC that made it
explicit.

>but for our use case it would definitely make sense to have that
>metadata thing ... :) - i will give it a look tomorrow ...

Cool. You can use this ticket to track the implementation, test, doc
changes etc. There should be no reason to bump the API version. The
@etag value in the envelope should just be ignored by clients that
don't use it.
msg7438 Author: [hidden] (rouilj) Date: 2022-01-09 02:33
Note that implementing <etag>-<content-encoding> implementation was tracked in issue 2551175.
msg7457 Author: [hidden] (rouilj) Date: 2022-03-06 03:16
Hi Marcus:

I am just scanning over open tickets.

Did you ever a patch to add the etag in metadata for Post, Put or Patch?

Have a great week.

-- rouilj
msg7461 Author: [hidden] (marcus.priesch) Date: 2022-03-22 08:20
Hello John,

no, i am sorry, i was busy with other tasks, but will take a look at it 
soon.

regards,
marcus.
msg7603 Author: [hidden] (rouilj) Date: 2022-07-09 22:24
Hi Marcus:

It's too late to put this into roundup 2.2.0, but did you ever code this enhancement?

-- rouilj
History
Date User Action Args
2022-07-09 22:24:15rouiljsetmessages: + msg7603
2022-03-22 08:20:47marcus.prieschsetmessages: + msg7461
2022-03-06 03:16:44rouiljsetmessages: + msg7457
2022-01-09 02:33:33rouiljsetsuperseder: Make ETag content-encoding aware.
messages: + msg7438
2021-11-30 20:34:35rouiljsetmessages: + msg7383
2021-11-30 19:57:48marcus.prieschsetmessages: + msg7382
2021-11-30 18:25:37rouiljsetnosy: + rouilj
messages: + msg7381
2021-11-30 10:26:48marcus.prieschcreate