Regarding the previously raised issue about natural invalidation of PUT/POST requests: Further testing revealed that the solution provided in the previous email was incorrect.
I have been testing with the attached patch (patch against 4.1.2), which with
fairly extensive testing appears to be working well. However there are three
concerns with it:
1> The patch calls do_cache_delete_all_alts() directly, rather than using
the state machine to specify the 'next_action'. Without having a detailed
knowledge of the state machine and valid next-states, I felt it was safer to do
this - at least for proof of concept pending a more complete patch.
Looking for suggestion on correct TRANSACT_RETURN or next_action for this, or
if direct call rather than state machine management is correct in this case.
2> The invalidation happens at the entry of the PUT/POST rather than at the
exit/return to client. This would not normally be a problem. However I have a
use case where a POST calls a separate webapp to validate the data, which then
calls back with a GET using the ID. This GET in the middle of the POST caches
the object again, before the POST took effect.
Looking for suggestions on where to put the invalidation so that it happens at
the exit. (Answer to <1> may correct <2>)
3> This patch only invalidates alternates if the requested object is in the
cache.
Scenario: (Think large multi-faceted web application with multiple client
plugins, where some plugins use XML and others use JSON)
GET application/xml puts the object in the cache. POST application/json tries
to find itself in the cache, but non-existent so doesn't invalidate the
alternates. The now-stale application/xml version of the object stays in the
cache.
Looking for suggestions to invalidate alternates even if the
requested object is not in the cache.
Thanks in advance.
-Norm Paxton
From: Norm Paxton [mailto:[email protected]]
Sent: Wednesday, February 05, 2014 11:49 AM
To: [email protected]
Subject: RE: Cache invalidation for varying media types
Whereas our desire is that a PUT or POST of one content-type causes an
invalidation not only of that instance of the object, but also of all
alternates of that object, does it make sense to simply add the PUT and POST as
additional options to the two s->method checks in
HttpTransact::delete_all_document_alternates_and_return(), as follows?
if ((s->method != HTTP_WKSIDX_GET) && (s->method == HTTP_WKSIDX_DELETE ||
s->method == HTTP_WKSIDX_PURGE)) {
to:
if ((s->method != HTTP_WKSIDX_GET) && (s->method == HTTP_WKSIDX_DELETE ||
s->method == HTTP_WKSIDX_PURGE ||
s->method == HTTP_WKSIDX_POST || s->method == HTTP_WKSIDX_PUT)) {
and
if (s->method == HTTP_WKSIDX_PURGE || (valid_max_forwards && max_forwards
<= 0)) {
to:
if (s->method == HTTP_WKSIDX_PURGE || s->method == HTTP_WKSIDX_POST ||
s->method == HTTP_WKSIDX_PUT ||
(valid_max_forwards && max_forwards <= 0)) {
I have built and tested as such, and it behaves at a high level as we desire.
Will we experience unanticipated side-effects from doing this? Is there a
suite of tests that I can run to ensure that all other behavior is unaffected?
Thanks.
-Norm Paxton
From: Norm Paxton [mailto:[email protected]]
Sent: Monday, February 03, 2014 4:01 PM
To: [email protected]<mailto:[email protected]>
Subject: Cache invalidation for varying media types
There was a thread in June or 2013 regarding cache invalidation of multiple
media types.
(thread:
http://mail-archives.apache.org/mod_mbox/trafficserver-users/201306.mbox/%3ccdd7790c.27424%[email protected]%3E<https://urldefense.proofpoint.com/v1/url?u=http://mail-archives.apache.org/mod_mbox/trafficserver-users/201306.mbox/%253CCDD7790C.27424%2525steve.owens%40email.disney.com%253E&k=wlPCrglRP6kzT4RbABWMaw%3D%3D%0A&r=l7OFdfe5oRC7g5sT34NHIt1cmKfudAyQPYr1BB%2BqIuk%3D%0A&m=HOhRBpsAhe%2BsasH%2FLUaxjZm1wVAYUQrlKZZovk7nueQ%3D%0A&s=0758fdcd5daf78c120a6a9bf6c7d5b1c9b749556b91a2c66f54a7df6f0b8868a>)
Summary of the initial question (Steve Owens):
Is there a configuration (Vary header, proxy.config.http.cache.vary_default_*,
etc) that would cause that a PUT or POST on a specific URL would cause a cache
invalidation on all variant media types of that URL?
Summary of ultimate response (Leif Hedstrom):
This sounds like PURGE, but not currently in PUT or POST. We could look at
putting that behavior into PUT/POST.
My revisit:
Has any action been taken on this? (I have searched the changes.log and not
found anything pertinent, but it is possible that I missed something)
I have a REST api serving data with both 'application/json' and
'application/xml' content-types for each URL. Different clients will consume
one or both of these content-types. When a POST is invoked with
'application/json', my assumption is that the cache server should invalidate
the object for 'application/xml' as well, as this is a variant representation
of the same object. In my testing, I consistently get unexpected (according
to my understanding and expectations) results:
Here's a summarized log of what I see
GET application/xml TCP_HIT
baseline
GET application/json TCP_HIT
baseline
POST application/xml TCP_REFRESH_MISS expected
GET application/json TCP_HIT
unexpected/incorrect/stale data
GET application/xml TCP_MISS
expected
GET application/xml TCP_HIT
expected
I have tried setting the 'proxy.config.http.cache.vary_default_text' and
'...vary_default_other' to Accept, and also setting the Vary header to Accept
in my response, all to no avail. I have also tried Accept values of
"text/html", "text/json", "text/xml", "text/plain" - in case th
e'proxy.config.http.cache.vary_default_text' only looks at 'text/' media type
prefixes, also with no luck. (Based on ATS documentation, and other sources, I
am lead to believe that the correct header to Vary on is Accept, as opposed to
Content-Type. Correct?)
I have confirmed that I can invoke a 'PURGE' on the URL, which will cause
desired end-results, but this requires that upon any PUT/POST request, I will
need to, as part of my processing, explicitly invoke a PURGE in code to
invalidate the cache for that object, which is undesirable. For some
developers/situations, it may even be prohibitive.
Is there plans to change the behavior in a future release, or am I stuck with
self-invoking a PURGE request inside code handling my PUT/POST request?
Or are there other options for configuring this that I have not found/tried?
Thanks
-Norm Paxton
NOTICE: This email message is for the sole use of the intended recipient(s) and
may contain confidential and privileged information. Any unauthorized review,
use, disclosure or distribution is prohibited. If you are not the intended
recipient, please contact the sender by reply email and destroy all copies of
the original message.
NOTICE: This email message is for the sole use of the intended recipient(s) and
may contain confidential and privileged information. Any unauthorized review,
use, disclosure or distribution is prohibited. If you are not the intended
recipient, please contact the sender by reply email and destroy all copies of
the original message.
NOTICE: This email message is for the sole use of the intended recipient(s)
and may contain confidential and privileged information. Any unauthorized
review, use, disclosure or distribution is prohibited. If you are not the
intended recipient, please contact the sender by reply email and destroy all
copies of the original message.
0001-Invalidate-PUT-and-POST.patch
Description: 0001-Invalidate-PUT-and-POST.patch
