Re: [squid-users] negative_ttl & squid 3.1
Calvin Park wrote: Hello squid users~ I am using on squid 3.1.4 , and I need to cache about 404 code. In squid 2.7 , there have rule like "refresh_pattern .0 20% 4320 negative_ttl=xxx" When can I use negative_ttl syntax in squid 3.1 ? negative-ttl=N is an option for Squid caching of helper process lookups and similar when the lookups can be cached. Force-caching HTTP 404 pages is a very bad idea. It is done with a negative_ttl gobal option which caches ALL 4xx and 5xx pages which Squid receives. It extends an artificial DoS against all clients using the proxy if one single user hits a temporary upstream problem. 404 pages should come with useful cache-control expiry information that allows them to be stored for a short while anyway. No special action is required in the 3.1 to cache these HTTP-compliant pages for the proper time (unless you have negative_ttl set, in which case you need to remove it for their caching headers to work properly). Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.5
[squid-users] negative_ttl & squid 3.1
Hello squid users~ I am using on squid 3.1.4 , and I need to cache about 404 code. In squid 2.7 , there have rule like "refresh_pattern .0 20% 4320 negative_ttl=xxx" When can I use negative_ttl syntax in squid 3.1 ? Thanks.
Re: [squid-users] negative_ttl
Thank you for the quick reply to my question and I have responded to some of your questions below. I also have a question about the acl that Chris sent. This use http_status which is a 3.x feature and not a 2.7 so I am trying to use the rep_header and I am not having any luck with it. I have examples below and if someone could shed some light on how I can use the "status" values 50x that would be great. > acl HTTPStatus503 http_status 503 > cache deny HTTPStatus503 acl HTTPStatus503 rep_header Status -i 503 cache deny HTTPStatus503 or acl HTTPStatus503 rep_header status -i 503 cache deny HTTPStatus503 I have tried both and I have enabled debug and they are not getting hit. So I have tethereal traces to verify the status values contain 503 adn they do. I am using a very simple perl script to generate the 503 for testing and I am only using "Cache-control: max-age=240" so its short lived. HTTP/1.1 503 Service Temporarily Unavailable Date: Tue, 22 Sep 2009 12:45:43 GMT Server: Apache/2.2.3 (Red Hat) Expires: Tue, 22 Sep 2009 07:49:43 CDT Cache-control: max-age=240 Connection: close Content-Type: text/html; charset=ISO-8859-1 Please see below for answers to other questions.. - Original Message From: Amos Jeffries To: squid-users@squid-cache.org Sent: Monday, September 21, 2009 10:18:48 PM Subject: Re: [squid-users] negative_ttl On Mon, 21 Sep 2009 17:12:44 -0800, Chris Robertson wrote: > Quin Guin wrote: >> Hi, >> >> I am seeing a behavior with the negative_ttl option and I would like to >> get confirmation on its behavior. >> >> >> I am using 2.7.Stable6 >> >> I am having an issue with a content provider that is setting the >> max_age=604800 on 503 error pages and so their 503 error pages are >> getting cached for the length expire time. > > If it's just 503's you are having trouble with... > > acl HTTPStatus503 http_status 503 > cache deny HTTPStatus503 > > ...will deny caching of any response with a 503 code. Fine tune it with > an additional dstdomain acl as needed. > >> I know that the content provider should correct this and I have >> communicated that to them several times but it gets fixed and then it >> gets set again..ugh!! So everyone saying SQUID has a bug or broke.. Set again? (a) you mean the provider is undoing their max-age fix? or (b) that the pages coming out of squid have it set that way despite the provider being correctly set at the time? Ans.. Yes provider is undoing their fixes.. I have not seen the (b) option happen. (b) is a Squid problem, probably resolved by purging the relevant URLs from cache after the provider fix happens. 2.7 does not contain bug #7 so should self-correct when that week is over. Ans.. Purging does resolve it..yes it does self correct per the age value. (a) does seem to be a issue somewhere between the provider web Server and Squid. It may be the provider themselves, or a cache between you two. Ans.. it is between the provider and SQUID then squid reply with what was given but I do think they are using a reverse proxy because I have seen errors come it. /personal opinion:: Specifying that temporary (possibly from only a single request) network failures should be reported to all visitors for a week after they occur is very excessive. IMHO the caching timeouts of 5xx should be in the order of minutes, 4xx possibly hours. Not days or weeks for either. Ans.. I agree with you 100%. >> >> I have set the "negative_ttl 0" in hopes that the negatively cached pages >> doesn't get cached at all not even for the default 5 min. This works for >> pages that don't have max_age values or very low ones.. I just want to >> confirm that this is the expected behavior for negative_ttl. This will not impact on your problem, but ... you should have that anyway. Setting it to zero disables Squids forced minimum caching time, leaving squid to follow the correct RFC-compliant behavior. Which is defined by the 4xx/5xx reply Expires: and Cache-Control: headers received, or to discard immediately if they send none. I thought 2.7 had the correct max-age handling. I suspect there may be another header or CC: value sent which impacts on the caching. 2.7.STABLE7 has a fix for re-prioritizing the stale-* CC: value, and Expires: header being present has priority over max-age. Chris Robertsons solution will get you around the problem providers headers. >> >> If so I think my next course of action in the 2.7 build line is to use >> and acl with deny on http status values? If anyone has done this and >> would like to share what they did or can point me to some docs or >> something similar I would appreciate that. >> >> >> I know 3.1 have the ability to do what I need but I am not ready to roll >> that out to production yet. >> >> Thanks, >> >> Quinguin >> > > Chris Amos
Re: [squid-users] negative_ttl
On Mon, 21 Sep 2009 17:12:44 -0800, Chris Robertson wrote: > Quin Guin wrote: >> Hi, >> >> I am seeing a behavior with the negative_ttl option and I would like to >> get confirmation on its behavior. >> >> >> I am using 2.7.Stable6 >> >> I am having an issue with a content provider that is setting the >> max_age=604800 on 503 error pages and so their 503 error pages are >> getting cached for the length expire time. > > If it's just 503's you are having trouble with... > > acl HTTPStatus503 http_status 503 > cache deny HTTPStatus503 > > ...will deny caching of any response with a 503 code. Fine tune it with > an additional dstdomain acl as needed. > >> I know that the content provider should correct this and I have >> communicated that to them several times but it gets fixed and then it >> gets set again..ugh!! So everyone saying SQUID has a bug or broke.. Set again? (a) you mean the provider is undoing their max-age fix? or (b) that the pages coming out of squid have it set that way despite the provider being correctly set at the time? (b) is a Squid problem, probably resolved by purging the relevant URLs from cache after the provider fix happens. 2.7 does not contain bug #7 so should self-correct when that week is over. (a) does seem to be a issue somewhere between the provider web Server and Squid. It may be the provider themselves, or a cache between you two. /personal opinion:: Specifying that temporary (possibly from only a single request) network failures should be reported to all visitors for a week after they occur is very excessive. IMHO the caching timeouts of 5xx should be in the order of minutes, 4xx possibly hours. Not days or weeks for either. >> >> I have set the "negative_ttl 0" in hopes that the negatively cached pages >> doesn't get cached at all not even for the default 5 min. This works for >> pages that don't have max_age values or very low ones.. I just want to >> confirm that this is the expected behavior for negative_ttl. This will not impact on your problem, but ... you should have that anyway. Setting it to zero disables Squids forced minimum caching time, leaving squid to follow the correct RFC-compliant behavior. Which is defined by the 4xx/5xx reply Expires: and Cache-Control: headers received, or to discard immediately if they send none. I thought 2.7 had the correct max-age handling. I suspect there may be another header or CC: value sent which impacts on the caching. 2.7.STABLE7 has a fix for re-prioritizing the stale-* CC: value, and Expires: header being present has priority over max-age. Chris Robertsons solution will get you around the problem providers headers. >> >> If so I think my next course of action in the 2.7 build line is to use >> and acl with deny on http status values? If anyone has done this and >> would like to share what they did or can point me to some docs or >> something similar I would appreciate that. >> >> >> I know 3.1 have the ability to do what I need but I am not ready to roll >> that out to production yet. >> >> Thanks, >> >> Quinguin >> > > Chris Amos
Re: [squid-users] negative_ttl
Quin Guin wrote: Hi, I am seeing a behavior with the negative_ttl option and I would like to get confirmation on its behavior. I am using 2.7.Stable6 I am having an issue with a content provider that is setting the max_age=604800 on 503 error pages and so their 503 error pages are getting cached for the length expire time. If it's just 503's you are having trouble with... acl HTTPStatus503 http_status 503 cache deny HTTPStatus503 ...will deny caching of any response with a 503 code. Fine tune it with an additional dstdomain acl as needed. I know that the content provider should correct this and I have communicated that to them several times but it gets fixed and then it gets set again..ugh!! So everyone saying SQUID has a bug or broke.. I have set the "negative_ttl 0" in hopes that the negatively cached pages doesn't get cached at all not even for the default 5 min. This works for pages that don't have max_age values or very low ones.. I just want to confirm that this is the expected behavior for negative_ttl. If so I think my next course of action in the 2.7 build line is to use and acl with deny on http status values? If anyone has done this and would like to share what they did or can point me to some docs or something similar I would appreciate that. I know 3.1 have the ability to do what I need but I am not ready to roll that out to production yet. Thanks, Quinguin Chris
[squid-users] negative_ttl
Hi, I am seeing a behavior with the negative_ttl option and I would like to get confirmation on its behavior. I am using 2.7.Stable6 I am having an issue with a content provider that is setting the max_age=604800 on 503 error pages and so their 503 error pages are getting cached for the length expire time. I know that the content provider should correct this and I have communicated that to them several times but it gets fixed and then it gets set again..ugh!! So everyone saying SQUID has a bug or broke.. I have set the "negative_ttl 0" in hopes that the negatively cached pages doesn't get cached at all not even for the default 5 min. This works for pages that don't have max_age values or very low ones.. I just want to confirm that this is the expected behavior for negative_ttl. If so I think my next course of action in the 2.7 build line is to use and acl with deny on http status values? If anyone has done this and would like to share what they did or can point me to some docs or something similar I would appreciate that. I know 3.1 have the ability to do what I need but I am not ready to roll that out to production yet. Thanks, Quinguin
Re: [squid-users] negative_ttl vs. an Expires header -- which should win?
Amos Jeffries wrote: negative_ttl wins. That seems a violation of the intent of section 13.4 of rfc2616, specifically the "unless" clause of the following sentence which applies to "any other status code" (despite the limitation of examples to 302 and 307): # A response received with any other status code (e.g. status codes 302 # and 307) MUST NOT be returned in a reply to a subsequent request # unless there are cache-control directives or another header(s) that # explicitly allow it. For example, these include the following: an # Expires header (section 14.21); a "max-age", "s-maxage", "must- # revalidate", "proxy-revalidate", "public" or "private" cache-control # directive (section 14.9). Or am I missing some other relevant section? It should be set to "0 seconds" in any case to retain HTTP standards. This has been fixed in recent squid releases, though older squid contains a bad default of more than 0. Yes, "0 seconds" is required by HTTP/1.1 in the absence of other headers. However, I think the above implies 'Expires' and other caching directive headers allow any response to be cacheable for a specified time. - Gordon @ IA Mark Nottingham wrote: What version of Squid are you using? This changed somewhat in 2.7; IIRC in 2.6 negative_ttl overrides response freshness, whereas in 2.7 response freshness (i.e., expires or cache-control) has precedence. Cheers, On 02/10/2008, at 3:56 PM, Gordon Mohr wrote: Using 2.6.14-1ubuntu2 in an reverse/accelerator setup. My backend/parent is by design setting explicit 'Expires' headers 1 day into the future, even on 404/403/302 response codes. I'm seeing the 4XX responses later served as TCP_NEGATIVE_HITs, which is good. It appears, from my testing, that they are sometimes cached a bit longer than 'negative_ttl', but they are not cached as long as the Expires header suggests, even with plentiful cache space. What is the designed intent of Squid -- should the 'negative_ttl' or the Expires header be definitive? - Gordon @ IA -- Mark Nottingham [EMAIL PROTECTED] negative_ttl wins. It should be set to "0 seconds" in any case to retain HTTP standards. This has been fixed in recent squid releases, though older squid contains a bad default of more than 0. Amos
Re: [squid-users] negative_ttl vs. an Expires header -- which should win?
Mark Nottingham wrote: What version of Squid are you using? This changed somewhat in 2.7; IIRC in 2.6 negative_ttl overrides response freshness, whereas in 2.7 response freshness (i.e., expires or cache-control) has precedence. Cheers, On 02/10/2008, at 3:56 PM, Gordon Mohr wrote: Using 2.6.14-1ubuntu2 in an reverse/accelerator setup. My backend/parent is by design setting explicit 'Expires' headers 1 day into the future, even on 404/403/302 response codes. I'm seeing the 4XX responses later served as TCP_NEGATIVE_HITs, which is good. It appears, from my testing, that they are sometimes cached a bit longer than 'negative_ttl', but they are not cached as long as the Expires header suggests, even with plentiful cache space. What is the designed intent of Squid -- should the 'negative_ttl' or the Expires header be definitive? - Gordon @ IA -- Mark Nottingham [EMAIL PROTECTED] negative_ttl wins. It should be set to "0 seconds" in any case to retain HTTP standards. This has been fixed in recent squid releases, though older squid contains a bad default of more than 0. Amos -- Please use Squid 2.7.STABLE4 or 3.0.STABLE9
Re: [squid-users] negative_ttl vs. an Expires header -- which should win?
What version of Squid are you using? This changed somewhat in 2.7; IIRC in 2.6 negative_ttl overrides response freshness, whereas in 2.7 response freshness (i.e., expires or cache-control) has precedence. Cheers, On 02/10/2008, at 3:56 PM, Gordon Mohr wrote: Using 2.6.14-1ubuntu2 in an reverse/accelerator setup. My backend/parent is by design setting explicit 'Expires' headers 1 day into the future, even on 404/403/302 response codes. I'm seeing the 4XX responses later served as TCP_NEGATIVE_HITs, which is good. It appears, from my testing, that they are sometimes cached a bit longer than 'negative_ttl', but they are not cached as long as the Expires header suggests, even with plentiful cache space. What is the designed intent of Squid -- should the 'negative_ttl' or the Expires header be definitive? - Gordon @ IA -- Mark Nottingham [EMAIL PROTECTED]
[squid-users] negative_ttl vs. an Expires header -- which should win?
Using 2.6.14-1ubuntu2 in an reverse/accelerator setup. My backend/parent is by design setting explicit 'Expires' headers 1 day into the future, even on 404/403/302 response codes. I'm seeing the 4XX responses later served as TCP_NEGATIVE_HITs, which is good. It appears, from my testing, that they are sometimes cached a bit longer than 'negative_ttl', but they are not cached as long as the Expires header suggests, even with plentiful cache space. What is the designed intent of Squid -- should the 'negative_ttl' or the Expires header be definitive? - Gordon @ IA
[squid-users] negative_ttl and HTTP/1.1 compliance
AFAICT having negative_ttl > 0 is a violation of HTTP/1.1 section 13.4 (last two paragraphs). If this is the case, I should set negative_ttl=0 when Squid is used as a general-purpose proxy server. I know that Squid doesn't claim full HTTP/1.1 support, but many clients nowadays use HTTP/1.1 would benefit from having Squid being to support so much of HTTP/1.1 as possible. Do you agree with this? http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html#sec13.4 Thanks, Christian
