Re: [MERGE] Squid Patch (revision 10487)
On 2014-05-13 11:53, Amos Jeffries wrote: On 13/05/2014 10:52 p.m., Amos Jeffries wrote: On 13/05/2014 4:56 a.m., Michael Pye wrote: On 2014-05-12 16:52, Amos Jeffries wrote: Here is the guts of the original patch as should be relevant to Squid-3.5 (or 3.4). http://master.squid-cache.org/~amosjeffries/patches/Pye_s35.patch Many thanks Amos, I've just tried this and it does indeed query sibling for stale content as we want. I think there is some logic that will need updating however in HttpHeader.cc possibly, as when an object becomes stale and the sibling does not have a fresh copy, the origin is queried but the "Age:" and "X-Cache:HIT from" headers are still on the new request, even though it was a fresh check from the origin. I think these headers need to be deleted/updated. Is this a fully new set of headers+object from the server? or a revalidation update/refresh from the server? On the former Age and X-Cache are correct because it is "just" a fresher update of the initial cache HIT object. Oops I meant s/former/latter/. Sorry - reread what you said. So if its a revalidation and the content hasn't changed (304 from origin), under normal squid *without* this patch, Age will return to 0 and count up to the max-age again before doing another revalidation. A max-age 60 second object will remain in cache for 60 seconds before revalidation. With the patch applied, if squidA gets a copy of the content from siblingB where Age already has X seconds set, when that object is next revalidated against the origin (now siblingB copy has expired), its Age remains at X seconds, counting up to max-age. The max-age 60 object is now revalidating against the origin every 30 seconds, I would have expected it to return to validating every 60 seconds. Thanks Michael
Re: [MERGE] Squid Patch (revision 10487)
On 2014-05-13 11:53, Amos Jeffries wrote: On 13/05/2014 10:52 p.m., Amos Jeffries wrote: On 13/05/2014 4:56 a.m., Michael Pye wrote: On 2014-05-12 16:52, Amos Jeffries wrote: Here is the guts of the original patch as should be relevant to Squid-3.5 (or 3.4). http://master.squid-cache.org/~amosjeffries/patches/Pye_s35.patch Many thanks Amos, I've just tried this and it does indeed query sibling for stale content as we want. I think there is some logic that will need updating however in HttpHeader.cc possibly, as when an object becomes stale and the sibling does not have a fresh copy, the origin is queried but the "Age:" and "X-Cache:HIT from" headers are still on the new request, even though it was a fresh check from the origin. I think these headers need to be deleted/updated. Is this a fully new set of headers+object from the server? or a revalidation update/refresh from the server? It's a revalidation, so it sends an "if-modified-since" to the origin which replies with a "304 not modified". At this point it seems the Age: needs recalculating. Thanks Michael
Re: [MERGE] Squid Patch (revision 10487)
On 13/05/2014 10:52 p.m., Amos Jeffries wrote: > On 13/05/2014 4:56 a.m., Michael Pye wrote: >> On 2014-05-12 16:52, Amos Jeffries wrote: >>> Here is the guts of the original patch as should be relevant to >>> Squid-3.5 (or 3.4). >>> http://master.squid-cache.org/~amosjeffries/patches/Pye_s35.patch >> >> Many thanks Amos, I've just tried this and it does indeed query sibling >> for stale content as we want. >> >> I think there is some logic that will need updating however in >> HttpHeader.cc possibly, as when an object becomes stale and the sibling >> does not have a fresh copy, the origin is queried but the "Age:" and >> "X-Cache:HIT from" headers are still on the new request, even though it >> was a fresh check from the origin. I think these headers need to be >> deleted/updated. > > Is this a fully new set of headers+object from the server? > or a revalidation update/refresh from the server? > > On the former Age and X-Cache are correct because it is "just" a fresher > update of the initial cache HIT object. Oops I meant s/former/latter/. Amos
Re: [MERGE] Squid Patch (revision 10487)
On 13/05/2014 4:56 a.m., Michael Pye wrote: > On 2014-05-12 16:52, Amos Jeffries wrote: >> Here is the guts of the original patch as should be relevant to >> Squid-3.5 (or 3.4). >> http://master.squid-cache.org/~amosjeffries/patches/Pye_s35.patch > > Many thanks Amos, I've just tried this and it does indeed query sibling > for stale content as we want. > > I think there is some logic that will need updating however in > HttpHeader.cc possibly, as when an object becomes stale and the sibling > does not have a fresh copy, the origin is queried but the "Age:" and > "X-Cache:HIT from" headers are still on the new request, even though it > was a fresh check from the origin. I think these headers need to be > deleted/updated. Is this a fully new set of headers+object from the server? or a revalidation update/refresh from the server? On the former Age and X-Cache are correct because it is "just" a fresher update of the initial cache HIT object. Amos
Re: [MERGE] Squid Patch (revision 10487)
On 2014-05-12 16:52, Amos Jeffries wrote: Here is the guts of the original patch as should be relevant to Squid-3.5 (or 3.4). http://master.squid-cache.org/~amosjeffries/patches/Pye_s35.patch Many thanks Amos, I've just tried this and it does indeed query sibling for stale content as we want. I think there is some logic that will need updating however in HttpHeader.cc possibly, as when an object becomes stale and the sibling does not have a fresh copy, the origin is queried but the "Age:" and "X-Cache:HIT from" headers are still on the new request, even though it was a fresh check from the origin. I think these headers need to be deleted/updated. Thanks again Michael
Re: [MERGE] Squid Patch (revision 10487)
On 13/05/2014 2:25 a.m., Michael Pye wrote: > On 2014-05-11 18:40, Amos Jeffries wrote: >> Not as yet. Henrik pointed out a problem that needed testing: >> http://www.squid-cache.org/mail-archive/squid-dev/201005/0156.html >> >> I dont think we send only-if-cached any longer. >> >> Can you confirm that, or answer Henriks question? > > Sorry - I can't confirm that, I don't know the internals well enough for > that. The check is whether the HTTP requests sent to siblings contain Cache-Control:only-if-cached. Or if things break when the client sends that header. > > I am still interested in getting this patch to be either included or > tested however. If I can help on testing this please let me know. > > I have compiled a version with the patch on squid 3.4.5 and it appears > to do what we want, however it seems to have a bug. When a stale object > needs updating, it does query the sibling, and if it has a fresher copy > it is returned. If however the sibling does not have a fresher copy the > original squid queries the origin, but seems to keep the "Age:" header > from the last stale request, so although it has updated the object it > will expire quicker than it should as it has an incorrectly calculated > "Age:" header on it. I assume this is due to the patch introducing > something new that is not being considered when refreshing from a > sibling. Anyone interested in helping on this ? Here is the guts of the original patch as should be relevant to Squid-3.5 (or 3.4). http://master.squid-cache.org/~amosjeffries/patches/Pye_s35.patch Cheers Amos
Re: [MERGE] Squid Patch (revision 10487)
On 2014-05-11 18:40, Amos Jeffries wrote: Not as yet. Henrik pointed out a problem that needed testing: http://www.squid-cache.org/mail-archive/squid-dev/201005/0156.html I dont think we send only-if-cached any longer. Can you confirm that, or answer Henriks question? Sorry - I can't confirm that, I don't know the internals well enough for that. I am still interested in getting this patch to be either included or tested however. If I can help on testing this please let me know. I have compiled a version with the patch on squid 3.4.5 and it appears to do what we want, however it seems to have a bug. When a stale object needs updating, it does query the sibling, and if it has a fresher copy it is returned. If however the sibling does not have a fresher copy the original squid queries the origin, but seems to keep the "Age:" header from the last stale request, so although it has updated the object it will expire quicker than it should as it has an incorrectly calculated "Age:" header on it. I assume this is due to the patch introducing something new that is not being considered when refreshing from a sibling. Anyone interested in helping on this ? Many thanks Michael
Re: [MERGE] Squid Patch (revision 10487)
On 12/05/2014 4:39 a.m., Michael Pye wrote:> On 24/05/2010 19:17, senad.cimic wrote: >> On 05/23/2010 09:52 PM, Mark Nottingham wrote: >>> How about: >>> >>> When Squid has a stale cached response available, it will not >>> query siblings, by default. When this setting is on Squid will >>> do so, as long as the cache_peer line for the sibling does not >>> have the 'allow-miss' option configured. > > Did this patch for "access_sibling_for_stale_resource" ever get added ? > I would like this functionality so a sibling is checked for stale > resources first rather than parent/originserver. > > Thanks > Michael > Not as yet. Henrik pointed out a problem that needed testing: http://www.squid-cache.org/mail-archive/squid-dev/201005/0156.html I dont think we send only-if-cached any longer. Can you confirm that, or answer Henriks question? Amos
Re: [MERGE] Squid Patch (revision 10487)
On 24/05/2010 19:17, senad.ci...@thomsonreuters.com wrote: On 05/23/2010 09:52 PM, Mark Nottingham wrote: How about: When Squid has a stale cached response available, it will not query siblings, by default. When this setting is on Squid will do so, as long as the cache_peer line for the sibling does not have the 'allow-miss' option configured. Did this patch for "access_sibling_for_stale_resource" ever get added ? I would like this functionality so a sibling is checked for stale resources first rather than parent/originserver. Thanks Michael
Re: [MERGE] Squid Patch (revision 10487)
senad.ci...@thomsonreuters.com wrote: Modified patch file is attached. I committed those documentation changes as well. -Senad +1. Looks okay now. Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.3
RE: [MERGE] Squid Patch (revision 10487)
Modified patch file is attached. I committed those documentation changes as well. -Senad -Original Message- From: Cimic, Senad (Legal) Sent: Monday, May 24, 2010 9:31 AM To: rouss...@measurement-factory.com; m...@yahoo-inc.com Cc: squid-dev@squid-cache.org Subject: RE: [MERGE] Squid Patch (revision 10487) Sounds good, I'll produce patch for proposed documentation changes. Thanks, Senad -Original Message- From: Alex Rousskov [mailto:rouss...@measurement-factory.com] Sent: Sunday, May 23, 2010 11:09 PM To: Mark Nottingham Cc: Cimic, Senad (Legal); squid-dev@squid-cache.org Subject: Re: [MERGE] Squid Patch (revision 10487) On 05/23/2010 09:52 PM, Mark Nottingham wrote: > How about: > > When Squid has a stale cached response available, it will not > query siblings, by default. When this setting is on Squid will > do so, as long as the cache_peer line for the sibling does not > have the 'allow-miss' option configured. > > ? Sounds better to me. Thank you, Alex. > On 23/05/2010, at 9:42 AM, Alex Rousskov wrote: > >> On 05/19/2010 08:00 PM, Mark Nottingham wrote: >>> Minor suggestions for the documentation: >>> >>> +NAME: access_sibling_when_stale >>> +COMMENT: on|off >>> +TYPE: onoff >>> +DEFAULT: off >>> +LOC: Config.onoff.access_sibling_for_stale_resource >>> +DOC_START >>> + By default, Squid will not contact siblings when it has >>> + a stale cached response available. If on, siblings >>> + that do not have the 'allow-miss' cache_peer option will >>> + be queried even when there is a stale cached response. >>> +DOC_END >> It is still not clear to an uninitiated reader whether "this" Squid or >> the sibling proxy has the stale response. "it has" and "there is" should >> be replaced with something more specific, IMHO. >> >> HTH, >> >> Alex. > > -- > Mark Nottingham m...@yahoo-inc.com > access_sibling_for_stale_resource.patch Description: access_sibling_for_stale_resource.patch
RE: [MERGE] Squid Patch (revision 10487)
Sounds good, I'll produce patch for proposed documentation changes. Thanks, Senad -Original Message- From: Alex Rousskov [mailto:rouss...@measurement-factory.com] Sent: Sunday, May 23, 2010 11:09 PM To: Mark Nottingham Cc: Cimic, Senad (Legal); squid-dev@squid-cache.org Subject: Re: [MERGE] Squid Patch (revision 10487) On 05/23/2010 09:52 PM, Mark Nottingham wrote: > How about: > > When Squid has a stale cached response available, it will not > query siblings, by default. When this setting is on Squid will > do so, as long as the cache_peer line for the sibling does not > have the 'allow-miss' option configured. > > ? Sounds better to me. Thank you, Alex. > On 23/05/2010, at 9:42 AM, Alex Rousskov wrote: > >> On 05/19/2010 08:00 PM, Mark Nottingham wrote: >>> Minor suggestions for the documentation: >>> >>> +NAME: access_sibling_when_stale >>> +COMMENT: on|off >>> +TYPE: onoff >>> +DEFAULT: off >>> +LOC: Config.onoff.access_sibling_for_stale_resource >>> +DOC_START >>> + By default, Squid will not contact siblings when it has >>> + a stale cached response available. If on, siblings >>> + that do not have the 'allow-miss' cache_peer option will >>> + be queried even when there is a stale cached response. >>> +DOC_END >> It is still not clear to an uninitiated reader whether "this" Squid or >> the sibling proxy has the stale response. "it has" and "there is" should >> be replaced with something more specific, IMHO. >> >> HTH, >> >> Alex. > > -- > Mark Nottingham m...@yahoo-inc.com >
Re: [MERGE] Squid Patch (revision 10487)
On 05/23/2010 09:52 PM, Mark Nottingham wrote: > How about: > > When Squid has a stale cached response available, it will not > query siblings, by default. When this setting is on Squid will > do so, as long as the cache_peer line for the sibling does not > have the 'allow-miss' option configured. > > ? Sounds better to me. Thank you, Alex. > On 23/05/2010, at 9:42 AM, Alex Rousskov wrote: > >> On 05/19/2010 08:00 PM, Mark Nottingham wrote: >>> Minor suggestions for the documentation: >>> >>> +NAME: access_sibling_when_stale >>> +COMMENT: on|off >>> +TYPE: onoff >>> +DEFAULT: off >>> +LOC: Config.onoff.access_sibling_for_stale_resource >>> +DOC_START >>> + By default, Squid will not contact siblings when it has >>> + a stale cached response available. If on, siblings >>> + that do not have the 'allow-miss' cache_peer option will >>> + be queried even when there is a stale cached response. >>> +DOC_END >> It is still not clear to an uninitiated reader whether "this" Squid or >> the sibling proxy has the stale response. "it has" and "there is" should >> be replaced with something more specific, IMHO. >> >> HTH, >> >> Alex. > > -- > Mark Nottingham m...@yahoo-inc.com >
Re: [MERGE] Squid Patch (revision 10487)
How about: When Squid has a stale cached response available, it will not query siblings, by default. When this setting is on Squid will do so, as long as the cache_peer line for the sibling does not have the 'allow-miss' option configured. ? On 23/05/2010, at 9:42 AM, Alex Rousskov wrote: > On 05/19/2010 08:00 PM, Mark Nottingham wrote: >> Minor suggestions for the documentation: >> >> +NAME: access_sibling_when_stale >> +COMMENT: on|off >> +TYPE: onoff >> +DEFAULT: off >> +LOC: Config.onoff.access_sibling_for_stale_resource >> +DOC_START >> +By default, Squid will not contact siblings when it has >> +a stale cached response available. If on, siblings >> +that do not have the 'allow-miss' cache_peer option will >> +be queried even when there is a stale cached response. >> +DOC_END > > It is still not clear to an uninitiated reader whether "this" Squid or > the sibling proxy has the stale response. "it has" and "there is" should > be replaced with something more specific, IMHO. > > HTH, > > Alex. -- Mark Nottingham m...@yahoo-inc.com
Re: [MERGE] Squid Patch (revision 10487)
On 05/19/2010 08:00 PM, Mark Nottingham wrote: > Minor suggestions for the documentation: > > +NAME: access_sibling_when_stale > +COMMENT: on|off > +TYPE: onoff > +DEFAULT: off > +LOC: Config.onoff.access_sibling_for_stale_resource > +DOC_START > + By default, Squid will not contact siblings when it has > + a stale cached response available. If on, siblings > + that do not have the 'allow-miss' cache_peer option will > + be queried even when there is a stale cached response. > +DOC_END It is still not clear to an uninitiated reader whether "this" Squid or the sibling proxy has the stale response. "it has" and "there is" should be replaced with something more specific, IMHO. HTH, Alex.
Re: [MERGE] Squid Patch (revision 10487)
tor 2010-05-20 klockan 12:00 +1000 skrev Mark Nottingham: > H, is setting the default to on a bad thing? Probably not, as the code takes care of allow-miss. related to this the behavior when the sibling rejects the request need to be verified. This change will make only-if-cached rejections in sibling relations much more frequent than today. Regards Henrik
Re: [MERGE] Squid Patch (revision 10487)
Minor suggestions for the documentation: +NAME: access_sibling_when_stale +COMMENT: on|off +TYPE: onoff +DEFAULT: off +LOC: Config.onoff.access_sibling_for_stale_resource +DOC_START + By default, Squid will not contact siblings when it has + a stale cached response available. If on, siblings + that do not have the 'allow-miss' cache_peer option will + be queried even when there is a stale cached response. +DOC_END H, is setting the default to on a bad thing? I'll produce the patch for 2.HEAD. Regards, -- Mark Nottingham m...@yahoo-inc.com
RE: [MERGE] Squid Patch (revision 10487)
On Wed, 19 May 2010 10:03:11 -0500, wrote: > Not sure what went wrong, but since changes in each of the 3 files are > minimal I included differences as attachments (new version for each file is > on top, old version is on bottom): > > neighbor.cc_diff.jpg - shows changes in neighbor.cc file in the > peerAllowedToUse method. > structs.h_diff.jpg - shows a single variable that has been added to onoff > struct in the structs.h file. > cf.data.pre_diff.jpg - shows info that has been added for a new > access_sibling_for_stale_resource directive in cf.data.pre. > Okay. Thats more readable at least. +1 provided this has some testing behind it. > Let me know what you think. I can try couple different things with Bazaar > to generate correct patch file if needed. Not sure beyond the ones I use daily ...to see unsaved changes: bzr diff ...to see changes in a branch back to its parent (ie trunk): bzr diff -rsubmit: I suspect there might be something like whitespacing or \n \r differences in your OS or editor from the stored type the rest of us use. Are you using Eclipse or Emacs for an editor? they are known to change end-of-line characters by default sometimes. Amos > > Thanks in advance, > Senad > > -Original Message- > From: Amos Jeffries [mailto:squ...@treenet.co.nz] > Sent: Tuesday, May 18, 2010 6:55 PM > To: squid-dev@squid-cache.org > Subject: Re: [MERGE] Squid Patch (revision 10487) > > On Tue, 18 May 2010 16:25:31 -0500, > wrote: >> Attached file is generated patch for the issue where squid does not >> contact its siblings for the stale resources. New directive has been >> added to make this behavior optional. >> >> Let me know if any issues. > > Something has gone wrong in the patch creation or formatting process. Each > file that you have altered even slightly is removed completely and replaced > with a whole new version. This makes it extremely hard to tell what changes > you have made. > > Amos
Re: [MERGE] Squid Patch (revision 10487)
tis 2010-05-18 klockan 16:25 -0500 skrev senad.ci...@thomsonreuters.com: > Attached file is generated patch for the issue where squid does not > contact its siblings for the stale resources. New directive has been > added to make this behavior optional. Corrected patch attached showing only your changes. Regards Henrik === modified file 'src/cf.data.pre' --- src/cf.data.pre 2010-05-14 05:37:19 + +++ src/cf.data.pre 2010-05-19 18:29:28 + @@ -6941,4 +6941,15 @@ Whether to lookup the EUI or MAC address of a connected client. DOC_END +NAME: access_sibling_for_stale_resource +COMMENT: on|off +TYPE: onoff +DEFAULT: off +LOC: Config.onoff.access_sibling_for_stale_resource +DOC_START + By default, Squid will not contact siblings for cached but + expired (stale) resources. If this directive is set to on, + that behavior will change unless sibling has allow-miss option set. +DOC_END + EOF === modified file 'src/neighbors.cc' --- src/neighbors.cc 2010-05-02 19:32:42 + +++ src/neighbors.cc 2010-05-19 18:29:28 + @@ -156,13 +156,19 @@ if (request->flags.nocache) return 0; -if (request->flags.refresh) +/*Ignore refresh flag if access_sibling_for_stale_resource flag is on unless allow_miss is enabled for this peer + (needed to avoid forwarding loops).*/ +if (request->flags.refresh && !Config.onoff.access_sibling_for_stale_resource || +request->flags.refresh && p->options.allow_miss) return 0; if (request->flags.loopdetect) return 0; -if (request->flags.need_validation) +/*Ignore need_validation flag if access_sibling_for_stale_resource flag is on unless allow_miss is enabled for this peer + (needed to avoid forwarding loops).*/ +if (request->flags.need_validation && !Config.onoff.access_sibling_for_stale_resource || +request->flags.need_validation && p->options.allow_miss) return 0; } === modified file 'src/structs.h' --- src/structs.h 2010-04-17 02:29:04 + +++ src/structs.h 2010-05-19 18:29:28 + @@ -437,6 +437,7 @@ int WIN32_IpAddrChangeMonitor; int memory_cache_first; int memory_cache_disk; +int access_sibling_for_stale_resource; } onoff; int forward_max_tries;
RE: [MERGE] Squid Patch (revision 10487)
ons 2010-05-19 klockan 10:03 -0500 skrev senad.ci...@thomsonreuters.com: > Not sure what went wrong, but since changes in each of the 3 files are > minimal I included differences as attachments (new version for each > file is on top, old version is on bottom): Probably got converted from UNIX to DOS line endings. Regards Henrik
Re: [MERGE] Squid Patch (revision 10487)
On Tue, 18 May 2010 16:25:31 -0500, wrote: > Attached file is generated patch for the issue where squid does not > contact its siblings for the stale resources. New directive has been > added to make this behavior optional. > > Let me know if any issues. Something has gone wrong in the patch creation or formatting process. Each file that you have altered even slightly is removed completely and replaced with a whole new version. This makes it extremely hard to tell what changes you have made. Amos
