Re: [squid-users] read_timeout and fwdServerClosed: re-forwarding
: So it kind of seems like i'm out of luck right? my only option being to : try 2.HEAD which *may* have the behavior i'm describing. : : Its part and parcel with free software. We can be paid to test it in a lab : and give you a certain answer if you'd like. :) Oh, believe me -- I know the score ... It seems like I chant the mantra of patches welcome! :) at least once a week over on the apache lists ... i just wanted to make sure i wasn't missing anything obvious about how to achieve this sort of thing with the current STABLE releases. I'll try to find some time to test out the HEAD and report back my findings (although being out sick for a week and a half has put me pretty far behind on some other work, so i'm not sure if i'll ever get to that) but i'll also open a bug to track that 10 retries is hardcoded in the event of read_timeout. (even if it never gets changed, at least it will be be out there for other people to find it) Thanks for all your help everybody. -Hoss
Re: [squid-users] read_timeout and fwdServerClosed: re-forwarding
sorry for the late reply, i was seriously sick last week and basically dead to the world... : The problem I'm running into is figuring out a way to get the analogous : behavior when the origin server is up but taking too long to respond : to the validation requests. Ideally (in my mind) squid would have a : Hmm.. might be a good idea to try Squid-2.HEAD. This kind of things : behaves a little differently there than 2.6.. Alas ... I don't think i could convince my boss to get on board the idea of using a devel releases. then again, i'm not too clear on how branch/release management is done in squid ... do merges happen from 2.HEAD to 2.6 (in which case does 2.6.STABLE17 have the behavior you are refering to?) or will 2.HEAD ultimately become 2.7 once it's more stable? : read_timeout was the only option I could find that seemed to relate to : how long squid would wait for an origin server once connected -- but it : has the retry problems previously discussed. Even if it didn't retry, and : returned the stale content as soon as the read_timeout was exceeded, : I'm guessing it wouldn't wait for the fresh response from the origin : server to cache it for future requests. : : read_timeout in combination with forward_timeout should take care of the : timeout part... what do you mean by in combination with forward_timeout ... forward_timeout is just the 'connect' timeout for origin server requests right? so i guess you mean that if i have a magic value of XX seconds that i'm willing to wait for data to come back, that i need to set fowrad_timeout and read_timeout such that they add up to XX right? but as you say, that just solves the tieout problem, it doesn't get me stale content. In my case, i'm not worried about the connect time for the origin server -- if it doesn't connect right away give up, not problem there. it's getting stale content to be returned if the total request time excedes XX seconds that i'm worried about (without getting a bunch of automatic retries) So it kind of seems like i'm out of luck right? my only option being to try 2.HEAD which *may* have the behavior i'm describing. : for a fresh response) -- but it doesn't seem to work as advertised (see : bug#2126). : : Haven't looked at that report yet.. but a guess is that the refresh : failed due to read_timeout? (actually that was totally orthoginal to the read_timeout issues ... with refresh_stale_hit set to Y seconds, all requets are still considered cache hits up to Y seconds afer they expire -- with no attempt to validate.) -Hoss
Re: [squid-users] read_timeout and fwdServerClosed: re-forwarding
On Wed, Dec 05, 2007, Chris Hostetter wrote: : Hmm.. might be a good idea to try Squid-2.HEAD. This kind of things : behaves a little differently there than 2.6.. Alas ... I don't think i could convince my boss to get on board the idea of using a devel releases. then again, i'm not too clear on how branch/release management is done in squid ... do merges happen from 2.HEAD to 2.6 (in which case does 2.6.STABLE17 have the behavior you are refering to?) or will 2.HEAD ultimately become 2.7 once it's more stable? Squid-2.HEAD should eventually become Squid-2.7. So it kind of seems like i'm out of luck right? my only option being to try 2.HEAD which *may* have the behavior i'm describing. Its part and parcel with free software. We can be paid to test it in a lab and give you a certain answer if you'd like. :) -- - Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support -
Re: [squid-users] read_timeout and fwdServerClosed: re-forwarding
On mån, 2007-11-26 at 14:48 -0800, Chris Hostetter wrote: But in situations like that, wouldn't the normal behavior of a long read_timeout (I believe the default is 15 minutes) be sufficient? yes, and it is.. the retries is only done if within the forward_timeout : Hm, what about retry_on_error ? Does that do anything in an accelerator : setup? It might do something, but I'm not sure what :) ... even when i set it explicitly to off squid still retries when the read_timeout is exceeded. It applies when Squid received an error from the contacted server only... In the event that a request is not in the cache at all, and an origin server takes too long to send a response, using the quick_abort 0 option in squid does exactly what I hoped it would: squid continues to wait around for the response so that it is available in the cache for future requests. good. In the event that stale content is already in the cache, and the origin server is down and won't accept any connections, squid does what I'd hoped it would: returns the stale content even though it can't be validated (albeit, without a proper warning, see bug#2119) good. The problem I'm running into is figuring out a way to get the analogous behavior when the origin server is up but taking too long to respond to the validation requests. Ideally (in my mind) squid would have a force_stale_response_after XX milliseconds option, such that if squid has a stale response available in the cache, it will return immediately once XX milliseconds have elapsed since the client connected. Any in progress validation requests would still be completed/cached if they met the conditions of the quick_abort option just as if the client had aborted the connection without receiving any response. Hmm.. might be a good idea to try Squid-2.HEAD. This kind of things behaves a little differently there than 2.6.. read_timeout was the only option I could find that seemed to relate to how long squid would wait for an origin server once connected -- but it has the retry problems previously discussed. Even if it didn't retry, and returned the stale content as soon as the read_timeout was exceeded, I'm guessing it wouldn't wait for the fresh response from the origin server to cache it for future requests. read_timeout in combination with forward_timeout should take care of the timeout part... FWIW: The refresh_stale_hit option seemed like a promising mechanism for ensuring that when concurrent requests come in, all but one would get a stale response while waiting for a fresh response to be cached (which could help minimize the number of clients that give up while waiting for a fresh response) -- but it doesn't seem to work as advertised (see bug#2126). Haven't looked at that report yet.. but a guess is that the refresh failed due to read_timeout? Regards Henrik signature.asc Description: This is a digitally signed message part
Re: [squid-users] read_timeout and fwdServerClosed: re-forwarding
: Tip: fwdReforwardableStatus() I think is the function which implements : the behaviour you're seeing. That and fwdCheckRetry. My C Fu isn't strong enough for me to feel confident that I would even know what to look for if I started digging into the code ... I mainly just wanted to clarify that: a) this is expected behavior b) there isn't a(n existing) config option available to change this behavior : You could set the HTTP Gateway timeout to return 0 so the request : isn't forwarded and see if that works, or the n_tries check in fwdCheckRetry(). I'm not sure I understand ... are you saying there is a squid option to set an explicit gateway timeout value? (such that origin requests which take longer then X cause squid to return a 504 to the client) ... This would ideal -- the only reason I was even experimenting with read_timeout was because I haven't found any documentation of anything like this. (but since the servers I'm dealing with don't write anything until the entire response is ready I figured I could make do with the read_timeout) : I could easily make the 10 retry count a configurable parameter. That might be prudent. It seems like strange behavior to have hardcoded in squid. : The feature, IIRC, was to work around transient network issues which : would bring up error pages in a traditional forward-proxying setup. But in situations like that, wouldn't the normal behavior of a long read_timeout (I believe the default is 15 minutes) be sufficient? : Hm, what about retry_on_error ? Does that do anything in an accelerator : setup? It might do something, but I'm not sure what :) ... even when i set it explicitly to off squid still retries when the read_timeout is exceeded. Perhaps I'm approaching things the wrong way -- I set out with some specific goals in mind, did some experimenting with various options to try and reach that goal, and then asked questions when i encountered behavior I couldn't explain. Let me back up and describe my goals, and perhaps someone can offer some insight into the appropriate way to achieve them I'm the middle man between origin servers which respond to every request by dynamicly generating (relatively small) responses; and clients that make GET requests to these servers but are only willing to wait around for a short amount of time (on the order of 100s of milliseconds) to get the responses before they abort the connection. The clients would rather get no response (or an error) then wait around for a long time -- the servers meanwhile would rather the clients got stale responses then no responses (or error responses). My goal, using squid as an accelerator, is to maximize the satisfaction of both the clients and the servers. In the event that a request is not in the cache at all, and an origin server takes too long to send a response, using the quick_abort 0 option in squid does exactly what I hoped it would: squid continues to wait around for the response so that it is available in the cache for future requests. In the event that stale content is already in the cache, and the origin server is down and won't accept any connections, squid does what I'd hoped it would: returns the stale content even though it can't be validated (albeit, without a proper warning, see bug#2119) The problem I'm running into is figuring out a way to get the analogous behavior when the origin server is up but taking too long to respond to the validation requests. Ideally (in my mind) squid would have a force_stale_response_after XX milliseconds option, such that if squid has a stale response available in the cache, it will return immediately once XX milliseconds have elapsed since the client connected. Any in progress validation requests would still be completed/cached if they met the conditions of the quick_abort option just as if the client had aborted the connection without receiving any response. Is there a way to get behavior like this (or close to it) from squid? read_timeout was the only option I could find that seemed to relate to how long squid would wait for an origin server once connected -- but it has the retry problems previously discussed. Even if it didn't retry, and returned the stale content as soon as the read_timeout was exceeded, I'm guessing it wouldn't wait for the fresh response from the origin server to cache it for future requests. FWIW: The refresh_stale_hit option seemed like a promising mechanism for ensuring that when concurrent requests come in, all but one would get a stale response while waiting for a fresh response to be cached (which could help minimize the number of clients that give up while waiting for a fresh response) -- but it doesn't seem to work as advertised (see bug#2126). -Hoss
Re: [squid-users] read_timeout and fwdServerClosed: re-forwarding
On Wed, Nov 21, 2007, Chris Hostetter wrote: Greetings, I'm trying to make sense of some behavior I'm observing related to the read_timeout. Tip: fwdReforwardableStatus() I think is the function which implements the behaviour you're seeing. That and fwdCheckRetry. You could set the HTTP Gateway timeout to return 0 so the request isn't forwarded and see if that works, or the n_tries check in fwdCheckRetry(). I could easily make the 10 retry count a configurable parameter. The feature, IIRC, was to work around transient network issues which would bring up error pages in a traditional forward-proxying setup. I'm dealing with an accelerator setup, where I'd rather return stale content (or an error) then wait for the origin server to return fresh content if it is taking too long to respond. Hm, what about retry_on_error ? Does that do anything in an accelerator setup? Adrian -- - Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support -