So I have some more information about this, but this time related to having keepalives OFF in mod_proxy.
I tried using the "SetEnv proxy-nokeepalive 1" option in httpd.conf, and it cleared up the proxy errors that I was having with an IIS backend server, and it may have decreased the proxy errors with Apache backend servers as well. I tried again without the proxy-nokeepalive option, and the proxy errors increased. Put it back, they decreased dramatically again. Turns out that mod_proxy always sends "Connection: Keep-Alive", but with "proxy-nokeepalive 1", mod_proxy will always send "Connection: Close". The backends in my case are all responding with "Connection: Close", so the backends aren't even allowing Keep-Alives, regardless of whether mod_proxy sends "Connection: Close" or "Connection: Keep-Alive". My theory is that mod_proxy is incorrectly using a socket pool even when the server responds with "Connection: Close" or when mod_proxy itself sends "Connection: Close". Most of the time it probably realizes that the sockets in the pool are actually closed, but sometimes not and it tries to use a dead socket. What should be happening is that mod_proxy needs to NOT use a socket pool if "Connection: Close" is in use. However, I have not looked very deeply into the code to see if my theory is accurate. Could someone more familiar with the mod_proxy code please fill me in? Does mod_proxy always use a connection pool? Thanks, Adam On Mon, May 12, 2008 at 4:15 PM, Adam Woodworth <[EMAIL PROTECTED]> wrote: > I haven't looked at the code in mod_proxy to see how it handles the > Keep-Alive header returned by the backend server, but what I'm seeing > in this tcpdump I have that shows the proxy error happening is that > the backend webserver is IIS, and it is not sending any Connection or > Keep-Alive headers back to mod_proxy, even though mod_proxy sent a > Connection: Keep-Alive header. But, this is HTTP 1.1 -- is the server > required to send back a Connection and Keep-Alive header? > > What I'm wondering is, if the server doesn't send back a Keep-Alive > header that specifies the timeout and max requests (e.g., "Keep-Alive: > timeout=15, max=500"), then does mod_proxy just default to an infinite > (or very large) timeout for that connection? And somehow it's not > noticing that the server closed the connection? What I see is that in > a nearly 700 second long tcp dump, there are a few mod_proxy requests > made to the backend w/o any SYN/SYN-ACK/ACK TCP handshaking happening > before the GET packet, which means that Apache must think that the > socket is still open -- i.e., it used the socket for a connection a > bunch of minutes ago, and still thinks it's alive. But I see this > happening 600 seconds into the dump, without any other activity on the > same socket prior to that in the dump, which leads me to think that > Apache is just keeping the socket around for quite a long time. But > it surprises me that IIS hasn't closed the socket by then, so that by > the time Apache tried to use it it would notice it was closed...hmmm, > very strange. > > Adam > > > On Mon, May 12, 2008 at 3:31 PM, Nick Kew <[EMAIL PROTECTED]> wrote: >> On Mon, 12 May 2008 13:52:18 -0400 >> "Adam Woodworth" <[EMAIL PROTECTED]> wrote: >> >> > Hi, >> > >> > I was wondering if anyone might have some more information on bug >> > #37770. I've added a comment there recently, at the end of the bug >> > report, #83. >> > >> > https://issues.apache.org/bugzilla/show_bug.cgi?id=37770 >> >> If this is indeed a race condition biting you, it's possible >> you could fix it by setting a sufficiently low ttl value that >> the backend's ttl will be guaranteed to outlive it - depending >> of course on backend behaviour. So if the backend has a >> 15 second ttl, set 10 secs in the proxy, and the remaining >> 5 secs provide a good buffer against a race condition. >> >> Except that ttl doesn't work exactly as documented. >> >> There's a similar issue with ttl and DBD database connections. >> >> I've just posted on the subject to [EMAIL PROTECTED] ("apr_reslist >> semantics"). This could enable you to configure apache >> to avoid this problem without sacrificing backend keepalives >> altogether. >> >> -- >> Nick Kew >> >> Application Development with Apache - the Apache Modules Book >> http://www.apachetutor.org/ >> >
