Hi List, We figured this out just in case anyone reads this in the archives.
It turned out we were doing a few things wrong in our setup - one was using prefork on a busy webserver whereas worker is recommended, and the other was running the Computer Associates Siteminder plugin despite vendor recommendations to use the worker handler. In more detail, it appears that all incoming connections to our site - including unencrypted requests for index.html - resulted in connections to a policy server. So it appears that three issues combined to cripple our site: (i) prefork simply isn't the best handler for a busy webserver; (ii) Siteminder & prefork don't agree with each other (because there is no connection pooling in the prefork handler, the overheads relating to setting up & tearing down TCP connections is expensive), and (iii) I suspect that our website code is badly written so that calls to the policy server are being made unnecessarily. NOTE: it turned out that the EAGAIN messages observed in strace were probably a red herring. I have seen other hits on google suggesting these are a problem but I have subsequently concluded that some of these errors are actually normal - even on site that's not under high load. Best regards, Alex