Justin wrote:
> Thanks for the links! But. I wasnt sure what in the first link
> was useful for this problem, and, the vacuum bots discussion
> is really a different topic.
> I'm not talking of vacuum bot load. This is real world load.
>
> Practical experiments (ok - the live site :) convinced me that
> the well recommended modperl setup of fe/be suffer from failure
> and much wasted page production when load rises just a little
> above *maximum sustainable throughput* ..
The fact that mod_proxy doesn't disconnect from the backend server when
the client goes away is definitely a problem. I remember some
discussion about this before but I don't think there was a solution for
it.
I think Vivek was correct in pointing out that your ultimate problem is
the fact that your system is not big enough for the load you're
getting. If you can't upgrade your system to safely handle the load,
one approach is to send some people away when the server gets too busy
and provide decent service to the ones you do allow through. You can
try lowering MaxClients on the proxy to help with this. Then any
requests going over that limit will get queued by the OS and you'll
never see them if the person on the other end gets tired of waiting and
cancels. It's tricky though, because you don't want a bunch of slow
clients to tie up all of your proxy processes.
It's easy to adapt the existing mod_perl throttling handlers to send a
short static "too busy" page when there are more than a certain number
of concurrent requests on the site. Better to do this on the proxy side
though, so maybe mod_throttle could do it for you.
- Perrin
