I'm interested to know how httpd 2.x can be made more scalable. Could we serve 10,000 clients with current platforms as discussed at http://www.kegel.com/c10k.html , without massive code churn and module breakage? I believe that reducing the number of active threads would help by reducing the stack memory requirements.
Bill Stoddard created an event driven socket I/O patch a couple of years ago that could serve pages. I picked it up and decided to see if I could simplify it to minimize the changes to request processing.
There are sites/workloads where keepalive timeouts tie up the majority of the
active threads or processes, such as our own web site as seen at
http://apache.org/server-status . I also see a lot of "K"s when running specweb99 on a stock httpd. Since we are between requests during keepalive waits, they can be handled differently without impacting a lot of code.
This patch decouples the connection from the worker thread, and offloads keepalive processing to a new event thread in the worker MPM. The event thread uses apr_pollset_* calls to wait for new requests or for the keepalive timeout to expire. When the event thread sees that a socket is readable or a timeout has occured, it passes the connection back to a worker thread, most likely a different thread than the one previously used by this connection.
Here is the patch - http://apache.org/~gregames/event.patch . It is currently
based on the worker MPM from 2.1 HEAD to make it easier to review. It is
working well for me under load on Linux and serves pages on Solaris. When I run specweb99 I've seen around 170 connections served by 5 to 20 worker threads. This is totally dependent on the time the workload spends in keepalive/user think time vs. time spent doing anything else, so YMMV.
This approach could very well hit scalability bottlenecks with various poll() implementations as described on Kegel's web site listed above. There are many alternatives emerging. I've gotten a few inquiries about using sys_epoll on Linux, which sounds like a great fit. I wanted to get something working with plain ol' generic (portable) poll() first before trying out any OS dependent variations. The current apr_pollset_remove will eventually become a bottleneck too -
#ifdef HAVE_POLL
for (i = 0; i < pollset->nelts; i++) {
etcOther issues are described at http://apache.org/~gregames/event.laundry_list . However I think the patch is robust enough now to post. I'm very interested in feedback and having people try it out before going further.
Greg
