On Sat, Nov 23, 2002 at 06:40:58PM -0800, Brian Pane wrote: > Here's an outline of my latest thinking on how to build a > multiple-connections-per-thread MPM for Apache 2.2. I'm > eager to hear feedback from others who have been researching > this topic.
You prodded me into finally writing up a proposal that's been bouncing around in my head for a while now. That was in a seperate message, this will be suggestions for your proposal. > 1. Listener thread > A Listener thread accept(2)s a connection, creates > a conn_rec for it, and sends it to the Reader thread. Some (Most?) protocols have the server initiate the protocol negotatiation instead of the client, so the listener needs to be able to pass off to the writer thread as well. > * Limiting the Reader and Writer pools to one thread each will > simplify the design and implementation. But will this impair > our ability to take advantage of lots of CPUs? I was actually wondering why the reader and writer were seperate threads. What gets more complex with a thread pool > 1? I know we'd have to add a mutex around the select+(read|write), but is there something else? > * Can we eliminate the listener thread? It would be faster to just > have the Reader thread include the listen socket(s) in its pollset. > But if we did that, we'd need some new way to synchronize the > accept handling among multiple child processes, because we can't > have the Reader thread blocking on an accept mutex when it has > existing connections to watch. You could dispense with the listener thread in the single-process case and just use an intraprocess mutex around select+(accept|read|write) > * Is there a more efficient way to interrupt a thread that's > blocked in a poll call? That's a crucial step in the Listener-to- > Reader and Request Processor-to-Writer handoffs. Writing a byte > to a pipe requires two extra syscalls (a read and a write) per > handoff. Sending a signal to the target thread is the only > other solution I can think of at the moment, but that's bad > because the target thread might be in the middle of a read > or write call, rather than a poll, at the moment when we hit > it with a signal, so the read or write will fail with EINTR. For Linux 2.6, file notifications could be done entirely in userland in the case where no blocking is needed, using "futexes". But if you want to avoid the extra system calls, you could put a mutex around maintenence of the pollset and just let the various threads dork with it directly. I do keep mentioning this mutex around the select/poll :). Is there a performance reason that you're trying to avoid it? In my past skimmings, I've seen you post a lot of benchmarks and such, so maybe you've studied this. I'm suspicious of signals, but as long as they are tightly controlled with sigprocmask or pthread_sigmask, I guess they aren't so bad.
