Aaron Bannert wrote:
>On Thu, Apr 11, 2002 at 02:09:27PM -0700, Brian Pane wrote:
>
>>The problem isn't that the busy worker threads will never become unbusy
>>and pick up new work from the queue. If the queue is full, and the listener
>>is blocked, the listener will (with the current code) be properly awakened
>>as soon as one of the workers finishes its current connection. The problem
>>is that it could be a long time before that happens, if all the workers are
>>handling very long-lived connections. And meanwhile, there could be other
>>child procs with idle worker threads.
>>
>
>Ok, now we're on the same page. I see this as a problem as well, but I
>don't think this is what is causing the problem described earlier in this
>thread. Considering how unlikely it is that all of the threads on one
>process are on long-lived connections, I don't see this as a critical
>short-term problem. What is more likely is that 'ab', used to observe
>this phenomon, is flawed in a way that prevents it from truly testing the
>concurrent processing capabilities of the worker MPM, when it is possible
>for a request on a different socket to be returned sooner than another.
>Flood would be much more appropriate for this kind of a test.
>
While flood definitely has more concurrency, I don't think the performance
problem in this test case was ab's fault. From what I was able to observe
when I recreated Nick De Decker's original test case on Linux, I think the
two big factors contributing to the poor performance are:
* Adding concurrency (to handle increasing traffic) is expensive
with the worker design, as it requires creating another child proc
and a whole bunch of threads.
* And while this expensive operation is happening, the existing child
processes, which by definition already have most of their worker
threads busy, are blindly accepting and queuing more connections.
And those connections pile up in the queue because the currently
busy worker threads are fighting for CPU cycles with the startup
of the new child process. When the new child proc finally starts
up, a lot of the connections that it ought to be handling are
stuck in the queues of the existing children.
>>Which all leads me back to the conclusion that we need to stop managing a
>>separate queue within each child process. (In the short term, though,
>>blocking the listener from doing an accept if the # of idle workers is
>>zero would be a reasonable alternative.)
>>
>
>At one point I had an alternative queue design for worker where slots
>in the queue represented an available worker as opposed to now when it
>represents an accepted connection. This way when the queue is empty
>(or full, depending on how you want to look at it) then there are no
>more immediately available worker threads.
>
Yep, your "time/space tradeoff" design is similar to leader/follower.
They're almost the same design, except that leader/follower keeps
a connection within the same thread that accepted it, to try to get
the context switch and a couple of mutex operations out of the critical
path of request processing.
--Brian
