Hi Luca,
some fragmentary answer:
Am 01.02.2016 um 10:17 schrieb Luca Toscano:
...
- AsyncRequestWorkerFactor is used to regulate the amount of requests
that a single process/threads block can handle, calculating the value
periodically using the idle threads/workers available. In case of
workers maxed out, the keep alive sockets/connections are closed to free
some space.
...
If my understanding is correct (I doubt it but let's assume this) then I
have the following questions:
- Would it be worth to add more info in the "how it works" section? A
first read may bring the user to think that the listening thread is the
one doing the actual work, rather than the workers, being a bit puzzled
when reading the AsyncRequestWorkerFactor section.
...
- The summary talks about "supporting threads" and given the fact that
AsyncRequestWorkerFactor is added to ThreadsPerChild, it raises the
question about how many of them are created at startup. Conversely, is
it a way to say: the number of threads for each process are
ThreadsPerChild but since they now perform also small bursts of work
(like keep alive house keeping and flushing data to clients) the total
amount of connections allowed should be more to make room for all these
connection/socket states?
The number of worker threads per process is constant during the lifetime
from the process creation to its end. It is equals to ThreadsPerChild
and the sole purpose of this configuration item.
AsyncRequestWorkerFactor comes into the image, because in contrast to
the traditional MPMs prefork, worker and winnt, event does not have a
1:1 relation between worker threads and client connections. Event is
designed to scale better than those in terms of connections. It should
handle a lot more connections with the same number of threads. How can
this work? It works by freeing the worker threads from the connection
handling in certain situations where the thread would actually simply
wait until it can write back the next part of the response or until the
next request on the same connection arrives. But this design results in
some over commitment. What happens if by bad luck over time we accepted
lots of connections, which were mostly idle and then suddenly many of
them start activity which needs a worker thread for each of them. Then
it might happen, that we do not have enough worker threads in the
process that accepted the connections. We don't have a way to move such
connections to another child process. Once the connection is accepted it
stays with the same process.
To reduce the likeliness of such shortage in free worker threads, the
number of connections we accept per process is limited. The limit is not
a fixed number or some fixed multiple of ThreadsPerChild, but instead it
is calculated based on the number of idle worker threads. If that number
gets small during run time, the number of additional connections we
accept decreases until at some point that process won't accept any new
connections. This situation can be monitored via mod_status.
Let's have a look at the formula: The docs say:
ThreadsPerChild + (AsyncRequestWorkerFactor * number of idle workers)
is the number of new connections a process will still accept (not
counting connections in closing state). Ignoring for a moment the
"closing state" stuff, we can write:
max_connections = ThreadsPerChild + (AsyncRequestWorkerFactor *
idle_workers)
Let's replace ThreadsPerChild by (idle_workers + busy_workers):
max_connections = (idle_workers + busy_workers) +
(AsyncRequestWorkerFactor * idle_workers)
= busy_workers + (AsyncRequestWorkerFactor + 1) * idle_workers
Now splitting connections in busy and idle ones and defining a busy
connection as one needing a worker thread so that the number of busy
connections equals the number of busy workers, we get:
max_idle_connections + busy_workers = busy_workers +
(AsyncRequestWorkerFactor + 1) * idle_workers
and thus
max_idle_connections = (AsyncRequestWorkerFactor + 1) * idle_workers
So although we only have idle_workers left, we accept up to
(AsyncRequestWorkerFactor + 1) * idle_workers as the number of idle
connections. Since AsyncRequestWorkerFactor by default has the value 2,
it means by default we accept 3 times as many connections, as we have
idle workers. That's a handy formulation for our type of over commitment.
Of course the formula is not always true. If many connections turn over
into busy ones, than we might suddenly have more (old) idle connections
than allowed by this formula. E.g. if for a moment all workers are idle,
we would allow 3*ThreadsPerChild (idle) connections. Then assume
ThreadsPerChild of them start to be busy, then we would allow no idle
connections because no idle workers are left, but we would still have
2*ThreadsPerChild old idle connections. The formula only tells us how
many we are willing to handle unless we are already over the limit
because to many turned into busy ones.
I have not done the math if we also add "connections in closing state"
into the game.
There's no easy way to tune AsyncRequestWorkerFactor since it somehow
depends on the activity pattern of connections, which is hard to
estimate. It is best to monitor the status via mod_status to get an
idea, if one can save more threads with a higher value (more over
commitment), or if one needs to stay on the conservative side. Sometimes
it is better to work with higher ThreadsPerChild, so that the activity
patterns on more connections per process gives a better average behavior.
HTH a bit w.r.t. AsyncRequestWorkerFactor (and also hoping that I didn't
make mistakes in explaining).
Regards,
Rainer
