Am 02.02.2016 um 18:01 schrieb Luca Toscano:
Hi Rainer,
thank you 100 times for this email, it was really helpful! Comments inline:
2016-02-02 17:12 GMT+01:00 Rainer Jung <rainer.j...@kippdata.de
<mailto:rainer.j...@kippdata.de>>:
The number of worker threads per process is constant during the
lifetime from the process creation to its end. It is equals to
ThreadsPerChild and the sole purpose of this configuration item.
This information is not completely trivial to establish simply reading
the documentation, even if all the information are there. Reading
mod_worker's doc can lead to think about this option, but there is
always the space for some doubts..
You are right and it should be made clear in the docs, that all those
MPMs work on a fixed number of threads per process basis. If they need
to grow, they can only grow by producing new processes (with each again
having the same fixed number of threads).
AsyncRequestWorkerFactor comes into the image, because in contrast
to the traditional MPMs prefork, worker and winnt, event does not
have a 1:1 relation between worker threads and client connections.
Event is designed to scale better than those in terms of
connections. It should handle a lot more connections with the same
number of threads. How can this work? It works by freeing the worker
threads from the connection handling in certain situations where the
thread would actually simply wait until it can write back the next
part of the response or until the next request on the same
connection arrives. But this design results in some over commitment.
What happens if by bad luck over time we accepted lots of
connections, which were mostly idle and then suddenly many of them
start activity which needs a worker thread for each of them. Then it
might happen, that we do not have enough worker threads in the
process that accepted the connections. We don't have a way to move
such connections to another child process. Once the connection is
accepted it stays with the same process.
The only thing that I would like to expand a bit would be: "in certain
situations where the thread would actually simply wait until it can
write back the next part of the response or until the next request on
the same connection arrives.", but as Eric told me on #httpd-dev it is a
bit complicate to summarise in few sentences (an easy one is keep
alive). It might not be needed in the documentation, but it would be
good for the people interested in an in depth look without reading the code.
Maybe the mod_status output can be a guidline about interesting states.
Have a look at the table part visible under
http://www.apache.org/server-status
The last three columns are all under "Async connections". Thos are
connection states which would need a fixed worker thread in the old
style MPMs (not event), but need no worker thread for event MPM:
- writing
Writing back the answer to the client, but the TCP write buffer is
full, because the connection to the client is too slow so packets pile
up. When the next write would block, we instead free the worker thread
and wait for that connection to get writable again. Not 100% sure, but I
think this only happens when we serve static content. Not in the case of
a proxy.
- keep-alive
The response is finished and we wait for the next request to arrive,
more precisely for the first packet of the next request.
- closing
Sometimes we need to do a lingering close. That means we want to send
back an error or similar, but the error condition happened, before we
have read the whole request from the client. If we would simply write
back the response and then close the connection, the client - still
trying to send the rest of the request -would get a connection reset and
could not read our answer. So in such cases, we try to read the rest of
the request although we are not interested any more in it, simply to
allow the client to consume our response after completing sending the
request. There's a time limit for this lingering close but it can take
relatively long time. The waits for the next packets from the clients
here again happen without a dedicated worker thread.
These three cases are connections, that are not really idle, but need to
wait for something to happen. During the waits we free the worker thread
(decouple from the connection and allow it to work on another one).
Looking further at the status output, roughly the number "Connections
total" would be equals to the sum of the three "Async connections" count
plus "Threads busy". In theory this is not rue, because the numbers are
retrieved while the output is generated, so some of them change during
counting others (the numbers are not retrieved in an atomic way).
If you see in the "accepting" column a "no", it either means, that our
over commitment limit is reached, or - more often - that this process is
scheduled to end because it has reached it MaxConnectionsPerChild, or we
have to many idle processes (MaxSpareThreads) or someone issued
"apachectl graceful" etc.
The rest of the explanation is simply great, very easy and
straightforward. I would really like to add some of it to the how it
works section because I think it would help a lot of people.
...
HTH a bit w.r.t. AsyncRequestWorkerFactor (and also hoping that I
didn't make mistakes in explaining).
Same thing for the second part of the email, I believe that the
directive AsyncRequestWorkerFactor should get more information, maybe
with a little section explaining some directions to take for tuning it.
I want to re-state my point: httpd's documentation contains often all
the information needed but sometimes it misses the necessary background
for the people not heavily involved in the httpd community to get the
whole picture straight away.
Fully agreed.
I'll work on a small patch for the mod_event's documentation, and I'll
send it to the mailing list to get some feedback to get your opinion.
Thanks a bunch!
Rainer
