On 19/05/2021 09:28, Paul P Wolf wrote:
<snip/>
In regards to point 5 and 6, let me try to point out my issues with the
documentation and your explanations:
- "Each incoming request requires a thread for the duration of that request. If more
simultaneous requests are received than can be handled by the currently available request
processing threads, additional threads will be created up to the configured maximum (the
value of the maxThreads attribute)." So far the documentation sounds good to me.
Ack.
- "If still more simultaneous requests are received, they are stacked up inside the server socket
created by the Connector, up to the configured maximum (the value of the acceptCount attribute)." This
is what I meant with "accept queue" or what you call "Tomcat's own queue". Would you
usually configure the acceptCount to be less than maxConnections or is it completely unrelated to
maxConnections? I would intuitively assume you would set it to less, but now I am not sure anymore.
maxConnections is the maximum number of concurrent connections Tomcat is
prepared to establish via a call to Socket.accept(). Once that limit is
reached Tomcat will not call Socket.accept() again until one or more
established connections is closed. Tomcat will then call Socket.accept()
until maxConnections is reached or Socket.accept() blocks waiting for an
incoming connection.
The acceptCount/backlog is maintained by the OS and is invisible to
Tomcat. It is the queue of connections that have been received by the
network stack and are waiting for Tomcat to call Socket.accept() to
start processing them.
maxConnections and acceptCount are entirely unrelated.
The correct setting for maxConnections is driven largely by the maximum
concurrent load you want (essentially maxThreads), whether you are using
keep-alive and the percentage of time kept-alive connections are idle vs
active.
For example if you had a system that could support 200 concurrently
processing requests, those connections use keep-alive had active
requests for 10% of the time then a good value for maxConnections would
be around 200/10% = 2000
The correct setting for acceptCount is driven by the maximum spike in
new connections you want to accept. The higher the value, the higher the
spike in new connection attempts you'll be able to support but those new
connections will then be queued in the acceptCount/backlog queue waiting
for Tomcat to process them. You need to consider how quickly Tomcat can
clear that backlog vs typical client timeouts.
Say Tomcat can process 2000 requests a second and the typical client
timeout is 5s, then an acceptCount/backlog of anything up to 10000
should be OK but above that some clients will time out because Tomcat
won't be able to clear the all backlog before the unprocessed client
connections timeout.
- "Any further simultaneous requests will receive "connection refused" errors, until
resources are available to process them." Now if that is the case, than what does
maxConnection have to do with anything at all?
See above. That is describing what happens when maxConnections has been
reached AND the acceptCount/backlog queue is full.
Also I just don't see any connections being refused, but instead they linger in SYN-SENT
state. You say, that acceptCount can be overridden by the OS, but why would that take
away tomcats possibility of monitoring those connections, as "they are stacked up
inside the server socket created by the Connector, up to the configured maximum (the
value of the acceptCount attribute)"?
No they are not. You are confusing the connections where Tomcat has
called Socket.accept() but not passed the socket to a thread for
processing with connections still in the acceptCount/backlog queue.
The way I see it, tomcat should always be able to monitor those connections or
never at. Are there separate acceptCounts for Tomcat/OS/TCP stack?
Tomcat has full visibility of the number of connections where
Socket.accept() has been called. Tomcat has no visibility of the number
of connections in the acceptCount/backlog queue.
- "Note that once the [maxConnection] limit has been reached, the operating system
may still accept connections based on the acceptCount setting". Here again my
confusion rises: Does this only apply/make sense if the OS overrides the acceptCount?
There is always an acceptCount. The issue is whether the OS follows the
setting provided by Tomcat or does its own thing anyway.
If so, would Tomcat still be able to monitor those connections in the server
socket created by the Connector or not?
See above. Tomcat tracks connections where Socket.accept() has been
called. Tomcat cannot track connection requests in the
acceptCount/backlog queue.
If the OS doesn't override the acceptCount, is Tomcat then able to monitor the
connections?
Whether the OS follows the acceptCount setting provided by Tomcat or not
has no impact on Tomcat's inability to monitor the acceptCount/backlog
queue.
Furthermore: The documentation implies that if acceptCount < maxConnections, than it will
only allow maxConnections, but as you pointed out yourself earlier, this doesn't seem to be
the case, but instead it's "maxConnections (3) + acceptCount (2) + 1".
Which text implies that?
What suggests to me that the OS doesn't override the acceptCount in my case
is, that the ESTABLISHED connections always correlate to my acceptCount
setting, but still connections don't get refused. This seems like a bug to me.
Your numbers are too close together. If you use numbers that are further
apart, the behaviour should be more obvious. Something like:
maxThreads=4
maxConnections=10
acceptCount=20
- "When you say "accept queue", I think "TCP stack backlog" which is 100% in
the OS and not visible to Tomcat.
Correct.
Tomcat itself has its own queue of incoming connections (ones that have been
accept()ed but no thread has yet been assigned).#
Correct.
I suppose you mean this Tomcat-queue when you say "requests outside the queue, but still within the
maxConnections limit"?"" - with accept queue I mean the internal one. How do you know what
goes into the TCP stack backlog and what goes into the internal queue? What I mean by "requests outside
the queue, but still within the maxConnections limit" is the following: Imagine your acceptCount is 5,
but you accept more, because maxConnection 13. So maxConnection - acceptCount = 8. What happens with those
requests? Do they land in the TCP stack backlog?
A new connection first gets added to the acceptCount/backlog queue. When
Tomcat calls Socket.accept() it is removed from the acceptCount/backlog
queue and added to the current connection count. It is then added to
the Poller. Ignoring writes, when there is data to read, the Poller
passes the connection to the thread pool. Once the data has been read,
the connection is returned to the Poller (or closed). When the
connection is closed the current connection count is reduced.
As discussed before, I actually see 19 ESTABLISHED connections with a
configuration like that. So do actually 13 land in the backlog? To me this
seems like a (documentation) bug.
That looks right to me, assuming that the OS is putting the connections
into the established state as soon as they land in the acceptQueue /
backlog. You have maxConnections(13) + acceptCount(5) + 1 = 19
established connections.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
