Guys,
I wanted to follow back around on some of the websocket load testing we've been
doing in EC2. The good news is, we were able to get 100K websockets
connected directly to a single Tomcat instance (EASILY) - once we got the
settings right. As a result, I wanted to post my results here for 2 reasons:
1) Hopefully this may benefit others
2) I'd like ask the Tomcat experts which of the changes we made is most
likely to have contributed to this working, or is it a combination?
My theory is it's a combination of: acceptorThreadCount + maxKeepAliveRequests.
Logically, it would make sense that running on with 16vCPU(s) should benefit
from more acceptor threads. In addition, reading the maxKeepAliveRequests, we
wondered if making this UNLIMITED would help when many concurrent websocket
requests come in because they are all HTTP requests initially that get upgraded.
So, if anyone can clarify whether our theory is correct; or, if not - what
settings below actually made the difference, that would be great! Without an
understanding of what Tomcat is doing under the hood, my theory is just that.
Thanks for all the support you guys provide on this list,
Bob
SUMMARY:
I posted earlier this week about having trouble just getting 10 - 20K
websockets connected to a single Tomcat instance running on a 16vCPU/60GB EC2
instance running JVM (G1GC, NUMA, 24G - 48G). The settings during those tests
were the following, and we were seeing websocket connects fail due to
TimeoutExceptions and EOFExceptions.
ORIGINAL SETTINGS
<Connector port="80"
protocol="org.apache.coyote.http11.Http11NioProtocol"
connectionTimeout="20000"
maxConnections="100000"
maxThreads="100000"
redirectPort="8443" />
After looking at the Tomcat Connector documentation closer, along with what
Glassfish recommends when deploying in production, we modified the settings to
the values shown below:
WORKING SETTINGS
<Connector port="80"
protocol="org.apache.coyote.http11.Http11NioProtocol"
acceptorThreadCount="8"
maxKeepAliveRequests="-1"
connectionTimeout="-1"
maxConnections="-1"
maxThreads="20000"
redirectPort="443" />
Bob DeRemer
Senior Director, Architecture and Development
[Description: Description: Description: Description:
cid:image001.png@01CBE3DE.51A12030]
http://www.thingworx.com<http://www.thingworx.com/>
Skype: bob.deremer.thingworx
O: 610.594.6200 x812
M: 717.881.3986
