Hi community, We are facing some challenges with performance tunning for embedded Tomcat using Spring Boot 3 (Tomcat version 10.1.7) and we would like to ask for advice. The following is an overview of how our workload looks like: - The client is a CDN distributed around the world - Tomcat serves files and media for video streaming, around hundreds of kilobytes by media file - The files and media are in memory (most of the time) - The CND opens a lot of keep alive HTTPs connections, we have seen up to 25000 - There is no proxy or similar in front of Tomcat. Tomcat is handling the HTTPs connection directly - We have only one instance of Tomcat running. - We are avoiding to scale Tomcat horizontally, as it is pretty hard for our domain problem - We can scale Tomcat up, today in some cases we are using an EC2 with 64 cores and 62 GiB memory. We can scale up more if we must, but better if we can downscale instead. - The EC2 is shared with other processes, like transcoders. This is to decrease the latency as much as we can between the components of the solution - We have virtual threads active in Tomcat - We have seen up to 2000 requests/second for light files (less than 10 kilobytes), and 500 requests/second for bigger files. - Spike requests happen in a short time, from 100 requests/second to 1700 requests/second in 2 minutes.
We have seen the server eating 75 % of CPU, so, we want to optimize as much as we can Tomcat to downscale the machine. We have researched and we found some possible points to check: - Should we use NIO or NIO2 connectors? I didn't find an answer for this, we are using NIO. Maybe NIO2 handles better a lot of keep alive connections? - Should we use tcnative to improve the performance for SSL? We are concerned about virtual threads and possible pinning here, as this might use JNI - Should we put a nginx or similar server in front of Tomcat to handle SSL? we are avoiding this for latency reasons, and also, nginx will add up to the other processes we have in the same machine - Should we increase maxKeepAliveRequests? We don't understand how this work entirely, is this the max of requests by one keep alive connection? parallel requests or sequential? seems like the default is 100, and probably we should increase it as the CND might not open more connections if he can send more requests in previous ones. - Should we increase socket.txBufSize? seems like we should, as we are sending media files, having a bigger buffer makes sense - Should we use direct buffers socket.directSslBuffer? - Should we increase the socket.appWriteBufSize? We are exploring JVM performance options also, but any help regarding Tomcat will be appreciated. Regards. -- Daniel Andrés Pelaez López --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
