Re: Virtual Thread Configuration In Tomcat 11
https://tomcat.apache.org/tomcat-11.0-doc/config/http.html Search for useVirtualThreads The same option exists in the latest 8.5.x, 9.0.x and 10.1.x releases. You need to be using Java 21 to use virtual threads. Mark On 22/08/2023 14:14, William Crowell wrote: Hi, To use virtual threads in Tomcat 11, do you need to configure anything with the connectors in server.xml, or does Tomcat 11 use virtual threads by default (with JDK 21)? Is there any documentation as of yet to configure virtual threads? The only samples I have seen are with Spring Boot using the embedded Tomcat server. Thanks in advance, William Crowell This e-mail may contain information that is privileged or confidential. If you are not the intended recipient, please delete the e-mail and any attachments and notify us immediately. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Virtual Thread Configuration In Tomcat 11
Mark, Thanks for your reply. Just to clarify…this is all I need in Tomcat 11’s server.xml (as well as JDK21): … … Regards, William Crowell This e-mail may contain information that is privileged or confidential. If you are not the intended recipient, please delete the e-mail and any attachments and notify us immediately.
Re: Virtual Thread Configuration In Tomcat 11
On 23/08/2023 10:07, William Crowell wrote: Mark, Thanks for your reply. Just to clarify…this is all I need in Tomcat 11’s server.xml (as well as JDK21): … Correct. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Virtual Thread Configuration In Tomcat 11
I did some performance testing with virtual threads on Apache Tomcat 11.0.0-M10 and JDK 21 (21+35-2513). I have a simple REST service using Spring 6.0.11 that does an insert into MySQL 8.0.32. I have 3 separate boxes all running Rocky Linux 9.2 on AWS (t3a.xlarge which is 4 vCPUs and 16GiB RAM): 1) An application server running Tomcat 11.0.0-M10 with JDK 21 2) MySQL 8.0.32 3) JMeter 5.6.2 I know that JDK 21 is nowhere near GA, but I got some interesting results. I did 10 test runs with useVirtualThreads=“true”, and 10 test runs without virtual threads (the default). Each run used 1000 threads in JMeter for about 3 minutes and 20 seconds for a ramp up time of 3 seconds. What I found was using platform threads performed at least twice as fast as virtual threads. Maybe I am interpreting the results wrong, but this is what I found consistently across each test. Again, I realize that performance is not the only goal of virtual threads. Just my observations. Regards, William Crowell This e-mail may contain information that is privileged or confidential. If you are not the intended recipient, please delete the e-mail and any attachments and notify us immediately.
Re: Virtual Thread Configuration In Tomcat 11
William, On 8/24/23 09:50, William Crowell wrote: I did some performance testing with virtual threads on Apache Tomcat 11.0.0-M10 and JDK 21 (21+35-2513). I have a simple REST service using Spring 6.0.11 that does an insert into MySQL 8.0.32. I have 3 separate boxes all running Rocky Linux 9.2 on AWS (t3a.xlarge which is 4 vCPUs and 16GiB RAM): 1) An application server running Tomcat 11.0.0-M10 with JDK 21 2) MySQL 8.0.32 3) JMeter 5.6.2 I know that JDK 21 is nowhere near GA, but I got some interesting results. I did 10 test runs with useVirtualThreads=“true”, and 10 test runs without virtual threads (the default). Each run used 1000 threads in JMeter for about 3 minutes and 20 seconds for a ramp up time of 3 seconds. What I found was using platform threads performed at least twice as fast as virtual threads. Maybe I am interpreting the results wrong, but this is what I found consistently across each test. Again, I realize that performance is not the only goal of virtual threads. Just my observations. This is "interesting", for some values of interesting. I'd be interested in what the application was doing. Virtual Threads seem to be aimed at things like Web Application Containers where there is a lot of wasted time in a thread's execution waiting on I/O (blocking). If you ignore the application's workload (hah!), the application server is almost entirely occupied with (a) reading the request and (b) writing the response. There is almost no "computation time" and so Virtual Threads make a whole lot of sense. Once inside the application, that may not be true. From what little reading I have done on Virtual Threads, there are some poison pills in there depending upon application design. For example, if your thread blocks while inside a synchronized{} block, then your Virtual Thread is "pinned" and will NOT be un-mounted from the platform thread during the blocking operation. So all of that high-speed context-switching and magic that VT is supposed to provide goes completely out the window when you used to have 200 threads with maybe 8 of them actually running at once but swapping-out preemptively, but now you have 200 virtual threads with 8 of them running at once, but of the 8 running at once, several of them are stopped dead unable to make any progress. At $work, we have plenty of read-through cache routines like this: private Object somethingBigLock = new Object(); private SomethingBig big = null; public Object getSomethingBigFromDatabase() { synchronized(somethingBigLock) { if(null == big) { big = reallyGetSomethingBigFromDb(); // Non-trivial } return big; } } Even if reallyGetSomethingBigFromDb() is mostly blocking on I/O -- e.g. waiting for the db to respond to a query -- it will consume one of your platform threads and lock it up, leaving fewer platform threads available to do the other work. Without a critical analysis of your application and the libraries it uses, it's hard to tell if what you are observing is that "virtual threads are not as fast as regular threads" or "your application is an anti-pattern for Virtual Threads". My guess is that your application and/or the libraries you are using need to be re-written so they no longer use "classic" synchronization and instead use things like ReentrantLock for a similar purpose to play-nice with VT. Or you can wait for the (likely) inevitable improvement of the JVM which allows for threads holding monitors to be unmounted from platform threads. My guess is that is in the future for the JVM. -chris - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Virtual Thread Configuration In Tomcat 11
Chris, Great post earlier today! This is a super interesting topic to me. You can find the performance testing results located here: http://ec2-18-188-185-212.us-east-2.compute.amazonaws.com:8080/web-report/ I did 10 runs with 1000 threads with a ramp up time of 3 seconds for a duration of 200 seconds. Ignore the 11th run. This is a really simple REST application with Spring. It just makes an insert into a MySQL table. When I get a moment I will put my code into GitHub, but again, there is not much to it. You can find a very similar Spring Boot example with embedded Tomcat here: https://medium.com/@anil.java.story/embracing-virtual-threads-in-spring-boot-4140d3b8a5a But he uses JDK 20 with --enable-preview turned on. The mistake I think I made was that the inserts are not really blocking because they happen so quickly. Maybe because I did not put enough load in the test? I have no synchronized blocks in my code. So, it appeared to me that the inserts were taking twice the amount of time with virtual threads. I think that extra time is just overhead of pinning virtual threads to a platform thread and managing the ForkJoinPool. What is going to happen is that people are going to “flip the switch”…useVirtualThreads=”true”. They will find that performance will not be as good as using an operating system thread and abandon virtual threads because their code does not block. So, I need to come up with an example where my code blocks and redo the perf test and prove this out. I don’t know. Regards, William Crowell This e-mail may contain information that is privileged or confidential. If you are not the intended recipient, please delete the e-mail and any attachments and notify us immediately.
Re: Virtual Thread Configuration In Tomcat 11
William, On 9/5/23 17:41, William Crowell wrote: Great post earlier today! This is a super interesting topic to me. You can find the performance testing results located here: http://ec2-18-188-185-212.us-east-2.compute.amazonaws.com:8080/web-report/ I did 10 runs with 1000 threads with a ramp up time of 3 seconds for a duration of 200 seconds. Ignore the 11th run. This is a really simple REST application with Spring. It just makes an insert into a MySQL table. When I get a moment I will put my code into GitHub, but again, there is not much to it. You can find a very similar Spring Boot example with embedded Tomcat here: https://medium.com/@anil.java.story/embracing-virtual-threads-in-spring-boot-4140d3b8a5a But he uses JDK 20 with --enable-preview turned on. The mistake I think I made was that the inserts are not really blocking because they happen so quickly. Maybe because I did not put enough load in the test? I have no synchronized blocks in my code. So, it appeared to me that the inserts were taking twice the amount of time with virtual threads. I think that extra time is just overhead of pinning virtual threads to a platform thread and managing the ForkJoinPool. What is going to happen is that people are going to “flip the switch”…useVirtualThreads=”true”. They will find that performance will not be as good as using an operating system thread and abandon virtual threads because their code does not block. So, I need to come up with an example where my code blocks and redo the perf test and prove this out. I don’t know. Try setting this system property before running your tests and see if it produces any output on the server while the load test is running: -Djdk.tracePinnedThreads=full -chris - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Virtual Thread Configuration In Tomcat 11
Chris, I did set -Djdk.tracePinnedThreads=full and found there were a few pinned threads: Thread[#43,ForkJoinPool-1-worker-4,5,CarrierThreads] java.base/java.lang.VirtualThread$VThreadContinuation.onPinned(VirtualThread.java:185) java.base/jdk.internal.vm.Continuation.onPinned0(Continuation.java:393) java.base/java.lang.VirtualThread.parkNanos(VirtualThread.java:631) java.base/java.lang.System$2.parkVirtualThread(System.java:2648) java.base/jdk.internal.misc.VirtualThreads.park(VirtualThreads.java:67) java.base/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:408) … com.mysql.cj.protocol.FullReadInputStream.readFully(FullReadInputStream.java:64) com.mysql.cj.protocol.a.SimplePacketReader.readHeaderLocal(SimplePacketReader.java:81) … com.mysql.cj.jdbc.ConnectionImpl.setAutoCommit(ConnectionImpl.java:2005) <== monitors:1 com.zaxxer.hikari.pool.ProxyConnection.setAutoCommit(ProxyConnection.java:401) com.zaxxer.hikari.pool.HikariProxyConnection.setAutoCommit(HikariProxyConnection.java) … Over the 12 test runs I have performed, I found a 53% decrease in throughput when using virtual threads: http://ec2-18-188-185-212.us-east-2.compute.amazonaws.com:8080/web-report/ My example code is located here: https://github.com/wcrowell/rest-mysql-app/ The JMX test I used is located here: https://github.com/wcrowell/rest-mysql-app/blob/main/virtual-thread-test.jmx This test makes a POST request to save a record to MySQL: http://ec2-18-117-196-158.us-east-2.compute.amazonaws.com:8080/vt/save Regards, William Crowell This e-mail may contain information that is privileged or confidential. If you are not the intended recipient, please delete the e-mail and any attachments and notify us immediately.
Re: Virtual Thread Configuration In Tomcat 11
When I set -Djdk.tracePinnedThreads=short, then I see this: … Thread[#41,ForkJoinPool-1-worker-4,5,CarrierThreads] com.mysql.cj.jdbc.ConnectionImpl.isValid(ConnectionImpl.java:2516) <== monitors:1 Thread[#39,ForkJoinPool-1-worker-2,5,CarrierThreads] com.mysql.cj.jdbc.ConnectionImpl.isValid(ConnectionImpl.java:2516) <== monitors:1 Thread[#41,ForkJoinPool-1-worker-4,5,CarrierThreads] com.mysql.cj.jdbc.ConnectionImpl.setAutoCommit(ConnectionImpl.java:2005) <== monitors:1 Thread[#43,ForkJoinPool-1-worker-5,5,CarrierThreads] com.mysql.cj.jdbc.ClientPreparedStatement.executeInternal(ClientPreparedStatement.java:893) <== monitors:1 com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdateInternal(ClientPreparedStatement.java:1061) <== monitors:1 com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdateInternal(ClientPreparedStatement.java:1009) <== monitors:1 Thread[#43,ForkJoinPool-1-worker-5,5,CarrierThreads] com.mysql.cj.jdbc.ConnectionImpl.commit(ConnectionImpl.java:791) <== monitors:1 Thread[#43,ForkJoinPool-1-worker-5,5,CarrierThreads] com.mysql.cj.jdbc.ConnectionImpl.setAutoCommit(ConnectionImpl.java:2005) <== monitors:1 … I started digging into the MySQL client code: https://github.com/mysql/mysql-connector-j/blob/8.0.33/src/main/user-impl/java/com/mysql/cj/jdbc/ConnectionImpl.java This class is littered with synchronized blocks, so now this makes sense to me why virtual threads would take longer than a regular OS thread. Regards, William Crowell This e-mail may contain information that is privileged or confidential. If you are not the intended recipient, please delete the e-mail and any attachments and notify us immediately.
Re: Virtual Thread Configuration In Tomcat 11
William, On 9/7/23 08:04, William Crowell wrote: When I set -Djdk.tracePinnedThreads=short, then I see this: … Thread[#41,ForkJoinPool-1-worker-4,5,CarrierThreads] com.mysql.cj.jdbc.ConnectionImpl.isValid(ConnectionImpl.java:2516) <== monitors:1 Thread[#39,ForkJoinPool-1-worker-2,5,CarrierThreads] com.mysql.cj.jdbc.ConnectionImpl.isValid(ConnectionImpl.java:2516) <== monitors:1 Thread[#41,ForkJoinPool-1-worker-4,5,CarrierThreads] com.mysql.cj.jdbc.ConnectionImpl.setAutoCommit(ConnectionImpl.java:2005) <== monitors:1 Thread[#43,ForkJoinPool-1-worker-5,5,CarrierThreads] com.mysql.cj.jdbc.ClientPreparedStatement.executeInternal(ClientPreparedStatement.java:893) <== monitors:1 com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdateInternal(ClientPreparedStatement.java:1061) <== monitors:1 com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdateInternal(ClientPreparedStatement.java:1009) <== monitors:1 Thread[#43,ForkJoinPool-1-worker-5,5,CarrierThreads] com.mysql.cj.jdbc.ConnectionImpl.commit(ConnectionImpl.java:791) <== monitors:1 Thread[#43,ForkJoinPool-1-worker-5,5,CarrierThreads] com.mysql.cj.jdbc.ConnectionImpl.setAutoCommit(ConnectionImpl.java:2005) <== monitors:1 … I started digging into the MySQL client code: https://github.com/mysql/mysql-connector-j/blob/8.0.33/src/main/user-impl/java/com/mysql/cj/jdbc/ConnectionImpl.java This class is littered with synchronized blocks, so now this makes sense to me why virtual threads would take longer than a regular OS thread. This is precisely the kind of thing I was expecting to see. If your "application code" (which includes everything your application does, including calling into libraries) contains lots of "synchronized" blocks, then I think this is the situation you will often find yourself in. I would expect that if you were to increase the size of the virtual thread pool things would start to get better, but in order to truly use virtual threads effectively, you have to track-down and eliminate many uses of synchronized blocks and methods in both your application and the libraries you use. I wouldn't expect virtual threads to become a very widely-deployed performance-optimization technique in the immediate future. I expect that, eventually, lots of code will be re-written to be more virtual-thread-friendly. -chris - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org