Thanks Romain,
I am well aware of the limits oof this attempt concerning async
processing. Still lots of applications can still be diagnosed with this
methodology and I am a bit frustrated, that project loom simply dropped
support for thread inspection via JMX and as a replacement provided a
full dump to a file (!).
But of course you are right: for complex async processing this approach
does not work. Interestingly although Tomcat NIO and NIO2 processors use
async APIs, they can still be inspected with this simple approach.
Best regards,
Rainer
Am 20.03.24 um 13:58 schrieb Romain Manni-Bucau:
Hi Rainer,
Think this is not related to virtual threads I think and way older (2010?).
Since some years we use AsyncContext and therefore thread dumps don't match
pending requests but just active computation (you can get 8K requests with
8 threads for ex).
Most of the sync instrumentation (assumptions) are broken since Servlet 3.0
and must not be used to conclude anything but generally other metrics and
data enable to monitor the server properly (request counter for example is
decoralated from threads, active connections for database - generic term -
etc).
Thread stacks stay useful for performance related work but not for hanging
investigation (for good since it enables a better scalability in general)
so guess it is mainly a habit switch in terms of tools, no?
Hope it makes sense.
Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> | Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>
Le mer. 20 mars 2024 à 13:11, Rainer Jung <rainer.j...@kippdata.de> a
écrit :
Hi hi,
I wanted to share an observation and I hope the things are correct how I
am describing them. Maybe things have already improved and I am not
aware of it, hints welcome.
Part of JEP 425 (Project Loom, Java virtual threads) discusses how to
handle observability of virtual threads from inside the JVM and tooling.
The final outcome is, that virtual threads are not included in the
typical JVM APIs which one can use to observe threads, like enumerating
them or accessing the stacks of individual threads. As a consequence,
also jstack and "kill -QUIT" do not show virtual threads at all, not
even when they are attached to a native thread and executing code.
There is one single method to help with observability of virtual threads
https://docs.oracle.com/en/java/javase/21/docs/api/jdk.management/com/sun/management/HotSpotDiagnosticMXBean.html#dumpThreads(java.lang.String,com.sun.management.HotSpotDiagnosticMXBean.ThreadDumpFormat)
which dumps a full thread dump to a file. Of course that is by no means
appropriate, if you want to do a fine grained observation. At least you
can choose to write a json structure instead of a hard to parse text
format, but that's it.
For instance I am often using a tool, that inspects our
RequestProcessors to find long running requests and then retrieves the
list of Java threads as ThreadInfo objects to find the one executing the
request and finally retrieves the stack of that request from the JVM.
Such an approach is no longer possible. Almost all of JMX does not show
any info about virtual threads.
It also seems Tomcat no longer uses a pool when a connector is
configured to use virtual threads. So there are no metrics like
currentThreadCount or currentThreadsBusy.
I guess this also limits the use of some manager features and probably
also the StuckThreadsDetectionValve when combined with virtual threads.
Of course Tomcat has solved most of the problems, that virtual threads
want to solve, by doing the hard work of using the NIO and NIO2 APIs. So
virtual threads are probably not that important for us. But we should be
aware of the observability deficiencies whenever we would discuss
whether we could switch from NIO or NIO2 to virtual threads to simplify
connector code in some distant future.
Best regards,
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
