Chris, added some comments inline. Le mer. 20 mars 2024 à 19:41, Christopher Schultz < ch...@christopherschultz.net> a écrit :
> Romain, > > On 3/20/24 13:34, Romain Manni-Bucau wrote: > > Thread dumps being dump of threads - literally os threads - and virtual > > threads not being threads at all - they are runnables in a dedicated > thread > > pool - it is quite fair to not make them the same and have their > scheduler > > - pool - only in the thread dump but not themselves no? > > Maybe. > > If you take a thread dump today (with a "real" thread), you only get a > Java stack trace, you get no "native stack trace" or anything like that. > So from that perspective, the "thread" is really the instance of > java.lang.Thread which could just as easily be a Virtual Thread. > > You also get no scheduling information, other than what the thread's > "priority" is... but you can't get any real-time data about where it > sits in the scheduling queue, etc. > Well, this does not work like that, as mentionned it is a ForkJoinPool so not a plain queue - except the design which is not a 1 task = 1 position, it has multiple queues - so the position in "the queue" does not give you the information you are mentionned there. If you really care about monitoring this pool you can just instrument java.lang.VirtualThread#DEFAULT_SCHEDULER (plain reflection works good enough if you open this part but agents too). > > I'm much less interested in what the "native thread" is doing _below_ > the Java part. Presumably, it's always running > Thread.cpp::runTheJavaCode and that's not useful information to anybody. > Depends how you instrument/query it, while at code level it does not change much things and you still get the thread stack context. For raw thread dumps you indeed need the new jcmd command (Thread.dump_to_file) and I agree there is no real point to not let it go in the plain jstack with a new toggle you can enable at need (but hopefully not by default since we'll end up with undownloadable/too big dumps). Ultimately using reflection and opening jdk.internal.vm package you can also just use ThreadContainers.root() and iterate over children()/threads() but not sure how portable it will be. There is a negative effect there, before we were often decorating executors (the Runnable to execute) to add before/after context and potentially track threads there but it does not work anymore since threads are totally unbounded. > > If the Virtual Thread is not mounted on an OS thread, then it's > "suspended" or "blocked" or whatever-it-is. If it's on an OS thread, it > had better be running: that's the whole point of the scheme. (I suppose > it could be BLOCKED-yet-on-an-OS-thread -- one of the current problems > which hopefully will be less of a problem in the future, but I'd like to > ignore that for now). > > I don't know what's wrong with having millions of threads, and still > being able to walk through them using existing APIs. Nobody walks > through threads for no good reason... if you want to see the threads, > you should be able to see the threads. > > If I wanted to walk-through every instance of java.lang.String in a JVM > (¡millions of them!), then I should be able to do it even if (a) it's a > weird thing to do and (b) it will take a while. Well, I want to do it, > so let me do it(!). > > If you have only one "real" Thread (e.g. the main thread) and everything > else is Virtual... when you ask for a thread dump or walk all the > threads, do you only see "main" and not the mounted-on-OS-threads > Virtual Threads? If the JVM were willing to consider Virtual Threads > "visible" and therefore dumpable, etc. through existing interfaces > _while they were mounted on an OS thread_ I could almost agree that > makes some sense. But if I have a unit of work not making any progress, > I'm gonna want to be able to see that (Virtual) Thread in a thread dump > or any other kind of analysis tool, including its stack trace. > > > Similarly to why threadlocal are not recommended for virtual thread this > > would probably make an useless pressure on the JVM IMHO - why there is an > > option to see it but it is mainly for debug purposes. > > See virtual threads as continuations (suspendable/resumable "Runnable") > in > > a dedicated and not programmatically configurable nor > selectable/proviable > > thread pool, they are not in thread dumps and this doesnt bother you ;) - > > ultimately if you want it you want all java objects to be monitored and > in > > the thread dump which would be weird - but I agree the semantic is > > misleading. > > I'm not sure why ThreadLocals are not recommended for Virtual Threads. > Honestly, the presence ThreadLocal is a gigantic hack for badly-written > APIs but if you accept that fact, there doesn't seem to be anything > really bad about it. It "doesn't make sense" for Virtual Threads because > it's wasteful: you probably use something once or twice and don't get > the benefit of it when the thread dies -- because you aren't supposed to > use thread pools anymore. But it's no less wasteful and dumb than it was > in the first place. > Guess the issue is they are bound to low level threads and for virtual threads this consumes too much mem and since they are shared between standard threads and virtual threads you can't optimize them by backing them by scopes like that, so let's blame the "history" maybe? > > IIRC, Tomcat uses ThreadLocal to store date-formatting objects for > access logs. That's a great use for ThreadLocal. If the threads are no > longer pooled, it's "wasteful" in that we create and discard a > SimpleDateFormat for every log, but what's the alternative? Use a shared > SimpleDateFormat and synchronize across threads? Yuck. So it's up to > Tomcat to decide if ThreadLocal makes sense in a Virtual Thread world. > DateTimeFormatter solved it some versions ago, a singleton is the best solution for such a thing. Anything else - when thread safety can't be reached - will end up in "pools" - a queue being a cheap pool - which should be sized as the backing forkjoin pool of virtual threads probably. > > (Honestly, we need to start using java.time. Now that Tomcat 8.5 no > longer needs to be supported, we should look at our options for moving-on.) > > > From my tests virtual threads do their job but they stay slower than a > > proper reactive/async impl mainly due to the overhead they add > *everywhere* > > compare to reactive programming plus this single thread pool issue which > > adds a lot of contention when all the app/lot of tasks is/are done using > it > > - vs bulkhead pattern for ex. > > But if you come from a plain sync application it can be very interesting > if > > it stays compatible and you don't need to add throttling to control the > > memory - often in the old style you throttle with the number of threads, > > now you need a semaphore or alike. > > Will not be a free lunch ;). > > Agreed, free lunches never exist. > > But. > > If an application exists that never bothered to convert to using > non-blocking I/O but uses many threads, it could be very easily changed > (slightly) to use Virtual Threads and get a performance boost out of the > deal. Or at least be able to schedule many more tasks without running > out of stack space. > > The price you pay for that is CPU usage. And most of us have CPU to spare. > Hmm, I have more than doubt. I currently have a case for a customer, we scaled to the desired perf level tuning threads to a high number - everything is old style and synchronous - but if we move to VT it will just blow up and explode in mem in ~5s of load - tested a close approach. So I suspect most of blocking apps also rely in their prod deployment on the implicit thread throttling so VT can just make them blow up so "easy" in terms of code, hard in terms of tuning since them the actual throttling was not planned anywhere else in the code I think. > > -chris > > >> Rainer, > >> > >> Thanks for writing this up. > >> > >> On 3/20/24 07:22, Rainer Jung wrote: > >>> I wanted to share an observation and I hope the things are correct how > I > >>> am describing them. Maybe things have already improved and I am not > >>> aware of it, hints welcome. > >>> > >>> Part of JEP 425 (Project Loom, Java virtual threads) discusses how to > >>> handle observability of virtual threads from inside the JVM and > tooling. > >>> > >>> The final outcome is, that virtual threads are not included in the > >>> typical JVM APIs which one can use to observe threads, like enumerating > >>> them or accessing the stacks of individual threads. As a consequence, > >>> also jstack and "kill -QUIT" do not show virtual threads at all, not > >>> even when they are attached to a native thread and executing code. > >> > >> O_O > >> > >>> There is one single method to help with observability of virtual > threads > >>> > >>> > >> > https://docs.oracle.com/en/java/javase/21/docs/api/jdk.management/com/sun/management/HotSpotDiagnosticMXBean.html#dumpThreads(java.lang.String,com.sun.management.HotSpotDiagnosticMXBean.ThreadDumpFormat) > >>> > >>> which dumps a full thread dump to a file. Of course that is by no means > >>> appropriate, if you want to do a fine grained observation. At least you > >>> can choose to write a json structure instead of a hard to parse text > >>> format, but that's it. > >> > >> Boy, that's a real miss from the JVM team. I'm surprised that they have > >> overlooked this. I don't see a real reason that a Virtual Thread would > >> be treated any differently than a regular thread. > >> > >>> For instance I am often using a tool, that inspects our > >>> RequestProcessors to find long running requests and then retrieves the > >>> list of Java threads as ThreadInfo objects to find the one executing > the > >>> request and finally retrieves the stack of that request from the JVM. > >>> Such an approach is no longer possible. Almost all of JMX does not show > >>> any info about virtual threads. > >>> > >>> It also seems Tomcat no longer uses a pool when a connector is > >>> configured to use virtual threads. So there are no metrics like > >>> currentThreadCount or currentThreadsBusy. > >> > >> When using virtual threads, the number of requests is the same as the > >> number of threads, since each request -> new Virtual Thread.... though I > >> think the request-count is only incremented when the request has > >> completed, so maybe there are threads running that can't be counted > (yet). > >> > >> But I understand that you were hoping to get some information about > >> these threads and though maybe Tomcat's metrics could help. I guess not > >> really. > >> > >> But thread-busy count is the same as in-flight-request count. And > >> current thread count is the same as in-flight-request count as well. > >> > >>> I guess this also limits the use of some manager features and probably > >>> also the StuckThreadsDetectionValve when combined with virtual threads. > >> > >> Most likely. I'm fairly sure that uses the "usual Thread-walker things" > >> which you are reporting do not work any more with VTs. > >> > >> What JVM are you using for your investigations? > >> > >>> Of course Tomcat has solved most of the problems, that virtual threads > >>> want to solve, by doing the hard work of using the NIO and NIO2 APIs. > So > >>> virtual threads are probably not that important for us. But we should > be > >>> aware of the observability deficiencies whenever we would discuss > >>> whether we could switch from NIO or NIO2 to virtual threads to simplify > >>> connector code in some distant future. > >> > >> +1 > >> > >> I've been enthusiastically talking with markt about dropping all that > >> nasty NIO stuff and "going back to BIO with virtual threads" which, at > >> this point, is mostly a threat and not a promise. But if VTs really > >> deliver everything they are claiming to deliver, then it may be possible > >> to go back to BIO as long as servlet-async is retired. And I'm not > >> holding my breath on that one. > >> > >> -chris > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > >> For additional commands, e-mail: dev-h...@tomcat.apache.org > >> > >> > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > >
