On 8/09/2023 12:38 am, Simone Bordet wrote:
Hello,
We switched the Jetty builds to Java 21 a while ago, and they randomly
fail with a hard deadlock during class initialization.
We tried to understand if we were doing something wrong, but the code
compiles fine, and at first glance there seems to be nothing wrong
with it, but it may well be a popular Java puzzler that I am not aware
of :)
We did not see these failures with Java 20, and we do not see them with Java 17.
I was wondering if something changed in Java 21 to cause this deadlock?
I've looked at the dump and there is no sign that the MutableHttpFields
is actively in use. It suggests to me that class initialization has
failed but the class state has not been correctly updated and the
waiters released. There were some changes in JDK 21 about how failures
in this area were handled, so it is possible I/we got something wrong.
Is it possible to try running this with additional logging enabled e.g.
-Xlog:class+init=debug -Xlog:exceptions=debug
I hope this is the right mailing list for this problem.
If not, please let me know.
It may well be a hotspot-runtime-dev issue
Thanks,
David
-----
I am an OpenJDK author so I can open an issue about this, but I don't
have a reproducer, since it seems a race condition happening randomly.
Below you can find more details about this deadlock.
Thanks!
----
Taking a thread dump we see one thread in this state:
java.lang.Thread.State: RUNNABLE
at
org.eclipse.jetty.http.HttpFields.build(org.eclipse.jetty.http@12.0.2-SNAPSHOT/HttpFields.java:76)
- waiting on the Class initialization monitor for
org.eclipse.jetty.http.MutableHttpFields
at
org.eclipse.jetty.http.HttpFields.<clinit>(org.eclipse.jetty.http@12.0.2-SNAPSHOT/HttpFields.java:67)
"ForkJoinPool-1-worker-1" #23 [1457161] daemon prio=5 os_prio=0
cpu=320.77ms elapsed=8870.98s allocated=18414K defined_classes=928
tid=0x00007f6ec0c6a6f0 nid=1457161 waiting on condition
[0x00007f6e91ffb000]
And other threads are in this state (or similar -- the bottom frame is
different but still triggering the clinit of HttpFields):
java.lang.Thread.State: RUNNABLE
at
org.eclipse.jetty.http.HttpTester.parseResponse(org.eclipse.jetty.http@12.0.2-SNAPSHOT/HttpTester.java:274)
- waiting on the Class initialization monitor for
org.eclipse.jetty.http.HttpFields
at
org.eclipse.jetty.http.HttpTester.parseResponse(org.eclipse.jetty.http@12.0.2-SNAPSHOT/HttpTester.java:261)
"ForkJoinPool-1-worker-2" #24 [1457162] daemon prio=5 os_prio=0
cpu=136.15ms elapsed=8870.96s allocated=3303K defined_classes=340
tid=0x00007f6e4c01ddb0 nid=1457162 waiting on condition
[0x00007f6e91efc000]
Class hierarchy:
MutableHttpFields implements HttpFields.Mutable
HttpFields.Mutable extends HttpFields
Full thread dump:
https://gist.github.com/olamy/7e883adcf51b2410337abfa775a79a0b
Code:
https://github.com/eclipse/jetty.project/blob/jetty-12.0.1/jetty-core/jetty-http/src/main/java/org/eclipse/jetty/http/HttpFields.java
