HyukjinKwon opened a new pull request, #56803:
URL: https://github.com/apache/spark/pull/56803
### What changes were proposed in this pull request?
Add a single JVM workaround flag to the shared `extraJavaTestArgs` in the
root `pom.xml`:
```
-XX:CompileCommand=exclude,org.apache.kafka.clients.producer.KafkaProducer::waitOnMetadata
```
This excludes only `KafkaProducer.waitOnMetadata` from C2 JIT compilation in
the test JVM, so it runs in the interpreter. It is preceded in
`extraJavaTestArgs` by the pre-existing `-XX:+IgnoreUnrecognizedVMOptions`, so
it is a no-op on JVMs that do not recognize or do not need it.
This is the `master` version so the fix can be backported down to
`branch-4.2` (where the failure is currently observed). Companion backport PR:
#56801.
### Why are the changes needed?
The Maven Kafka test job intermittently fails nearly the whole Kafka suite
(~227 tests) with:
```
java.lang.IllegalMonitorStateException: current thread is not owner
at java.base/java.lang.Object.wait(Native Method)
at org.apache.kafka.common.utils.SystemTime.waitObject(SystemTime.java:60)
at
org.apache.kafka.clients.producer.internals.ProducerMetadata.awaitUpdate(ProducerMetadata.java:123)
at
org.apache.kafka.clients.producer.KafkaProducer.waitOnMetadata(KafkaProducer.java:1180)
at
org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:1027)
...
```
`SystemTime.waitObject` in kafka-clients 3.9.2 calls `obj.wait()` correctly
inside `synchronized (obj)` (verified from the jar bytecode: `monitorenter` ->
`Object.wait` (line 60) -> `monitorexit`), so a correct JVM cannot throw
`IllegalMonitorStateException` here. This is the HotSpot **C2 JIT bug
[JDK-8298446](https://bugs.openjdk.org/browse/JDK-8298446)**: once C2 compiles
`KafkaProducer.waitOnMetadata`, the monitor on the synchronized `obj.wait()` is
dropped and every producer metadata wait throws "current thread is not owner".
It is triggered nondeterministically by whether C2 compiles that hot method
during the run, which is why the **identical** JDK fails on some runs and
passes on others.
CI evidence:
- Before (failing): https://github.com/apache/spark/actions/runs/28182892330
— scheduled `Build / Maven (branch-4.2, Scala 2.13, JDK 17)`, Kafka job
mass-fails with the stack above.
- After (this change, fork CI):
https://github.com/HyukjinKwon/spark/actions/runs/28220508364
### Does this PR introduce _any_ user-facing change?
No. This is a test-only JVM argument; it does not affect any Spark runtime
behavior or shipped artifact.
### How was this patch tested?
Existing Kafka test suites. The flag only forces one `kafka-clients`
producer method to run interpreted in the test JVM. Because the underlying C2
miscompilation is nondeterministic, the primary justification is the
JDK-8298446 analysis plus the `kafka-clients` bytecode (the `wait()` is
correctly guarded by `synchronized`); the linked fork CI run exercises the
change.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 4.8)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
