Qianlan Chen created SPARK-59821:
------------------------------------
Summary: Make Maven utility tests safe for concurrent runs
Key: SPARK-59821
URL: https://issues.apache.org/jira/browse/SPARK-59821
Project: Spark
Issue Type: Test
Components: Spark Core
Affects Versions: 4.4.0
Reporter: Qianlan Chen
Maven utility tests are not fully isolated when multiple test JVMs run
concurrently.
SparkTestUtils.createCompiledClass writes generated class files to the process
working directory before moving them to the requested destination. Concurrent
compilations of the same class name can therefore overwrite or move each
other's output.
MavenUtils also uses the fixed dummy/.m2/repository path during tests. One test
JVM can clean this repository while another JVM is resolving dependencies.
This race is exposed by high-concurrency flakiness runs, where existing
MavenUtilsSuite cases fail intermittently even though normal master CI runs are
generally green. It is a test-infrastructure race rather than a Maven
resolution product bug. The issue predates SPARK-59553 and was only exposed
while validating that change.
The tests should:
* Compile generated classes in invocation-specific temporary directories.
* Use a JVM-specific but stable temporary Maven repository.
* Include a regression test that compiles the same class name concurrently.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
