dongjoon-hyun opened a new pull request, #60046:
URL: https://github.com/apache/spark/pull/60046
### What changes were proposed in this pull request?
This PR aims to update `BenchmarkBase` to use `Java 21` as the base version.
- `BenchmarkBase` adds the JDK major version to the result file name only
for JDK 22+ (`version > 21`), so Java 21 results are written to `*-results.txt`.
- Remove the Java 17 results (`*-results.txt`, 126 files) and rename the
Java 21 results (`*-jdk21-results.txt`, 124 files) to `*-results.txt`. The file
contents are not changed.
- `HighlyCompressedMapStatusBenchmark` and `PipelinedShuffleBenchmark` had
only Java 21 results.
- `RTMKafka*Benchmark` (4 files) had only Java 17 results, so they have no
result file after this PR.
- Update the result file names and JDK versions in the doc comments of
`DecimalAggregatesBenchmark`, `PipelinedShuffleBenchmark`,
`SharedJsonParseBenchmark`, and in `docs/sql-arrow-cache-format.md`.
- Java 25 results (`*-jdk25-results.txt`) are not changed.
### Why are the changes needed?
Apache Spark 5.0.0 drops Java 17 support via SPARK-60669. After that, Java
17 results can not be regenerated and Java 21 is the base version.
```
$ ls core/benchmarks/KryoBenchmark*
core/benchmarks/KryoBenchmark-jdk25-results.txt
core/benchmarks/KryoBenchmark-results.txt
```
### Does this PR introduce _any_ user-facing change?
No. This is a test-only change.
### How was this patch tested?
Manually run a benchmark with Java 21 and Java 25.
```
$ JAVA_HOME=$(/usr/libexec/java_home -v 21) SPARK_GENERATE_BENCHMARK_FILES=1
\
build/sbt "core/Test/runMain
org.apache.spark.util.collection.PercentileHeapBenchmark"
$ head -5 core/benchmarks/PercentileHeapBenchmark-results.txt | grep VM
OpenJDK 64-Bit Server VM 21.0.13+9-LTS on Mac OS X 27.3
$ JAVA_HOME=$(/usr/libexec/java_home -v 25) SPARK_GENERATE_BENCHMARK_FILES=1
\
build/sbt "core/Test/runMain
org.apache.spark.util.collection.PercentileHeapBenchmark"
$ head -5 core/benchmarks/PercentileHeapBenchmark-jdk25-results.txt | grep VM
OpenJDK 64-Bit Server VM 25.0.5+8-LTS on Mac OS X 27.3
```
In addition, all `*-results.txt` files without a JDK suffix are generated by
Java 21 now.
```
$ git ls-files | grep -E "benchmarks/[^/]*-results\.txt$" | grep -vE --
"-jdk[0-9]+" | \
xargs grep -h -m1 -oE "VM [0-9]+" | sort | uniq -c
123 VM 21
```
`WholeStageCodegenSizeBenchmark-results.txt` has no JVM header.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 5.5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
