+1 to Dongjoon's proposal. I support Java 21 as the minimum version for Spark 5.0. I do not support Java 25 as the minimum.
Java 25 has useful improvements, including G1 region pinning and FFM support. These are good reasons to support Java 25, but not to require it. Java 21 is a mature LTS release. Many enterprise users need more time to upgrade their Java environments, libraries, and infrastructure. If we require Java 25, we will make the Spark 5.0 upgrade more difficult for these users. This could slow Spark 5.0 adoption. We should also keep the minimum supported Java version separate from the recommended version. Users who want Java 25 features can use Java 25. Other users should still be able to use Java 21. Dropping Java 17 already reduces our maintenance work. Java 21 gives us a good balance between new features and compatibility. My preference is to make Java 21 the minimum for Spark 5.0, fully support and test Java 25, and review the minimum version again for a future major release. Thanks, Xiao Dongjoon Hyun <[email protected]> 于2026年10月28日周三 15:44写道： > Hi, All. > > I'd like to propose dropping Java 17 support in Apache Spark 5.0.0, making > Java 21 > the minimum supported Java version. This is the same step we took in > Apache Spark > 4.0.0 for Java 8 and 11 (SPARK-44112). > > https://issues.apache.org/jira/browse/SPARK-60669 > https://github.com/apache/spark/pull/59897 > > Here are the reasons. > > 1. Java 21 is mature in our community. Apache Spark has officially > supported Java 21 > since 4.0.0 (May 2025), and our daily CI has tested Java 21 since June > 2023 > (SPARK-43835). > > 2. Supporting both Java 17 and 21 has a maintenance cost. Since the PR > builder uses > Java 17, we keep separate Java 21 SQL golden files > (`*.sql.out.java21`). They have > become stale after merges and had to be regenerated separately several > times > (SPARK-51937, SPARK-52400, SPARK-57560, SPARK-57649). We also keep Java > version-specific code paths and tests such as > `Utils.isJavaVersionAtLeast21`. > > 3. The ecosystem is moving on. For example, our Hive 4.2 metastore support > (SPARK-57263) requires Java 21, and newer dependency versions such as > Apache Derby 10.17 no longer support Java 17. > > 4. We can use Java 21 APIs and language features directly, e.g., virtual > threads > without reflection. > > This affects only the `master` branch (Apache Spark 5.0.0). `branch-4.x` > (Apache Spark 4.4.0) and the other 4.x branches keep Java 17 support, so > Java 17 > users can stay on the Apache Spark 4.x line. Apache Spark 5.0.0 will > support > Java 21 and 25. > > SPARK-60669 changes the minimum Java version in the build, the default > Java version > in GitHub Actions, and the documentation. I'll clean up the remaining Java > 17-specific code, tests, CI images, and release scripts in separate > PRs. > > Please let me know if you have any concerns. > > Thanks, > Dongjoon Hyun. > > --------------------------------------------------------------------- > To unsubscribe e-mail: [email protected] > >
