+1 On Wed, Oct 28, 2026 at 3:45 PM Dongjoon Hyun <[email protected]> wrote:
> Hi, All. > > I'd like to propose dropping Java 17 support in Apache Spark 5.0.0, making > Java 21 > the minimum supported Java version. This is the same step we took in > Apache Spark > 4.0.0 for Java 8 and 11 (SPARK-44112). > > https://issues.apache.org/jira/browse/SPARK-60669 > https://github.com/apache/spark/pull/59897 > > Here are the reasons. > > 1. Java 21 is mature in our community. Apache Spark has officially > supported Java 21 > since 4.0.0 (May 2025), and our daily CI has tested Java 21 since June > 2023 > (SPARK-43835). > > 2. Supporting both Java 17 and 21 has a maintenance cost. Since the PR > builder uses > Java 17, we keep separate Java 21 SQL golden files > (`*.sql.out.java21`). They have > become stale after merges and had to be regenerated separately several > times > (SPARK-51937, SPARK-52400, SPARK-57560, SPARK-57649). We also keep Java > version-specific code paths and tests such as > `Utils.isJavaVersionAtLeast21`. > > 3. The ecosystem is moving on. For example, our Hive 4.2 metastore support > (SPARK-57263) requires Java 21, and newer dependency versions such as > Apache Derby 10.17 no longer support Java 17. > > 4. We can use Java 21 APIs and language features directly, e.g., virtual > threads > without reflection. > > This affects only the `master` branch (Apache Spark 5.0.0). `branch-4.x` > (Apache Spark 4.4.0) and the other 4.x branches keep Java 17 support, so > Java 17 > users can stay on the Apache Spark 4.x line. Apache Spark 5.0.0 will > support > Java 21 and 25. > > SPARK-60669 changes the minimum Java version in the build, the default > Java version > in GitHub Actions, and the documentation. I'll clean up the remaining Java > 17-specific code, tests, CI images, and release scripts in separate > PRs. > > Please let me know if you have any concerns. > > Thanks, > Dongjoon Hyun. > > --------------------------------------------------------------------- > To unsubscribe e-mail: [email protected] > >
