+1 On Thu, 29 Oct 2026 at 09:07, Peter Toth <[email protected]> wrote:
> +1 > > On Wed, Oct 28, 2026 at 11:44 PM Dongjoon Hyun <[email protected]> > wrote: > > > > Hi, All. > > > > I'd like to propose dropping Java 17 support in Apache Spark 5.0.0, > making Java 21 > > the minimum supported Java version. This is the same step we took in > Apache Spark > > 4.0.0 for Java 8 and 11 (SPARK-44112). > > > > https://issues.apache.org/jira/browse/SPARK-60669 > > https://github.com/apache/spark/pull/59897 > > > > Here are the reasons. > > > > 1. Java 21 is mature in our community. Apache Spark has officially > supported Java 21 > > since 4.0.0 (May 2025), and our daily CI has tested Java 21 since > June 2023 > > (SPARK-43835). > > > > 2. Supporting both Java 17 and 21 has a maintenance cost. Since the PR > builder uses > > Java 17, we keep separate Java 21 SQL golden files > (`*.sql.out.java21`). They have > > become stale after merges and had to be regenerated separately > several times > > (SPARK-51937, SPARK-52400, SPARK-57560, SPARK-57649). We also keep > Java > > version-specific code paths and tests such as > `Utils.isJavaVersionAtLeast21`. > > > > 3. The ecosystem is moving on. For example, our Hive 4.2 metastore > support > > (SPARK-57263) requires Java 21, and newer dependency versions such as > > Apache Derby 10.17 no longer support Java 17. > > > > 4. We can use Java 21 APIs and language features directly, e.g., virtual > threads > > without reflection. > > > > This affects only the `master` branch (Apache Spark 5.0.0). `branch-4.x` > > (Apache Spark 4.4.0) and the other 4.x branches keep Java 17 support, so > Java 17 > > users can stay on the Apache Spark 4.x line. Apache Spark 5.0.0 will > support > > Java 21 and 25. > > > > SPARK-60669 changes the minimum Java version in the build, the default > Java version > > in GitHub Actions, and the documentation. I'll clean up the remaining > Java 17-specific code, tests, CI images, and release scripts in separate > > PRs. > > > > Please let me know if you have any concerns. > > > > Thanks, > > Dongjoon Hyun. > > > > --------------------------------------------------------------------- > > To unsubscribe e-mail: [email protected] > > > > --------------------------------------------------------------------- > To unsubscribe e-mail: [email protected] > >
