Hi, Manu. Apache Spark 4.0.0 is designed to drop all Java versions (< JDK 17) in all aspects (Source Code, Binary, Dependency).
Since there are a bunch of related stuff, please refer to the umbrella JIRA issues: SPARK-45315 Drop JDK 8/11 and make JDK 17 by default (4.0.0) SPARK-53047 Modernize Spark to leverage the latest Java features (4.1.0) To Hyukjin, there is no reason to be blocked. Thanks, Dongjoon. On Fri, Sep 26, 2025 at 5:34 PM Hyukjin Kwon <[email protected]> wrote: > I think it was by > https://github.com/apache/spark/commit/6ff9edcaf16c90007508f15de98fac361e234381 > > Adding Daniel and Chris. > > Let's don't block it the preview tho. We will have another one next month > (Nov) > > On Sat, 27 Sept 2025 at 09:08, Manu Zhang <[email protected]> wrote: > >> Hi all, >> >> I'm integrating Spark 4.1(preview2-rc1) with Iceberg [1] and have to >> change target byte code level from 11 to 17. Otherwise, Scala compile would >> fail with following error. It compiled fine with preview1. >> >> "spark/v4.1/spark/src/main/scala/org/apache/spark/sql/stats/ThetaSketchAgg.scala:52:12: >> Class java.lang.Record not found" >> >> May I know which change could have caused this between preview1 and >> preview2-rc1? >> >> 1. https://github.com/apache/iceberg/pull/14155 >> >> Thanks, >> Manu >> >> >> Cheng Pan <[email protected]>于2025年9月26日 周五17:23写道： >> >>> +1 (non-binding) >>> >>> I tested it with Hadoop 3.4.2, in both SIMPLE and KERBEROS mode - >>> SparkPi, Connect Server, and History Server work as expected. >>> >>> Thanks, >>> Cheng Pan >>> >>> On Fri, Sep 26, 2025 at 4:56 PM Max Gekk <[email protected]> wrote: >>> > >>> > +1 >>> > >>> > On Wed, Sep 24, 2025 at 3:49 PM Wenchen Fan <[email protected]> >>> wrote: >>> >> >>> >> +1 >>> >> >>> >> On Wed, Sep 24, 2025 at 7:29 PM <[email protected]> wrote: >>> >>> >>> >>> Please vote on releasing the following candidate as Apache Spark >>> version 4.1.0-preview2. >>> >>> >>> >>> The vote is open until Sat, 27 Sep 2025 05:26:22 PDT and passes if a >>> majority +1 PMC votes are cast, with >>> >>> a minimum of 3 +1 votes. >>> >>> >>> >>> [ ] +1 Release this package as Apache Spark 4.1.0-preview2 >>> >>> [ ] -1 Do not release this package because ... >>> >>> >>> >>> To learn more about Apache Spark, please see >>> https://spark.apache.org/ >>> >>> >>> >>> The tag to be voted on is v4.1.0-preview2-rc1 (commit c5ff48cc2b2): >>> >>> https://github.com/apache/spark/tree/v4.1.0-preview2-rc1 >>> >>> >>> >>> The release files, including signatures, digests, etc. can be found >>> at: >>> >>> >>> https://dist.apache.org/repos/dist/dev/spark/v4.1.0-preview2-rc1-bin/ >>> >>> >>> >>> Signatures used for Spark RCs can be found in this file: >>> >>> https://downloads.apache.org/spark/KEYS >>> >>> >>> >>> The staging repository for this release can be found at: >>> >>> >>> https://repository.apache.org/content/repositories/orgapachespark-1503/ >>> >>> >>> >>> The documentation corresponding to this release can be found at: >>> >>> >>> https://dist.apache.org/repos/dist/dev/spark/v4.1.0-preview2-rc1-docs/ >>> >>> >>> >>> The list of bug fixes going into 4.1.0-preview2 can be found at the >>> following URL: >>> >>> https://issues.apache.org/jira/projects/SPARK/versions/12355581 >>> >>> >>> >>> FAQ >>> >>> >>> >>> ========================= >>> >>> How can I help test this release? >>> >>> ========================= >>> >>> >>> >>> If you are a Spark user, you can help us test this release by taking >>> >>> an existing Spark workload and running on this release candidate, >>> then >>> >>> reporting any regressions. >>> >>> >>> >>> If you're working in PySpark you can set up a virtual env and install >>> >>> the current RC via "pip install >>> https://dist.apache.org/repos/dist/dev/spark/v4.1.0-preview2-rc1-bin/pyspark-4.1.0.dev2.tar.gz >>> " >>> >>> and see if anything important breaks. >>> >>> In the Java/Scala, you can add the staging repository to your >>> project's resolvers and test >>> >>> with the RC (make sure to clean up the artifact cache before/after so >>> >>> you don't end up building with an out of date RC going forward). >>> >>> >>> >>> --------------------------------------------------------------------- >>> >>> To unsubscribe e-mail: [email protected] >>> >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe e-mail: [email protected] >>> >>>
