Hi, Jason. Apache Spark 4.0.0 depends on only Apache Hadoop client library.
You can track all `Apache Spark 4` activities including Hadoop dependency here. https://issues.apache.org/jira/browse/SPARK-44111 (Prepare Apache Spark 4.0.0) According to the release history, the original suggested timeline was June, 2024. - Spark 1: 2014.05 (1.0.0) ~ 2016.11 (1.6.3) - Spark 2: 2016.07 (2.0.0) ~ 2021.05 (2.4.8) - Spark 3: 2020.06 (3.0.0) ~ 2026.xx (3.5.x) - Spark 4: 2024.06 (4.0.0, NEW) Thanks, Dongjoon. On 2023/12/08 23:50:15 Jason Xu wrote: > Hi Spark devs, > > According to the Spark 3.5 release notes, Spark 4 will no longer support > Java 8 and 11 (link > <https://spark.apache.org/releases/spark-release-3-5-0.html#upcoming-removal> > ). > > My company is using Spark on Yarn with Java 8 now. When considering a > future upgrade to Spark 4, one issue we face is that the latest version of > Hadoop (3.3) does not yet support Java 17. There is an open ticket ( > HADOOP-17177 <https://issues.apache.org/jira/browse/HADOOP-17177>) for this > issue, which has been open for over two years. > > My question is: Does the release of Spark 4 depend on the availability of > Java 17 support in Hadoop? Additionally, do we have a rough estimate for > the release of Spark 4? Thanks! > > > Cheers, > > Jason Xu > --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
