Re: Time for Spark 3.3.1 release?

Although it's irrelevant to Apache Spark 3.3.1 release discussion because 3.3.1 is a maintenance release to 3.3.0, you may want to lead it for Apache Spark 3.4 in a separate thread. For your info, Apache Spark 3.3.1 RC1 does not include Hadoop 3.3.4 either. Previously, since we don't want to

Re: Jupyter notebook on Dataproc versus GKE

Mitch: Why I'm switching from Jupyter Notebooks to JupyterLab...Such a better experience! DegreeTutors.com tir. 6. sep. 2022 kl. 20:28 skrev Holden Karau : > I’ve used Argo for K8s scheduling, for awhile it’s also what Kubeflow used > underneath for scheduling. > >

Re: Time for Spark 3.3.1 release?

At least we should upgrade hadoop to the latest version https://hadoop.apache.org/release/2.10.2.html Are there some spesial reasons why we have a hadoop version that is 7 years old? ons. 14. sep. 2022, 20:25 skrev Dongjoon Hyun : > Ya, +1 for Sean's comment. > > In addition, all Apache Spark's

Re: Time for Spark 3.3.1 release?

Ya, +1 for Sean's comment. In addition, all Apache Spark's Maven artifacts are depending on Hadoop 3.3.x already. https://mvnrepository.com/artifact/org.apache.spark/spark-core_2.12/3.3.0 https://mvnrepository.com/artifact/org.apache.spark/spark-core_2.13/3.3.0 Apache Spark has been moving

Re: Time for Spark 3.3.1 release?

Yeah we're not going to make convenience binaries for all possible combinations. It's a pretty good assumption that anyone moving to later Scala versions is also off old Hadoop versions. You can of course build the combo you like. On Wed, Sep 14, 2022 at 11:26 AM Denis Bolshakov wrote: >

Re: Time for Spark 3.3.1 release?

Unfortunately it's for hadoop 3 only. ср, 14 сент. 2022 г., 19:04 Dongjoon Hyun : > Hi, Denis. > > Apache Spark community already provides both Scala 2.12 and 2.13 pre-built > distributions. > Please check the distribution site and Apache Spark download page. > >

Re: Time for Spark 3.3.1 release?

Hi, Denis. Apache Spark community already provides both Scala 2.12 and 2.13 pre-built distributions. Please check the distribution site and Apache Spark download page. https://dlcdn.apache.org/spark/spark-3.3.0/ spark-3.3.0-bin-hadoop3-scala2.13.tgz spark-3.3.0-bin-hadoop3.tgz [image:

Re: Time for Spark 3.3.1 release?

Hello, It would be great if it's possible to provide a spark distro for both scala 2.12 and scala 2.13. It will encourage spark users to switch to scala 2.13. I know that spark jar artifacts available for both scala versions, but it does not make sense to migrate to scala 2.13 while there is no