Re: Apache Spark 3.1 Preparation Status (Oct. 2020)

2020-10-04 Thread Xiao Li
As pointed out by Dongjoon, the 2nd half of December is the holiday season in most countries. If we do the code freeze in mid November and release the first RC in mid December. I am afraid the community will not be active to verify the release candidates during the holiday season. Normally, the RC

New ASF guidelines for docker images

2020-10-04 Thread Holden Karau
Hi Folks, It seems like there are new guidelines for incubator projects that might make it easier for us to publish Spark docker images. While we are not an incubator project, I think this is probably a sign we should revisit publishing docker images. Cheers, Holden -- Forwarded

Re: Apache Spark 3.1 Preparation Status (Oct. 2020)

2020-10-04 Thread Xiao Li
> > Apache Spark 3.1.0 should be compared with Apache Spark 2.1.0. I think we made a change in release cadence since Spark 2.3. See the commit: https://github.com/apache/spark-website/commit/88990968962e5cc47db8bc2c11a50742d2438daa Thus, Spark 3.1 might just follow the release cadence of Spark

Re: Apache Spark 3.1 Preparation Status (Oct. 2020)

2020-10-04 Thread Dongjoon Hyun
For Xiao's comment, I want to point out that Apache Spark 3.1.0 is different from 2.3 or 2.4. Apache Spark 3.1.0 should be compared with Apache Spark 2.1.0. - Apache Spark 2.0.0 was released on July 26, 2016. - Apache Spark 2.1.0 was released on December 28, 2016. Bests, Dongjoon. On Sun, Oct

Re: [FYI] Kubernetes GA Preparation (SPARK-33005)

2020-10-04 Thread Dongjoon Hyun
Sure, German. Please add your comment on that issue. If possible, please provide a reproducible example which you did. On Sat, Oct 3, 2020 at 10:31 PM German Schiavon wrote: > Hi! > I just run to this same issue while testing k8s in local mode > >

Re: Apache Spark 3.1 Preparation Status (Oct. 2020)

2020-10-04 Thread Dongjoon Hyun
Thank you all. BTW, Xiao and Mridul, I'm wondering what date you have in your mind specifically. Usually, `Christmas and New Year season` doesn't give us much additional time. If you think so, could you make a PR for Apache Spark website according to your expectation?

Re: Apache Spark 3.1 Preparation Status (Oct. 2020)

2020-10-04 Thread Mridul Muralidharan
+1 on pushing the branch cut for increased dev time to match previous releases. Regards, Mridul On Sat, Oct 3, 2020 at 10:22 PM Xiao Li wrote: > Thank you for your updates. > > Spark 3.0 got released on Jun 18, 2020. If Nov 1st is the target date of > the 3.1 branch cut, the feature