Re: [FYI] Kubernetes GA Preparation (SPARK-33005)

2020-10-03 Thread German Schiavon
Hi! I just run to this same issue while testing k8s in local mode https://issues.apache.org/jira/browse/SPARK-31800 Note that the tittle shouldn't be "*Unable to disable Kerberos when submitting jobs to Kubernetes" *(based on the comments) and something more related with the

Re: Apache Spark 3.1 Preparation Status (Oct. 2020)

2020-10-03 Thread Xiao Li
Thank you for your updates. Spark 3.0 got released on Jun 18, 2020. If Nov 1st is the target date of the 3.1 branch cut, the feature development time window is less than 5 months. This is shorter than what we did in Spark 2.3 and 2.4 releases. Below are three highly desirable feature work I am

Re: Apache Spark 3.1 Preparation Status (Oct. 2020)

2020-10-03 Thread Hyukjin Kwon
Nice summary. Thanks Dongjoon. One minor correction -> I believe we dropped R 3.5 and below at branch 2.4 as well. On Sun, 4 Oct 2020, 09:17 Dongjoon Hyun, wrote: > Hi, All. > > As of today, master branch (Apache Spark 3.1.0) resolved > 852+ JIRA issues and 606+ issues are 3.1.0-only patches. >

Apache Spark 3.1 Preparation Status (Oct. 2020)

2020-10-03 Thread Dongjoon Hyun
Hi, All. As of today, master branch (Apache Spark 3.1.0) resolved 852+ JIRA issues and 606+ issues are 3.1.0-only patches. According to the 3.1.0 release window, branch-3.1 will be created on November 1st and enters QA period. Here are some notable updates I've been monitoring. *Language* 01.