Re: Revisiting the idea of a Spark 2.5 transitional release

2020-06-13 Thread Sean Owen
These two are coupled, and in tension: don't want to take much change, but do want changes that will unfortunately be somewhat breaking. A 2.5 release with these items would be different enough as to strain the general level of compatibility implied by a minor release. Sure, it's not 'just' a

Re: Revisiting the idea of a Spark 2.5 transitional release

2020-06-13 Thread DB Tsai
For example, JDK11 requires dependency changes which can not go into 2.4.7. Recent development on Kube such as supporting dynamical allocation in Spark 3.0 in Kube (without shuffle service) will be hard to go in 2.4.7. Sent from my iPhone > On Jun 12, 2020, at 11:50 PM, Reynold Xin wrote: >

Re: Revisiting the idea of a Spark 2.5 transitional release

2020-06-13 Thread Reynold Xin
Echoing Sean's earlier comment … What is the functionality that would go into a 2.5.0 release, that can't be in a 2.4.7 release? On Fri, Jun 12, 2020 at 11:14 PM, Holden Karau < hol...@pigscanfly.ca > wrote: > > Can I suggest we maybe decouple this conversation a bit? First, if there > is an

Re: Revisiting the idea of a Spark 2.5 transitional release

2020-06-13 Thread Holden Karau
Can I suggest we maybe decouple this conversation a bit? First, if there is an agreement in making a transitional release in principle and then folks who feel strongly about specific backports can have their respective discussions.It's not like we normally know or have agreement on everything