I didn't realize that Java 11 would require breaking changes. What breaking changes are required?
On Fri, Sep 20, 2019 at 11:18 AM Sean Owen <sro...@gmail.com> wrote: > Narrowly on Java 11: the problem is that it'll take some breaking > changes, more than would be usually appropriate in a minor release, I > think. I'm still not convinced there is a burning need to use Java 11 > but stay on 2.4, after 3.0 is out, and at least the wheels are in > motion there. Java 8 is still free and being updated. > > On Fri, Sep 20, 2019 at 12:48 PM Ryan Blue <rb...@netflix.com.invalid> > wrote: > > > > Hi everyone, > > > > In the DSv2 sync this week, we talked about a possible Spark 2.5 release > based on the latest Spark 2.4, but with DSv2 and Java 11 support added. > > > > A Spark 2.5 release with these two additions will help people migrate to > Spark 3.0 when it is released because they will be able to use a single > implementation for DSv2 sources that works in both 2.5 and 3.0. Similarly, > upgrading to 3.0 won't also require also updating to Java 11 because users > could update to Java 11 with the 2.5 release and have fewer major changes. > > > > Another reason to consider a 2.5 release is that many people are > interested in a release with the latest DSv2 API and support for DSv2 SQL. > I'm already going to be backporting DSv2 support to the Spark 2.4 line, so > it makes sense to share this work with the community. > > > > This release line would just consist of backports like DSv2 and Java 11 > that assist compatibility, to keep the scope of the release small. The > purpose is to assist people moving to 3.0 and not distract from the 3.0 > release. > > > > Would a Spark 2.5 release help anyone else? Are there any concerns about > this plan? > > > > > > rb > > > > > > -- > > Ryan Blue > > Software Engineer > > Netflix > -- Ryan Blue Software Engineer Netflix
