(Here are a few that have already been flagged for 3.0: https://issues.apache.org/jira/browse/SPARK-22236?jql=project%20%3D%20SPARK%20AND%20%22Target%20Version%2Fs%22%20%20%3D%203.0.0 )
On Fri, Jan 19, 2018 at 11:43 AM Holden Karau <holden.ka...@gmail.com> wrote: > I think an interesting exercise would be to consider what changes we are > putting off for a major version and if they make enough of a change to > warrent the work involved or keep pushing it off. > > Personally the first thing that comes to mind is I'd like to revisit the > accumulator APIs again and see if we can do something with then. What's top > of everyone else's mind? > > On Jan 20, 2018 6:32 AM, "Sean Owen" <so...@cloudera.com> wrote: > >> Forking this thread to muse about Spark 3. Like Spark 2, I assume it >> would be more about making all those accumulated breaking changes and >> updating lots of dependencies. Hadoop 3 looms large in that list as well as >> Scala 2.12. >> >> Spark 1 was release in May 2014, and Spark 2 in July 2016. If Spark 2.3 >> is out in Feb 2018 and it takes the now-usual 6 months until a next >> release, Spark 3 could reasonably be next. >> >> However the release cycles are naturally slowing down, and it could also >> be said that 2019 would be more on schedule for Spark 3. >> >> Nothing particularly urgent about deciding, but I'm curious if anyone had >> an opinion on whether to move on to Spark 3 next or just continue with 2.4 >> later this year. >> >> On Fri, Jan 19, 2018 at 11:13 AM Sean Owen <so...@cloudera.com> wrote: >> >>> Yeah, if users are using Kryo directly, they should be insulated from a >>> Spark-side change because of shading. >>> However this also entails updating (unshaded) Chill from 0.8.x to 0.9.x. >>> I am not sure if that causes problems for apps. >>> >>> Normally I'd avoid any major-version change in a minor release. This one >>> looked potentially entirely internal. >>> I think if there are any doubts, we can leave it for Spark 3. There was >>> a bug report that needed a fix from Kryo 4, but it might be minor after all. >>> >>>> >>>>
