Re: Interesting implications of supporting Scala 2.13

2019-05-29 Thread Sean Owen
I think the particular issue here isn't resolved by scala-collection-compat: TraversableOnce goes away. However I hear that maybe Scala 2.13 retains it as a deprecated alias, which might help. On Wed, May 29, 2019 at 4:59 PM antonkulaga wrote: > > There is

Re: Interesting implications of supporting Scala 2.13

2019-05-29 Thread antonkulaga
There is https://github.com/scala/scala-collection-compat to enable 2.13 collections in Scala 2.12, so probably you can use it to avoid having separate source trees for 2.12 and 2.13 -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/

Re: Interesting implications of supporting Scala 2.13

2019-05-11 Thread Reynold Xin
If the number of changes that would require two source trees are small, another thing we can do is to reach out to the Scala team and kindly ask them whether they could change Scala 2.13 itself so it'd be easier to maintain compatibility with Scala 2.12. On Sat, May 11, 2019 at 4:25 PM, Sean

Re: Interesting implications of supporting Scala 2.13

2019-05-11 Thread Sean Owen
For those interested, here's the first significant problem I see that will require separate source trees or a breaking change: https://issues.apache.org/jira/browse/SPARK-27683?focusedCommentId=16837967=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16837967 Interested

Re: Interesting implications of supporting Scala 2.13

2019-05-10 Thread Reynold Xin
Yea my main point is when we do support 2.13, it'd be great if we don't have to break APIs. That's why doing the prep work in 3.0 would be great. On Fri, May 10, 2019 at 1:30 PM, Imran Rashid < iras...@cloudera.com > wrote: > > +1 on making whatever api changes we can now for 3.0. > > > I

Re: Interesting implications of supporting Scala 2.13

2019-05-10 Thread Imran Rashid
+1 on making whatever api changes we can now for 3.0. I don't think that is making any commitments to supporting scala 2.13 in any specific version. We'll have to deal with all the other points you raised when we do cross that bridge, but hopefully those are things we can cover in a minor

Re: Interesting implications of supporting Scala 2.13

2019-05-10 Thread Sean Owen
I really hope we don't have to have separate source trees for some files, but yeah it's an option too. OK, will start looking into changes we can make now that don't break things now, and deprecations we need to make now proactively. I should also say that supporting Scala 2.13 will mean

Re: Interesting implications of supporting Scala 2.13

2019-05-10 Thread Reynold Xin
Looks like a great idea to make changes in Spark 3.0 to prepare for Scala 2.13 upgrade. Are there breaking changes that would require us to have two different source code for 2.12 vs 2.13? On Fri, May 10, 2019 at 11:41 AM, Sean Owen < sro...@gmail.com > wrote: > > > > While that's not

Interesting implications of supporting Scala 2.13

2019-05-10 Thread Sean Owen
While that's not happening soon (2.13 isn't out), note that some of the changes to collections will be fairly breaking changes. https://issues.apache.org/jira/browse/SPARK-25075 https://docs.scala-lang.org/overviews/core/collections-migration-213.html Some of this may impact a public API, so may