as long as we don't lock ourselves into supporting scala 2.10 for the entire spark 2 lifespan it sounds reasonable to me
On Wed, Mar 30, 2016 at 3:25 PM, Michael Armbrust <mich...@databricks.com> wrote: > +1 to Matei's reasoning. > > On Wed, Mar 30, 2016 at 9:21 AM, Matei Zaharia <matei.zaha...@gmail.com> > wrote: > >> I agree that putting it in 2.0 doesn't mean keeping Scala 2.10 for the >> entire 2.x line. My vote is to keep Scala 2.10 in Spark 2.0, because it's >> the default version we built with in 1.x. We want to make the transition >> from 1.x to 2.0 as easy as possible. In 2.0, we'll have the default >> downloads be for Scala 2.11, so people will more easily move, but we >> shouldn't create obstacles that lead to fragmenting the community and >> slowing down Spark 2.0's adoption. I've seen companies that stayed on an >> old Scala version for multiple years because switching it, or mixing >> versions, would affect the company's entire codebase. >> >> Matei >> >> On Mar 30, 2016, at 12:08 PM, Koert Kuipers <ko...@tresata.com> wrote: >> >> oh wow, had no idea it got ripped out >> >> On Wed, Mar 30, 2016 at 11:50 AM, Mark Hamstra <m...@clearstorydata.com> >> wrote: >> >>> No, with 2.0 Spark really doesn't use Akka: >>> https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/SparkConf.scala#L744 >>> >>> On Wed, Mar 30, 2016 at 9:10 AM, Koert Kuipers <ko...@tresata.com> >>> wrote: >>> >>>> Spark still runs on akka. So if you want the benefits of the latest >>>> akka (not saying we do, was just an example) then you need to drop scala >>>> 2.10 >>>> On Mar 30, 2016 10:44 AM, "Cody Koeninger" <c...@koeninger.org> wrote: >>>> >>>>> I agree with Mark in that I don't see how supporting scala 2.10 for >>>>> spark 2.0 implies supporting it for all of spark 2.x >>>>> >>>>> Regarding Koert's comment on akka, I thought all akka dependencies >>>>> have been removed from spark after SPARK-7997 and the recent removal >>>>> of external/akka >>>>> >>>>> On Wed, Mar 30, 2016 at 9:36 AM, Mark Hamstra <m...@clearstorydata.com> >>>>> wrote: >>>>> > Dropping Scala 2.10 support has to happen at some point, so I'm not >>>>> > fundamentally opposed to the idea; but I've got questions about how >>>>> we go >>>>> > about making the change and what degree of negative consequences we >>>>> are >>>>> > willing to accept. Until now, we have been saying that 2.10 support >>>>> will be >>>>> > continued in Spark 2.0.0. Switching to 2.11 will be non-trivial for >>>>> some >>>>> > Spark users, so abruptly dropping 2.10 support is very likely to >>>>> delay >>>>> > migration to Spark 2.0 for those users. >>>>> > >>>>> > What about continuing 2.10 support in 2.0.x, but repeatedly making an >>>>> > obvious announcement in multiple places that such support is >>>>> deprecated, >>>>> > that we are not committed to maintaining it throughout 2.x, and that >>>>> it is, >>>>> > in fact, scheduled to be removed in 2.1.0? >>>>> > >>>>> > On Wed, Mar 30, 2016 at 7:45 AM, Sean Owen <so...@cloudera.com> >>>>> wrote: >>>>> >> >>>>> >> (This should fork as its own thread, though it began during >>>>> discussion >>>>> >> of whether to continue Java 7 support in Spark 2.x.) >>>>> >> >>>>> >> Simply: would like to more clearly take the temperature of all >>>>> >> interested parties about whether to support Scala 2.10 in the Spark >>>>> >> 2.x lifecycle. Some of the arguments appear to be: >>>>> >> >>>>> >> Pro >>>>> >> - Some third party dependencies do not support Scala 2.11+ yet and >>>>> so >>>>> >> would not be usable in a Spark app >>>>> >> >>>>> >> Con >>>>> >> - Lower maintenance overhead -- no separate 2.10 build, >>>>> >> cross-building, tests to check, esp considering support of 2.12 will >>>>> >> be needed >>>>> >> - Can use 2.11+ features freely >>>>> >> - 2.10 was EOL in late 2014 and Spark 2.x lifecycle is years to come >>>>> >> >>>>> >> I would like to not support 2.10 for Spark 2.x, myself. >>>>> >> >>>>> >> >>>>> --------------------------------------------------------------------- >>>>> >> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org >>>>> >> For additional commands, e-mail: dev-h...@spark.apache.org >>>>> >> >>>>> > >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org >>>>> For additional commands, e-mail: dev-h...@spark.apache.org >>>>> >>>>> >>> >> >> >
