+1 for Matei's point. On Thursday, October 27, 2016, Matei Zaharia <matei.zaha...@gmail.com> wrote:
> Just to comment on this, I'm generally against removing these types of > things unless they create a substantial burden on project contributors. It > doesn't sound like Python 2.6 and Java 7 do that yet -- Scala 2.10 might, > but then of course we need to wait for 2.12 to be out and stable. > > In general, this type of stuff only hurts users, and doesn't have a huge > impact on Spark contributors' productivity (sure, it's a bit unpleasant, > but that's life). If we break compatibility this way too quickly, we > fragment the user community, and then either people have a crappy > experience with Spark because their corporate IT doesn't yet have an > environment that can run the latest version, or worse, they create more > maintenance burden for us because they ask for more patches to be > backported to old Spark versions (1.6.x, 2.0.x, etc). Python in particular > is pretty fundamental to many Linux distros. > > In the future, rather than just looking at when some software came out, it > may be good to have some criteria for when to drop support for something. > For example, if there are really nice libraries in Python 2.7 or Java 8 > that we're missing out on, that may be a good reason. The maintenance > burden for multiple Scala versions is definitely painful but I also think > we should always support the latest two Scala releases. > > Matei > > On Oct 27, 2016, at 12:15 PM, Reynold Xin <r...@databricks.com > <javascript:_e(%7B%7D,'cvml','r...@databricks.com');>> wrote: > > I created a JIRA ticket to track this: https://issues.apache. > org/jira/browse/SPARK-18138 > > > > On Thu, Oct 27, 2016 at 10:19 AM, Steve Loughran <ste...@hortonworks.com > <javascript:_e(%7B%7D,'cvml','ste...@hortonworks.com');>> wrote: > >> >> On 27 Oct 2016, at 10:03, Sean Owen <so...@cloudera.com >> <javascript:_e(%7B%7D,'cvml','so...@cloudera.com');>> wrote: >> >> Seems OK by me. >> How about Hadoop < 2.6, Python 2.6? Those seem more removeable. I'd like >> to add that to a list of things that will begin to be unsupported 6 months >> from now. >> >> >> If you go to java 8 only, then hadoop 2.6+ is mandatory. >> >> >> On Wed, Oct 26, 2016 at 8:49 PM Koert Kuipers <ko...@tresata.com >> <javascript:_e(%7B%7D,'cvml','ko...@tresata.com');>> wrote: >> >>> that sounds good to me >>> >>> On Wed, Oct 26, 2016 at 2:26 PM, Reynold Xin <r...@databricks.com >>> <javascript:_e(%7B%7D,'cvml','r...@databricks.com');>> wrote: >>> >>>> We can do the following concrete proposal: >>>> >>>> 1. Plan to remove support for Java 7 / Scala 2.10 in Spark 2.2.0 >>>> (Mar/Apr 2017). >>>> >>>> 2. In Spark 2.1.0 release, aggressively and explicitly announce the >>>> deprecation of Java 7 / Scala 2.10 support. >>>> >>>> (a) It should appear in release notes, documentations that mention how >>>> to build Spark >>>> >>>> (b) and a warning should be shown every time SparkContext is started >>>> using Scala 2.10 or Java 7. >>>> >>>> >> > >
