Removing compatibility (with jdk, etc) can be done with a major release- given that 7 has been EOLed a while back and is now unsupported, we have to decide if we drop support for it in 2.0 or 3.0 (2+ years from now).
Given the functionality & performance benefits of going to jdk8, future enhancements relevant in 2.x timeframe ( scala, dependencies) which requires it, and simplicity wrt code, test & support it looks like a good checkpoint to drop jdk7 support. As already mentioned in the thread, existing yarn clusters are unaffected if they want to continue running jdk7 and yet use spark2 (install jdk8 on all nodes and use it via JAVA_HOME, or worst case distribute jdk8 as archive - suboptimal). I am unsure about mesos (standalone might be easier upgrade I guess ?). Proposal is for 1.6x line to continue to be supported with critical fixes; newer features will require 2.x and so jdk8 Regards Mridul On Thursday, March 24, 2016, Marcelo Vanzin <van...@cloudera.com> wrote: > On Thu, Mar 24, 2016 at 4:50 PM, Reynold Xin <r...@databricks.com > <javascript:;>> wrote: > > If you want to go down that route, you should also ask somebody who has > had > > experience managing a large organization's applications and try to update > > Scala version. > > I understand both sides. But if you look at what I've been asking > since the beginning, it's all about the cost and benefits of dropping > support for java 1.7. > > The biggest argument in your original e-mail is about testing. And the > testing cost is much bigger for supporting scala 2.10 than it is for > supporting java 1.7. If you read one of my earlier replies, it should > be even possible to just do everything in a single job - compile for > java 7 and still be able to test things in 1.8, including lambdas, > which seems to be the main thing you were worried about. > > > > On Thu, Mar 24, 2016 at 4:48 PM, Marcelo Vanzin <van...@cloudera.com > <javascript:;>> wrote: > >> > >> On Thu, Mar 24, 2016 at 4:46 PM, Reynold Xin <r...@databricks.com > <javascript:;>> wrote: > >> > Actually it's *way* harder to upgrade Scala from 2.10 to 2.11, than > >> > upgrading the JVM runtime from 7 to 8, because Scala 2.10 and 2.11 are > >> > not > >> > binary compatible, whereas JVM 7 and 8 are binary compatible except > >> > certain > >> > esoteric cases. > >> > >> True, but ask anyone who manages a large cluster how long it would > >> take them to upgrade the jdk across their cluster and validate all > >> their applications and everything... binary compatibility is a tiny > >> drop in that bucket. > >> > >> -- > >> Marcelo > > > > > > > > -- > Marcelo > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org <javascript:;> > For additional commands, e-mail: dev-h...@spark.apache.org <javascript:;> > >
