+1 to making 2.6 the last JDK6 release. If we want, 2.7 could be a parallel release or one soon after 2.6. We could upgrade other dependencies that require JDK7 as well.
On Fri, Jun 27, 2014 at 3:01 PM, Arun C. Murthy <a...@hortonworks.com> wrote: > Thanks everyone for the discussion. Looks like we have come to a pragmatic > and progressive conclusion. > > In terms of execution of the consensus plan, I think a little bit of > caution is in order. > > Let's give downstream projects more of a runway. > > I propose we inform HBase, Pig, Hive etc. that we are considering making > 2.6 (not 2.5) the last JDK6 release and solicit their feedback. Once they > are comfortable we can pull the trigger in 2.7. > > thanks, > Arun > > > > On Jun 27, 2014, at 11:34 AM, Karthik Kambatla <ka...@cloudera.com> > wrote: > > > > As someone else already mentioned, we should announce one future release > > (may be, 2.5) as the last JDK6-based release before making the move to > JDK7. > > > > I am comfortable calling 2.5 the last JDK6 release. > > > > > > On Fri, Jun 27, 2014 at 11:26 AM, Andrew Wang <andrew.w...@cloudera.com> > > wrote: > > > >> Hi all, responding to multiple messages here, > >> > >> Arun, thanks for the clarification regarding MR classpaths. It sounds > like > >> the story there is improved and still improving. > >> > >> However, I think we still suffer from this at least on the HDFS side. We > >> have a single JAR for all of HDFS, and our clients need to have all the > fun > >> deps like Guava on the classpath. I'm told Spark sticks a newer Guava at > >> the front of the classpath and the HDFS client still works okay, but > this > >> is more happy coincidence than anything else. While we're leaking deps, > >> we're in a scary situation. > >> > >> API compat to me means that an app should be able to run on a new minor > >> version of Hadoop and not have anything break. MAPREDUCE-4421 sounds > like > >> it allows you to run e.g. 2.3 MR jobs on a 2.4 YARN cluster, but what > >> should also be possible is running an HDFS 2.3 app with HDFS 2.4 JARs > and > >> have nothing break. If we muck with the classpath, my understanding is > that > >> this could break. > >> > >> Owen, bumping the minimum JDK version in a minor release like this > should > >> be a one-time exception as Tucu stated. A number of people have pointed > out > >> how painful a forced JDK upgrade is for end users, and it's not > something > >> we should be springing on them in a minor release unless we're *very* > >> confident like in this case. > >> > >> Chris, thanks for bringing up the ecosystem. For CDH5, we standardized > on > >> JDK7 across the CDH stack, so I think that's an indication that most > >> ecosystem projects are ready to make the jump. Is that sufficient in > your > >> mind? > >> > >> For the record, I'm also +1 on the Tucu plan. Is it too late to do this > for > >> 2.5? I'll offer to help out with some of the mechanics. > >> > >> Thanks, > >> Andrew > >> > >> On Wed, Jun 25, 2014 at 4:18 PM, Chris Nauroth < > cnaur...@hortonworks.com> > >> wrote: > >> > >>> I understood the plan for avoiding JDK7-specific features in our code, > >> and > >>> your suggestion to add an extra Jenkins job is a great way to guard > >> against > >>> that. The thing I haven't seen discussed yet is how downstream > projects > >>> will continue to consume our built artifacts. If a downstream project > >>> upgrades to pick up a bug fix, and the jar switches to 1.7 class files, > >> but > >>> their project is still building with 1.6, then it would be a nasty > >>> surprise. > >>> > >>> These are the options I see: > >>> > >>> 1. Make sure all other projects upgrade first. This doesn't sound > >>> feasible, unless all other ecosystem projects have moved to JDK7 > already. > >>> If not, then waiting on a single long pole project would hold up our > >>> migration indefinitely. > >>> > >>> 2. We switch to JDK7, but run javac with -target 1.6 until the whole > >>> ecosystem upgrades. I find this undesirable, because in a certain > sense, > >>> it still leaves a bit of 1.6 lingering in the project. (I'll assume > that > >>> end-of-life for JDK6 also means end-of-life for the 1.6 bytecode > format.) > >>> > >>> 3. Just declare a clean break on some version (your earlier email said > >> 2.5) > >>> and start publishing artifacts built with JDK7 and no -target option. > >>> Overall, this is my preferred option. However, as a side effect, this > >>> sets us up for longer-term maintenance and patch releases off of the > 2.4 > >>> branch if a downstream project that's still on 1.6 needs to pick up a > >>> critical bug fix. > >>> > >>> Of course, this is all a moot point if all the downstream ecosystem > >>> projects have already made the switch to JDK7. I don't know the status > >> of > >>> that off the top of my head. Maybe someone else out there knows? If > >> not, > >>> then I expect I can free up enough in a few weeks to volunteer for > >> tracking > >>> down that information. > >>> > >>> Chris Nauroth > >>> Hortonworks > >>> http://hortonworks.com/ > >>> > >>> > >>> > >>> On Wed, Jun 25, 2014 at 3:12 PM, Alejandro Abdelnur <t...@cloudera.com > > > >>> wrote: > >>> > >>>> Chris, > >>>> > >>>> Compiling with jdk7 and doing javac -target 1.6 is not sufficient, you > >>> are > >>>> still using jdk7 libraries and you could use new APIs, thus breaking > >> jdk6 > >>>> both at compile and runtime. > >>>> > >>>> you need to compile with jdk6 to ensure you are not running into that > >>>> scenario. that is why i was suggesting the nightly jdk6 build/test > >>> jenkins > >>>> job. > >>>> > >>>> > >>>> On Wed, Jun 25, 2014 at 2:04 PM, Chris Nauroth < > >> cnaur...@hortonworks.com > >>>> > >>>> wrote: > >>>> > >>>>> I'm also +1 for getting us to JDK7 within the 2.x line after reading > >>> the > >>>>> proposals and catching up on the discussion in this thread. > >>>>> > >>>>> Has anyone yet considered how to coordinate this change with > >> downstream > >>>>> projects? Would we request downstream projects to upgrade to JDK7 > >>> first > >>>>> before we make the move? Would we switch to JDK7, but run javac > >>> -target > >>>>> 1.6 to maintain compatibility for downstream projects during an > >> interim > >>>>> period? > >>>>> > >>>>> Chris Nauroth > >>>>> Hortonworks > >>>>> http://hortonworks.com/ > >>>>> > >>>>> > >>>>> > >>>>>> On Wed, Jun 25, 2014 at 9:48 AM, Owen O'Malley <omal...@apache.org> > >>>>> wrote: > >>>>> > >>>>>> On Tue, Jun 24, 2014 at 4:44 PM, Alejandro Abdelnur < > >>> t...@cloudera.com > >>>>> > >>>>>> wrote: > >>>>>> > >>>>>>> After reading this thread and thinking a bit about it, I think it > >>>>> should > >>>>>> be > >>>>>>> OK such move up to JDK7 in Hadoop > >>>>>> > >>>>>> > >>>>>> I agree with Alejandro. Changing minimum JDKs is not an > >> incompatible > >>>>> change > >>>>>> and is fine in the 2 branch. (Although I think it is would *not* be > >>>>>> appropriate for a patch release.) Of course we need to do it with > >>>>>> forethought and testing, but moving off of JDK 6, which is EOL'ed > >> is > >>> a > >>>>> good > >>>>>> thing. Moving to Java 8 as a minimum seems much too aggressive and > >> I > >>>>> would > >>>>>> push back on that. > >>>>>> > >>>>>> I'm also think that we need to let the dust settle on the Hadoop 2 > >>> line > >>>>> for > >>>>>> a while before we talk about Hadoop 3. It seems that it has only > >> been > >>>> in > >>>>>> the last 6 months that Hadoop 2 adoption has reached the main > >> stream > >>>>> users. > >>>>>> Our user community needs time to digest the changes in Hadoop 2.x > >>>> before > >>>>> we > >>>>>> fracture the community by starting to discuss Hadoop 3 releases. > >>>>>> > >>>>>> .. Owen > >>>>> > >>>>> -- > >>>>> CONFIDENTIALITY NOTICE > >>>>> NOTICE: This message is intended for the use of the individual or > >>> entity > >>>> to > >>>>> which it is addressed and may contain information that is > >> confidential, > >>>>> privileged and exempt from disclosure under applicable law. If the > >>> reader > >>>>> of this message is not the intended recipient, you are hereby > >> notified > >>>> that > >>>>> any printing, copying, dissemination, distribution, disclosure or > >>>>> forwarding of this communication is strictly prohibited. If you have > >>>>> received this communication in error, please contact the sender > >>>> immediately > >>>>> and delete it from your system. Thank You. > >>>> > >>>> > >>>> > >>>> -- > >>>> Alejandro > >>> > >>> -- > >>> CONFIDENTIALITY NOTICE > >>> NOTICE: This message is intended for the use of the individual or > entity > >> to > >>> which it is addressed and may contain information that is confidential, > >>> privileged and exempt from disclosure under applicable law. If the > reader > >>> of this message is not the intended recipient, you are hereby notified > >> that > >>> any printing, copying, dissemination, distribution, disclosure or > >>> forwarding of this communication is strictly prohibited. If you have > >>> received this communication in error, please contact the sender > >> immediately > >>> and delete it from your system. Thank You. > >> > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. >
