A bit of a different angle. As the bottom of the stack Hadoop has to be conservative in adopting things, but it should not preclude consumers of Hadoop (downstream projects and Hadoop application developers) to have additional requirements such as a higher JDK API than JDK6.
Hadoop 2.x should stick to using JDK6 API Hadoop 2.x should be tested with multiple runtimes: JDK6, JDK7 and eventually JDK8 Downstream projects and Hadoop application developers are free to require any JDK6+ version for development and runtime. Hadoop 3.x should allow using JDK7 API, bumping the minimum runtime requirement to JDK7 and be tested with JDK7 and JDK8 runtimes. Thanks. On Thu, Apr 10, 2014 at 10:04 AM, Eli Collins <e...@cloudera.com> wrote: > On Thu, Apr 10, 2014 at 1:11 AM, Steve Loughran <ste...@hortonworks.com > >wrote: > > > On 9 April 2014 23:52, Eli Collins <e...@cloudera.com> wrote: > > > > > > > > > > > For the sake of this discussion we should separate the runtime from > > > the programming APIs. Users are already migrating to the java7 runtime > > > for most of the reasons listed below (support, performance, bugs, > > > etc), and the various distributions cert their Hadoop 2 based > > > distributions on java7. This gives users many of the benefits of > > > java7, without forcing users off java6. Ie Hadoop does not need to > > > switch to the java7 programming APIs to make sure everyone has a > > > supported runtime. > > > > > > > > +1: you can use Java 7 today; I'm not sure how tested Java 8 is > > > > > > > The question here is really about when Hadoop, and the Hadoop > > > ecosystem (since adjacent projects often end up in the same classpath) > > > start using the java7 programming APIs and therefore break > > > compatibility with java6 runtimes. I think our java6 runtime users > > > would consider dropping support for their java runtime in an update of > > > a major release to be an incompatible change (the binaries stop > > > working on their current jvm). > > > > > > do you mean major 2.x -> 3.y or minor 2.x -> 2.(x+1) here? > > > > I mean 2.x --> 2.(x+1). Ie I'm running the 2.4 stable and upgrading to > 2.5. > > > > > > > > > That may be worth it if we can > > > articulate sufficient value to offset the cost (they have to upgrade > > > their environment, might make rolling upgrades stop working, etc), but > > > I've not yet heard an argument that articulates the value relative to > > > the cost. Eg upgrading to the java7 APIs allows us to pull in > > > dependencies with new major versions, but only if those dependencies > > > don't break compatibility (which is likely given that our classpaths > > > aren't so isolated), and, realistically, only if the entire Hadoop > > > stack moves to java7 as well > > > > > > > > > > > (eg we have to recompile HBase to > > > generate v1.7 binaries even if they stick on API v1.6). I'm not aware > > > of a feature, bug etc that really motivates this. > > > > > > I don't see that being needed unless we move up to new java7+ only > > libraries and HBase needs to track this. > > > > The big "recompile to work" issue is google guava, which is troublesome > > enough I'd be tempted to say "can we drop it entirely" > > > > > > > > > An alternate approach is to keep the current stable release series > > > (v2.x) as is, and start using new APIs in trunk (for v3). This will be > > > a major upgrade for Hadoop and therefore an incompatible change like > > > this is to be expected (it would be great if this came with additional > > > changes to better isolate classpaths and dependencies from each > > > other). It allows us to continue to support multiple types of users > > > with different branches, vs forcing all users onto a new version. It > > > of course means that 2.x users will not get the benefits of the new > > > API, but its unclear what those benefits are given theIy can already > > > get the benefits of adopting the newer java runtimes today. > > > > > > > > > > > I'm (personally) +1 to this, I also think we should plan to do the switch > > some time this year to not only get the benefits, but discover the costs > > > > > Agree > > > > > -- > > CONFIDENTIALITY NOTICE > > NOTICE: This message is intended for the use of the individual or entity > to > > which it is addressed and may contain information that is confidential, > > privileged and exempt from disclosure under applicable law. If the reader > > of this message is not the intended recipient, you are hereby notified > that > > any printing, copying, dissemination, distribution, disclosure or > > forwarding of this communication is strictly prohibited. If you have > > received this communication in error, please contact the sender > immediately > > and delete it from your system. Thank You. > > > -- Alejandro
