Is there broad consensus that, by end of 3Q2014 at the latest, "the average" contributor to Hadoop should be free to use Java7 features? And start pulling in libraries that have a Java7 dependency? And start doing the "janitorial" work of taking advantage of the Java7 APIs? Or do we think that the bulk of Hadoop work will be done against Java6 APIs (and avoiding Java7-dependent libraries) through the end of the year?
If the consensus is that we introduce Java7 into the bulk of Hadoop coding, what's the plan for getting there? The answer can't be "right now, in trunk." Even if we agreed to start allowing Java7 dependencies into trunk, as a practical matter this isn't enough. Right now, if I'm a random Hadoop contributor, I'd be stupid to contribute to trunk: I know that any stable release in the near term will be from branch2, so if I want a prayer of seeing my change in a stable release, I'd better contribute to branch2. If we want a path to allowing Java7 dependencies by Q4, then we need one of the following: 1) "branch3 plan:" The major Hadoop vendors (you know who you are) commit to shipping a "v3" of Hadoop in Q4 that allows Java7 dependencies and show signs of living up to that commitment (e.g., a branch3 is created sometime soon). This puts us all on a path towards a "real" release of Hadoop that allows Java7 dependencies. 2) "branch2 plan:" deprecate Java6 as a runtime environment now, publicly declare a time frame (e.g., 4Q2014) when _future development_ stops supporting Java6 runtime, and work with our customers in the meantime to get them off a crazy-old version of Java (that's what we're doing right now). I don't see another path to allowing Java7 dependencies. In the current state of indecision, the smart programmer would be assuming no Java7 dependencies into 2015. On the one hand, I don't see the branch3 plan actually happening. This is a big decision involving marketing, engineering, customer support. Plus it creates a problem for sales: Come summertime, they'll have a hard time selling 2.x-based releases because they've pre-announced support for 3.x. It's just not going to happen. On the other hand, I don't see the problem with the branch2 plan. The branch2 plan also requires the commitment from the major vendors, but this decision is not nearly as galactic. By the time 3Q2014 comes along, this problem will be very rarified. Also, don't forget that it typically takes a customer 3-6 months to upgrade their Hadoop -- and a customer who's afraid to shift off Java6 in 3Q2014 will probably take a year to upgrade. The branch2 plan implies a last Java6 release of Hadoop in 3Q2014. If we assume a Java7-averse customer will take a year to upgrade to this release -- and then will take another year to upgrade their cluster after that -- then they can be happily using Java6 all the way into 2016. (Another point, if 3Q2014 comes along and vendors find they have so many customers still on Java6 that they can't afford the discontinuity, then they can shift their MAJOR version number of their product to communicate the discontinuity -- there's nothing that says that a vendor's versioning scheme must agree exactly with Hadoop's.) In short, we don't currently have a realistic path for introducing Java7 dependencies into Hadoop. Simply allowing them into trunk will NOT solve this problem: any contributor who wants to see their code in a stable release knows it'll have to flow through branch2 -- and thus they'll have to avoid Java6 dependencies. The branch2 plan is the only plan proposed so far that gets us to Java7 dependencies by Q4. And the important part of the branch2 plan is we make the decision soon -- so we have time to notify folks and otherwise work that decision out into the field. Raymie On Tue, Apr 8, 2014 at 9:19 AM, Eli Collins <e...@cloudera.com> wrote: > On Tue, Apr 8, 2014 at 2:00 AM, Ottenheimer, Davi > <davi.ottenhei...@emc.com> wrote: >>> From: Eli Collins [mailto:e...@cloudera.com] >>> Sent: Monday, April 07, 2014 11:54 AM >>> >>> >>> IMO we should not drop support for Java 6 in a minor update of a stable >>> release (v2). I don't think the larger Hadoop user base would find it >>> acceptable that upgrading to a minor update caused their systems to stop >>> working because they didn't upgrade Java. There are people still getting >>> support for Java 6. ... >>> >>> Thanks, >>> Eli >> >> Hi Eli, >> >> Technically you are correct those with extended support get critical >> security fixes for 6 until the end of 2016. I am curious whether many of >> those are in the Hadoop user base. Do you know? My guess is the vast >> majority are within Oracle's official public end of life, which was over 12 >> months ago. Even Premier support ended Dec 2013: >> >> http://www.oracle.com/technetwork/java/eol-135779.html >> >> The end of Java 6 support carries much risk. It has to be considered in >> terms of serious security vulnerabilities such as CVE-2013-2465 with CVSS >> score 10.0. >> >> http://www.cvedetails.com/cve/CVE-2013-2465/ >> >> Since you mentioned "caused systems to stop" as an example of what would be >> a concern to Hadoop users, please note the CVE-2013-2465 availability impact: >> >> "Complete (There is a total shutdown of the affected resource. The attacker >> can render the resource completely unavailable.)" >> >> This vulnerability was patched in Java 6 Update 51, but post end of life. >> Apple pushed out the update specifically because of this vulnerability >> (http://support.apple.com/kb/HT5717) as did some other vendors privately, >> but for the majority of people using Java 6 means they have a ticking time >> bomb. >> >> Allowing it to stay should be considered in terms of accepting the whole >> risk posture. >> > > There are some who get extended support, but I suspect many just have > a if-it's-not-broke mentality when it comes to production deployments. > The current code supports both java6 and java7 and so allows these > people to remain compatible, while enabling others to upgrade to the > java7 runtime. This seems like the right compromise for a stable > release series. Again, absolutely makes sense for trunk (ie v3) to > require java7 or greater.
