I think the bottom line here is that as long as our stable release uses JDK6, there is going to be a very, very strong disincentive to put any code which can't run on JDK6 into trunk.
Like I said earlier, the traditional reason for putting something in trunk but not the stable release is that it needs more testing. If a stable release that drops support for JDK6 is more than a year away, does it make sense to put anything in trunk like that? What might need more than a year of testing? Certainly not changes to LocalFileSystem to use the new APIs. I also don't think an upgrade to various libraries qualifies. It might be best to shelve this for now, like we've done in the past, until we're ready to talk about a stable release that requires JDK7+. At least that's my feeling. If we're really desperate for the new file APIs JDK7 provides, we could consider using loadable modules for it in branch-2. This is similar to how we provide JNI versions of certain things on certain platforms, without dropping support for the other platforms. best, Colin On Sun, Apr 13, 2014 at 10:39 AM, Raymie Stata <rst...@altiscale.com> wrote: > There's an outstanding question addressed to me: "Are there particular > features or new dependencies that you would like to contribute (or see > contributed) that require using the Java 1.7 APIs?" The question > misses the point: We'd figure out how to write something we wanted to > contribute to Hadoop against the APIs of Java4 if that's what it took > to get them into a stable release. And at current course and speed, > that's how ridiculous things could get. > > To summarize, it seems like there's a vague consensus that it might be > okay to eventually allow the use of Java7 in trunk, but there's no > decision. And there's been no answer to the concern that even if such > dependencies were allowed in Java7, the only people using them would > be people who uninterested in getting their patches into a stable > release of Hadoop on any knowable timeframe, which doesn't bode well > for the ability to stabilize that Java7 code when it comes time to > attempt to. > > I don't have more to add, so I'll go back to lurking. It'll be > interesting to see where we'll be standing a year from now. > > On Sun, Apr 13, 2014 at 2:09 AM, Tsuyoshi OZAWA > <ozawa.tsuyo...@gmail.com> wrote: >> Hi, >> >> +1 for Karthik's idea(non-binding). >> >> IMO, we should keep the compatibility between JDK 6 and JDK 7 on both >> branch-1 >> and branch-2, because users can be using them. For future releases that we >> can >> declare breaking compatibility(e.g. 3.0.0 release), we can use JDK 7 >> features if we >> can get benefits. However, it can increase maintenance costs and distributes >> the >> efforts of contributions to maintain branches. Then, I think it is >> reasonable approach >> that we use limited and minimum JDK-7 APIs when we have reasons we need to >> use >> the features. >> By the way, if we start to use JDK 7 APIs, we should declare the basis >> when to use >> JDK 7 APIs on Wiki not to confuse contributors. >> >> Thanks, >> - Tsuyoshi >> >> On Wed, Apr 9, 2014 at 11:44 AM, Raymie Stata <rst...@altiscale.com> wrote: >>>> It might make sense to try to enumerate the benefits of switching to >>>> Java7 APIs and dependencies. >>> >>> - Java7 introduced a huge number of language, byte-code, API, and >>> tooling enhancements! Just to name a few: try-with-resources, newer >>> and stronger encyrption methods, more scalable concurrency primitives. >>> See http://www.slideshare.net/boulderjug/55-things-in-java-7 >>> >>> - We can't update current dependencies, and we can't add cool new ones. >>> >>> - Putting language/APIs aside, don't forget that a huge amount of effort >>> goes into qualifying for Java6 (at least, I hope the folks claiming to >>> support Java6 are putting in such an effort :-). Wouldn't Hadoop >>> users/customers be better served if qualification effort went into >>> Java7/8 versus Java6/7? >>> >>> Getting to Java7 as a development env (and Java8 as a runtime env) >>> seems like a no-brainer. Question is: How? >>> >>> On Tue, Apr 8, 2014 at 10:21 AM, Sandy Ryza <sandy.r...@cloudera.com> wrote: >>>> It might make sense to try to enumerate the benefits of switching to Java7 >>>> APIs and dependencies. IMO, the ones listed so far on this thread don't >>>> make a compelling enough case to drop Java6 in branch-2 on any time frame, >>>> even if this means supporting Java6 through 2015. For example, the change >>>> in RawLocalFileSystem semantics might be an incompatible change for >>>> branch-2 any way. >>>> >>>> >>>> On Tue, Apr 8, 2014 at 10:05 AM, Karthik Kambatla >>>> <ka...@cloudera.com>wrote: >>>> >>>>> +1 to NOT breaking compatibility in branch-2. >>>>> >>>>> I think it is reasonable to require JDK7 for trunk, if we limit use of >>>>> JDK7-only API to security fixes etc. If we make other optimizations (like >>>>> IO), it would be a pain to backport things to branch-2. I guess this all >>>>> depends on when we see ourselves shipping Hadoop-3. Any ideas on that? >>>>> >>>>> >>>>> On Tue, Apr 8, 2014 at 9:19 AM, Eli Collins <e...@cloudera.com> wrote: >>>>> >>>>> > On Tue, Apr 8, 2014 at 2:00 AM, Ottenheimer, Davi >>>>> > <davi.ottenhei...@emc.com> wrote: >>>>> > >> From: Eli Collins [mailto:e...@cloudera.com] >>>>> > >> Sent: Monday, April 07, 2014 11:54 AM >>>>> > >> >>>>> > >> >>>>> > >> IMO we should not drop support for Java 6 in a minor update of a >>>>> stable >>>>> > >> release (v2). I don't think the larger Hadoop user base would find >>>>> > >> it >>>>> > >> acceptable that upgrading to a minor update caused their systems to >>>>> stop >>>>> > >> working because they didn't upgrade Java. There are people still >>>>> getting >>>>> > >> support for Java 6. ... >>>>> > >> >>>>> > >> Thanks, >>>>> > >> Eli >>>>> > > >>>>> > > Hi Eli, >>>>> > > >>>>> > > Technically you are correct those with extended support get critical >>>>> > security fixes for 6 until the end of 2016. I am curious whether many of >>>>> > those are in the Hadoop user base. Do you know? My guess is the vast >>>>> > majority are within Oracle's official public end of life, which was over >>>>> 12 >>>>> > months ago. Even Premier support ended Dec 2013: >>>>> > > >>>>> > > http://www.oracle.com/technetwork/java/eol-135779.html >>>>> > > >>>>> > > The end of Java 6 support carries much risk. It has to be considered >>>>> > > in >>>>> > terms of serious security vulnerabilities such as CVE-2013-2465 with >>>>> > CVSS >>>>> > score 10.0. >>>>> > > >>>>> > > http://www.cvedetails.com/cve/CVE-2013-2465/ >>>>> > > >>>>> > > Since you mentioned "caused systems to stop" as an example of what >>>>> would >>>>> > be a concern to Hadoop users, please note the CVE-2013-2465 availability >>>>> > impact: >>>>> > > >>>>> > > "Complete (There is a total shutdown of the affected resource. The >>>>> > attacker can render the resource completely unavailable.)" >>>>> > > >>>>> > > This vulnerability was patched in Java 6 Update 51, but post end of >>>>> > life. Apple pushed out the update specifically because of this >>>>> > vulnerability (http://support.apple.com/kb/HT5717) as did some other >>>>> > vendors privately, but for the majority of people using Java 6 means >>>>> > they >>>>> > have a ticking time bomb. >>>>> > > >>>>> > > Allowing it to stay should be considered in terms of accepting the >>>>> whole >>>>> > risk posture. >>>>> > > >>>>> > >>>>> > There are some who get extended support, but I suspect many just have >>>>> > a if-it's-not-broke mentality when it comes to production deployments. >>>>> > The current code supports both java6 and java7 and so allows these >>>>> > people to remain compatible, while enabling others to upgrade to the >>>>> > java7 runtime. This seems like the right compromise for a stable >>>>> > release series. Again, absolutely makes sense for trunk (ie v3) to >>>>> > require java7 or greater. >>>>> > >>>>> >> >> >> >> -- >> - Tsuyoshi
