How well is trunk tested? Does anyone deploy it with real applications running on top? When will the trunk codebase next be the basis for a production release? An impromptu diff of hadoop-common trunk against branch-2 as of today is 38,625 lines. Can they be said to be the same animal? I ask because any disincentive toward putting code in trunk is beside the point, if the only target worth pursuing today is branch-2 unless one doesn't care if the code is released for production use. Questions on whither JDK6 or JDK7+ (or JRE6 versus JRE7+) only matter for the vast majority of Hadoopers if talking about branch-2.
On Mon, Apr 14, 2014 at 9:22 AM, Colin McCabe <cmcc...@alumni.cmu.edu>wrote: > I think the bottom line here is that as long as our stable release > uses JDK6, there is going to be a very, very strong disincentive to > put any code which can't run on JDK6 into trunk. > > Like I said earlier, the traditional reason for putting something in > trunk but not the stable release is that it needs more testing. If a > stable release that drops support for JDK6 is more than a year away, > does it make sense to put anything in trunk like that? What might > need more than a year of testing? Certainly not changes to > LocalFileSystem to use the new APIs. I also don't think an upgrade to > various libraries qualifies. > > It might be best to shelve this for now, like we've done in the past, > until we're ready to talk about a stable release that requires JDK7+. > At least that's my feeling. > > If we're really desperate for the new file APIs JDK7 provides, we > could consider using loadable modules for it in branch-2. This is > similar to how we provide JNI versions of certain things on certain > platforms, without dropping support for the other platforms. > > best, > Colin > > On Sun, Apr 13, 2014 at 10:39 AM, Raymie Stata <rst...@altiscale.com> > wrote: > > There's an outstanding question addressed to me: "Are there particular > > features or new dependencies that you would like to contribute (or see > > contributed) that require using the Java 1.7 APIs?" The question > > misses the point: We'd figure out how to write something we wanted to > > contribute to Hadoop against the APIs of Java4 if that's what it took > > to get them into a stable release. And at current course and speed, > > that's how ridiculous things could get. > > > > To summarize, it seems like there's a vague consensus that it might be > > okay to eventually allow the use of Java7 in trunk, but there's no > > decision. And there's been no answer to the concern that even if such > > dependencies were allowed in Java7, the only people using them would > > be people who uninterested in getting their patches into a stable > > release of Hadoop on any knowable timeframe, which doesn't bode well > > for the ability to stabilize that Java7 code when it comes time to > > attempt to. > > > > I don't have more to add, so I'll go back to lurking. It'll be > > interesting to see where we'll be standing a year from now. > > > > On Sun, Apr 13, 2014 at 2:09 AM, Tsuyoshi OZAWA > > <ozawa.tsuyo...@gmail.com> wrote: > >> Hi, > >> > >> +1 for Karthik's idea(non-binding). > >> > >> IMO, we should keep the compatibility between JDK 6 and JDK 7 on both > branch-1 > >> and branch-2, because users can be using them. For future releases that > we can > >> declare breaking compatibility(e.g. 3.0.0 release), we can use JDK 7 > >> features if we > >> can get benefits. However, it can increase maintenance costs and > distributes the > >> efforts of contributions to maintain branches. Then, I think it is > >> reasonable approach > >> that we use limited and minimum JDK-7 APIs when we have reasons we need > to use > >> the features. > >> By the way, if we start to use JDK 7 APIs, we should declare the basis > >> when to use > >> JDK 7 APIs on Wiki not to confuse contributors. > >> > >> Thanks, > >> - Tsuyoshi > >> > >> On Wed, Apr 9, 2014 at 11:44 AM, Raymie Stata <rst...@altiscale.com> > wrote: > >>>> It might make sense to try to enumerate the benefits of switching to > >>>> Java7 APIs and dependencies. > >>> > >>> - Java7 introduced a huge number of language, byte-code, API, and > >>> tooling enhancements! Just to name a few: try-with-resources, newer > >>> and stronger encyrption methods, more scalable concurrency primitives. > >>> See http://www.slideshare.net/boulderjug/55-things-in-java-7 > >>> > >>> - We can't update current dependencies, and we can't add cool new > ones. > >>> > >>> - Putting language/APIs aside, don't forget that a huge amount of > effort > >>> goes into qualifying for Java6 (at least, I hope the folks claiming to > >>> support Java6 are putting in such an effort :-). Wouldn't Hadoop > >>> users/customers be better served if qualification effort went into > >>> Java7/8 versus Java6/7? > >>> > >>> Getting to Java7 as a development env (and Java8 as a runtime env) > >>> seems like a no-brainer. Question is: How? > >>> > >>> On Tue, Apr 8, 2014 at 10:21 AM, Sandy Ryza <sandy.r...@cloudera.com> > wrote: > >>>> It might make sense to try to enumerate the benefits of switching to > Java7 > >>>> APIs and dependencies. IMO, the ones listed so far on this thread > don't > >>>> make a compelling enough case to drop Java6 in branch-2 on any time > frame, > >>>> even if this means supporting Java6 through 2015. For example, the > change > >>>> in RawLocalFileSystem semantics might be an incompatible change for > >>>> branch-2 any way. > >>>> > >>>> > >>>> On Tue, Apr 8, 2014 at 10:05 AM, Karthik Kambatla <ka...@cloudera.com > >wrote: > >>>> > >>>>> +1 to NOT breaking compatibility in branch-2. > >>>>> > >>>>> I think it is reasonable to require JDK7 for trunk, if we limit use > of > >>>>> JDK7-only API to security fixes etc. If we make other optimizations > (like > >>>>> IO), it would be a pain to backport things to branch-2. I guess this > all > >>>>> depends on when we see ourselves shipping Hadoop-3. Any ideas on > that? > >>>>> > >>>>> > >>>>> On Tue, Apr 8, 2014 at 9:19 AM, Eli Collins <e...@cloudera.com> > wrote: > >>>>> > >>>>> > On Tue, Apr 8, 2014 at 2:00 AM, Ottenheimer, Davi > >>>>> > <davi.ottenhei...@emc.com> wrote: > >>>>> > >> From: Eli Collins [mailto:e...@cloudera.com] > >>>>> > >> Sent: Monday, April 07, 2014 11:54 AM > >>>>> > >> > >>>>> > >> > >>>>> > >> IMO we should not drop support for Java 6 in a minor update of a > >>>>> stable > >>>>> > >> release (v2). I don't think the larger Hadoop user base would > find it > >>>>> > >> acceptable that upgrading to a minor update caused their > systems to > >>>>> stop > >>>>> > >> working because they didn't upgrade Java. There are people still > >>>>> getting > >>>>> > >> support for Java 6. ... > >>>>> > >> > >>>>> > >> Thanks, > >>>>> > >> Eli > >>>>> > > > >>>>> > > Hi Eli, > >>>>> > > > >>>>> > > Technically you are correct those with extended support get > critical > >>>>> > security fixes for 6 until the end of 2016. I am curious whether > many of > >>>>> > those are in the Hadoop user base. Do you know? My guess is the > vast > >>>>> > majority are within Oracle's official public end of life, which > was over > >>>>> 12 > >>>>> > months ago. Even Premier support ended Dec 2013: > >>>>> > > > >>>>> > > http://www.oracle.com/technetwork/java/eol-135779.html > >>>>> > > > >>>>> > > The end of Java 6 support carries much risk. It has to be > considered in > >>>>> > terms of serious security vulnerabilities such as CVE-2013-2465 > with CVSS > >>>>> > score 10.0. > >>>>> > > > >>>>> > > http://www.cvedetails.com/cve/CVE-2013-2465/ > >>>>> > > > >>>>> > > Since you mentioned "caused systems to stop" as an example of > what > >>>>> would > >>>>> > be a concern to Hadoop users, please note the CVE-2013-2465 > availability > >>>>> > impact: > >>>>> > > > >>>>> > > "Complete (There is a total shutdown of the affected resource. > The > >>>>> > attacker can render the resource completely unavailable.)" > >>>>> > > > >>>>> > > This vulnerability was patched in Java 6 Update 51, but post end > of > >>>>> > life. Apple pushed out the update specifically because of this > >>>>> > vulnerability (http://support.apple.com/kb/HT5717) as did some > other > >>>>> > vendors privately, but for the majority of people using Java 6 > means they > >>>>> > have a ticking time bomb. > >>>>> > > > >>>>> > > Allowing it to stay should be considered in terms of accepting > the > >>>>> whole > >>>>> > risk posture. > >>>>> > > > >>>>> > > >>>>> > There are some who get extended support, but I suspect many just > have > >>>>> > a if-it's-not-broke mentality when it comes to production > deployments. > >>>>> > The current code supports both java6 and java7 and so allows these > >>>>> > people to remain compatible, while enabling others to upgrade to > the > >>>>> > java7 runtime. This seems like the right compromise for a stable > >>>>> > release series. Again, absolutely makes sense for trunk (ie v3) to > >>>>> > require java7 or greater. > >>>>> > > >>>>> > >> > >> > >> > >> -- > >> - Tsuyoshi > -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
