Er, that should read "order in which it ran unit tests." C.
On Fri, Jun 20, 2014 at 11:02 AM, Colin McCabe <cmcc...@alumni.cmu.edu> wrote: > I think the important thing to do right now is to ensure our code > works with jdk8. This is similar to the work we did last year to fix > issues that cropped up with jdk7. There were just a few minor things > like the error in which it ran unit tests that caused issues. > > I don't think it's out of the question to bump minVersion directly > from 1.6 to 1.8. Java 7 is EOL in less than a year, after all... at > least according to the current roadmap here: > http://www.oracle.com/technetwork/java/eol-135779.html > > We should try to do things in a way that causes the least disruption > for users upgrading clusters, if possible... > > cheers, > Colin > > On Thu, Jun 19, 2014 at 3:31 PM, Andrew Purtell <apurt...@apache.org> wrote: >> There are a number of security (algorithm, not vulnerability) and >> performance improvements that landed in 8, not 7. As a runtime for the >> performance conscious, it might be recommendable. I've come across GC >> issues in 6 or 7 where, talking with some Java platform folks, the first >> suggested course of action is try again with 8. Would be be more of the >> current moment if this discussion was about setting guidelines that >> prescribe when and when not to use 8+ language features, or concurrency >> library improvements that rely on intrinsics only available in the 8 >> runtime? Has the Java 6 ship sailed? Just set the minimum supported JDK and >> runtime at 7 at next release? Use of the <> operator or multicatch wouldn't >> and shouldn't need be debated, they are quite minor. On the other hand, I >> would imagine discussion and debate on what 8+ language features might be >> useful to use at some future time could be a lively one. >> >> >> >> On Wed, Jun 18, 2014 at 3:03 PM, Colin McCabe <cmcc...@alumni.cmu.edu> >> wrote: >> >>> In CDH5, Cloudera encourages people to use JDK7. JDK6 has been EOL >>> for a while now and is not something we recommend. >>> >>> As we discussed before, everyone is in favor of upgrading to JDK7. >>> Every cluster operator of a reasonably modern Hadoop should do it.... >>> whatever distro or release you run. As developers, we run JDK7 as >>> well. >>> >>> I'd just like to see a plan for when branch-2 (or some other branch) >>> will create a stable release that drops support for JDK1.6. If we >>> don't have such a plan, I feel like it's too early to talk about this >>> stuff. >>> >>> If we drop support for 1.6 in trunk but not in branch-2, we are >>> fragmenting the project. People will start writing unreleaseable code >>> (because it doesn't work on branch-2) and we'll be back to the bad old >>> days of Hadoop version fragmentation that branch-2 was intended to >>> solve. Backports will become harder. The biggest problem is that >>> trunk will start to depend on libraries or Maven plugins that branch-2 >>> can't even use, because they're JDK7+-only. >>> >>> Steve wrote: "if someone actually did file a bug on something on >>> branch-2 which didn't work on Java 6 but went away on Java7+, we'd >>> probably close it as a WORKSFORME". >>> >>> Steve, if this is true, we should just bump the minimum supported >>> version for branch-2 to 1.7 today and resolve this. If we truly >>> believe that there are no issues here, then let's just decide to drop >>> 1.6 in a specific future release of Hadoop 2. If there are issues >>> with releasing JDK1.7+ only code, then let's figure out what they are >>> before proceeding. >>> >>> best, >>> Colin >>> >>> >>> On Wed, Jun 18, 2014 at 1:41 PM, Sandy Ryza <sandy.r...@cloudera.com> >>> wrote: >>> > We do release warnings when we are aware of vulnerabilities in our >>> > dependencies. >>> > >>> > However, unless I'm grossly misunderstanding, the vulnerability that you >>> > point out is not a vulnerability within the context of our software. >>> > Hadoop doesn't try to sandbox within JVMs. In a secure setup, any JVM >>> > running non-trusted user code is running as that user, so "breaking out" >>> > doesn't offer the ability to do anything malicious. >>> > >>> > -Sandy >>> > >>> > On Wed, Jun 18, 2014 at 1:30 PM, Ottenheimer, Davi < >>> davi.ottenhei...@emc.com >>> >> wrote: >>> > >>> >> Andrew, >>> >> >>> >> >>> >> >>> >> “I don't see any point to switching” is an interesting perspective, >>> given >>> >> the well-known risks of running unsafe software. Clearly customer best >>> >> interest is stability. JDK6 is in a known unsafe state. The longer >>> anyone >>> >> delays the necessary transition to safety the longer the door is left >>> open >>> >> to predictable disaster. >>> >> >>> >> >>> >> >>> >> You also said "we still test and support JDK6". I searched but have not >>> >> been able to find Cloudera critical security fixes for JDK6. >>> >> >>> >> >>> >> >>> >> Can you clarify, for example, Java 6 Update 51 for CVE-2013-2465? In >>> other >>> >> words, did you release to your customers any kind of public alert or >>> >> warning of this CVSS 10.0 event as part of your JDK6 support? >>> >> >>> >> >>> >> >>> >> http://www.cvedetails.com/cve/CVE-2013-2465/ >>> >> >>> >> >>> >> >>> >> If you are not releasing your own security fixes for JDK6 post-EOL would >>> >> it perhaps be safer to say Cloudera is hands-off; neither supports, nor >>> >> opposes the known insecure and deprecated/unpatched JDK? >>> >> >>> >> >>> >> >>> >> I mentioned before in this thread the Oracle support timeline: >>> >> >>> >> >>> >> >>> >> - official public EOL (end of life) was more than a year ago >>> >> >>> >> - premier support ended more than six months ago >>> >> >>> >> - extended support may get critical security fixes until the end of 2016 >>> >> >>> >> >>> >> >>> >> Given this timeline, does Cloudera officially take responsibility for >>> >> Hadoop customer safety? Are you going to be releasing critical security >>> >> fixes to a known unsafe JDK? >>> >> >>> >> >>> >> >>> >> Davi >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> > -----Original Message----- >>> >> >>> >> > From: Andrew Wang [mailto:andrew.w...@cloudera.com] >>> >> >>> >> > Sent: Wednesday, June 18, 2014 12:33 PM >>> >> >>> >> > To: common-dev@hadoop.apache.org >>> >> >>> >> > Subject: Re: Plans of moving towards JDK7 in trunk >>> >> >>> >> > >>> >> >>> >> > Actually, a lot of our customers are still on JDK6, so if anything, >>> its >>> >> popularity >>> >> >>> >> > hasn't significantly decreased. We still test and support JDK6 for >>> CDH4 >>> >> and >>> >> >>> >> > CDH5. The claim that branch-2 is effectively JDK7 because no one >>> supports >>> >> >>> >> > JDK6 is untrue. >>> >> >>> >> > >>> >> >>> >> > One issue with your proposal is that java 7+ libraries can have >>> >> incompatible >>> >> >>> >> > APIs compared to their java 6 versions. Guava moves very quickly with >>> >> regard >>> >> >>> >> > to the deprecate+remove cycle. This means branch-2 and trunk >>> divergence, >>> >> >>> >> > as we're stuck using different Guava APIs to do the same thing. >>> >> >>> >> > >>> >> >>> >> > No one's arguing against moving to Java 7+ in trunk eventually, but >>> >> there isn't >>> >> >>> >> > a clear plan for a trunk-based release. I don't see any point to >>> >> switching trunk >>> >> >>> >> > over until that's true, for the aforementioned reasons. >>> >> >>> >> > >>> >> >>> >> > Best, >>> >> >>> >> > Andrew >>> >> >>> >> > >>> >> >>> >> > >>> >> >>> >> > On Wed, Jun 18, 2014 at 12:08 PM, Steve Loughran >>> >> >>> >> > <ste...@hortonworks.com<mailto:ste...@hortonworks.com>> >>> >> >>> >> > wrote: >>> >> >>> >> > >>> >> >>> >> > > I also think we need to recognise that its been three months since >>> >> >>> >> > > that last discussion, and Java 6 has not suddenly burst back into >>> >> >>> >> > > popularity >>> >> >>> >> > > >>> >> >>> >> > > >>> >> >>> >> > > - nobody providing commercial support for Hadoop is offering >>> >> branch-2 >>> >> >>> >> > > support on Java 6 AFAIK >>> >> >>> >> > > - therefore, nobody is testing it at scale except privately, and >>> >> they >>> >> >>> >> > > aren't reporting bugs if they are >>> >> >>> >> > > - if someone actually did file a bug on something on branch-2 >>> which >>> >> >>> >> > > didn't work on Java 6 but went away on Java7+, we'd probably >>> close >>> >> >>> >> > > it as a >>> >> >>> >> > > WORKSFORME >>> >> >>> >> > > >>> >> >>> >> > > >>> >> >>> >> > > whether we acknowledge it or not, Hadoop 2.x is now really Java 7+. >>> >> >>> >> > > >>> >> >>> >> > > We do all agree that hadoop 3 will not be java 6, so the only issue >>> is >>> >> >>> >> > > "when and how to make that transition". >>> >> >>> >> > > >>> >> >>> >> > > That patch of mine just makes it possible to do today. >>> >> >>> >> > > >>> >> >>> >> > > I have actually jumped to Java7 in the slider project, and actually >>> >> >>> >> > > being using Java 8 and twill; the new language features there are >>> >> >>> >> > > significant and would be great to use in Hadoop *at some point in >>> the >>> >> >>> >> > > future* >>> >> >>> >> > > >>> >> >>> >> > > For Java 7 though, based on that experience, the language changes >>> are >>> >> >>> >> > > convenient but not essential >>> >> >>> >> > > >>> >> >>> >> > > - try-with-resources simply swallows close failures without the >>> log >>> >> >>> >> > > integration we have with IOUtils.closeStream(), so shoudn't be >>> used >>> >> in >>> >> >>> >> > > hadoop core anyway. >>> >> >>> >> > > - string based switching: convenient, but not critical >>> >> >>> >> > > - type inference on template constructors. Modern IDEs handle the >>> >> pain >>> >> >>> >> > > anyway >>> >> >>> >> > > >>> >> >>> >> > > The only feature I like is multi-catch and typed rethrow >>> >> >>> >> > > >>> >> >>> >> > > catch(IOException | ExitException e) { log.warn(e.toString(); >>> throw >>> >> >>> >> > > e; } >>> >> >>> >> > > >>> >> >>> >> > > this would make "e" look like Exception, but when rethrown go back >>> to >>> >> >>> >> > > its original type. >>> >> >>> >> > > >>> >> >>> >> > > This reduces duplicate work, and is the bit l actually value. Is it >>> >> >>> >> > > enough to justify making code incompatible across branches? No. >>> >> >>> >> > > >>> >> >>> >> > > So i'm going to propose this, and would like to start a vote on it >>> >> >>> >> > > soon >>> >> >>> >> > > >>> >> >>> >> > > >>> >> >>> >> > > 1. we parameterize java versions in the POMs on all branches, >>> with >>> >> >>> >> > > separate JDK versions and Java language >>> >> >>> >> > > 2. branch-2: java-6-language and JDK-6 minimum JDK >>> >> >>> >> > > 3. trunk: java-6-language and JDK-7 minimum JDK >>> >> >>> >> > > >>> >> >>> >> > > This would guarantee that none of the java 7 language features went >>> >> >>> >> > > in, but we could move trunk up to java 7+ only libraries (jersey, >>> >> >>> >> > > guava). Adopting >>> >> >>> >> > > JDK7 features then becomes no more different from adopting java7+ >>> >> >>> >> > > libraries: those bits of code that have moved can't be backported. >>> >> >>> >> > > >>> >> >>> >> > > -Steve >>> >> >>> >> > > >>> >> >>> >> > > >>> >> >>> >> > > >>> >> >>> >> > > >>> >> >>> >> > > >>> >> >>> >> > > On 17 June 2014 22:08, Andrew Wang <andrew.w...@cloudera.com >>> <mailto: >>> >> andrew.w...@cloudera.com>> >>> >> >>> >> > wrote: >>> >> >>> >> > > >>> >> >>> >> > > > Reviving this thread, I noticed there's been a patch and +1 on >>> >> >>> >> > > > HADOOP-10530, and I don't think we actually reached a conclusion. >>> >> >>> >> > > > >>> >> >>> >> > > > I (and others) have expressed concerns about moving to JDK7 for >>> >> trunk. >>> >> >>> >> > > > Summarizing a few points: >>> >> >>> >> > > > >>> >> >>> >> > > > - We can't move to JDK7 in branch-2 because of compatibility >>> >> >>> >> > > > - branch-2 is currently the only Hadoop release vehicle, there are >>> >> >>> >> > > > no >>> >> >>> >> > > plans >>> >> >>> >> > > > for a trunk-based Hadoop 3 >>> >> >>> >> > > > - Introducing JDK7-only APIs in trunk will increase divergence >>> with >>> >> >>> >> > > > branch-2 and make backports harder >>> >> >>> >> > > > - Almost all developers care only about branch-2, since it is the >>> >> >>> >> > > > only release vehicle >>> >> >>> >> > > > >>> >> >>> >> > > > With this in mind, I struggle to see any upsides to introducing >>> >> >>> >> > > > JDK7-only APIs to trunk. Please let's not do anything on >>> >> >>> >> > > > HADOOP-10530 or related until we agree on this. >>> >> >>> >> > > > >>> >> >>> >> > > > Thanks, >>> >> >>> >> > > > Andrew >>> >> >>> >> > > > >>> >> >>> >> > > > >>> >> >>> >> > > > On Mon, Apr 14, 2014 at 3:31 PM, Steve Loughran >>> >> >>> >> > > > <ste...@hortonworks.com<mailto:ste...@hortonworks.com>> >>> >> >>> >> > > > wrote: >>> >> >>> >> > > > >>> >> >>> >> > > > > On 14 April 2014 17:46, Andrew Purtell <apurt...@apache.org >>> >> <mailto:apurt...@apache.org>> wrote: >>> >> >>> >> > > > > >>> >> >>> >> > > > > > How well is trunk tested? Does anyone deploy it with real >>> >> >>> >> > > applications >>> >> >>> >> > > > > > running on top? When will the trunk codebase next be the basis >>> >> >>> >> > > > > > for a production release? An impromptu diff of hadoop-common >>> >> >>> >> > > > > > trunk against >>> >> >>> >> > > > > > branch-2 as of today is 38,625 lines. Can they be said to be >>> the >>> >> >>> >> > > > > > same animal? I ask because any disincentive toward putting >>> code >>> >> >>> >> > > > > > in trunk >>> >> >>> >> > > is >>> >> >>> >> > > > > > beside the point, if the only target worth pursuing today is >>> >> >>> >> > > > > > branch-2 unless one doesn't care if the code is released for >>> >> >>> >> > production use. >>> >> >>> >> > > > > > Questions on whither JDK6 or JDK7+ (or JRE6 versus JRE7+) only >>> >> >>> >> > > > > > matter >>> >> >>> >> > > > for >>> >> >>> >> > > > > > the vast majority of Hadoopers if talking about branch-2. >>> >> >>> >> > > > > > >>> >> >>> >> > > > > > >>> >> >>> >> > > > > I think its partly a timescale issue; its also because the 1-2 >>> >> >>> >> > > transition >>> >> >>> >> > > > > was so significant, especially at the YARN layer, that it's >>> still >>> >> >>> >> > > taking >>> >> >>> >> > > > > time to trickle through. >>> >> >>> >> > > > > >>> >> >>> >> > > > > If you do want code to ship this year, branch-2 is where you are >>> >> >>> >> > > > > going >>> >> >>> >> > > to >>> >> >>> >> > > > > try and get it in -and like you say, that's where things get >>> tried >>> >> >>> >> > > > > in >>> >> >>> >> > > the >>> >> >>> >> > > > > field. At the same time, the constraints of stability are >>> holding >>> >> >>> >> > > > > us >>> >> >>> >> > > back >>> >> >>> >> > > > > -already-. >>> >> >>> >> > > > > >>> >> >>> >> > > > > I don't see why we should have such another major 1-2 transition >>> >> >>> >> > > > > in >>> >> >>> >> > > > future; >>> >> >>> >> > > > > the rate that Arun is pushing out 2.x releases its almost back >>> to >>> >> >>> >> > > > > the >>> >> >>> >> > > > 0.1x >>> >> >>> >> > > > > timescale -though at that point most people were fending for >>> >> >>> >> > > > > themselves >>> >> >>> >> > > > and >>> >> >>> >> > > > > expectations of stability were less. We do want smaller version >>> >> >>> >> > > > increments >>> >> >>> >> > > > > in future, which branch-2 is -mostly- delivering. >>> >> >>> >> > > > > >>> >> >>> >> > > > > While Java 7 doesn't have some must-have features, Java 8 is a >>> >> >>> >> > > > significant >>> >> >>> >> > > > > improvement in the language, and we should be looking ahead to >>> >> >>> >> > > > > that, >>> >> >>> >> > > > maybe >>> >> >>> >> > > > > even doing some leading-edge work on the side, so the same >>> >> >>> >> > > > > discussion doesn't come up in two years time when java 7 goes >>> EOL. >>> >> >>> >> > > > > >>> >> >>> >> > > > > >>> >> >>> >> > > > > -steve >>> >> >>> >> > > > > >>> >> >>> >> > > > > (personal opinions only, etc, ) >>> >> >>> >> > > > > >>> >> >>> >> > > > > >>> >> >>> >> > > > > > >>> >> >>> >> > > > > > On Mon, Apr 14, 2014 at 9:22 AM, Colin McCabe < >>> >> >>> >> > > cmcc...@alumni.cmu.edu<mailto:cmcc...@alumni.cmu.edu> >>> >> >>> >> > > > > > >wrote: >>> >> >>> >> > > > > > >>> >> >>> >> > > > > > > I think the bottom line here is that as long as our stable >>> >> >>> >> > > > > > > release uses JDK6, there is going to be a very, very strong >>> >> >>> >> > > > > > > disincentive to put any code which can't run on JDK6 into >>> >> trunk. >>> >> >>> >> > > > > > > >>> >> >>> >> > > > > > > Like I said earlier, the traditional reason for putting >>> >> >>> >> > > > > > > something >>> >> >>> >> > > in >>> >> >>> >> > > > > > > trunk but not the stable release is that it needs more >>> testing. >>> >> >>> >> > > If a >>> >> >>> >> > > > > > > stable release that drops support for JDK6 is more than a >>> year >>> >> >>> >> > > away, >>> >> >>> >> > > > > > > does it make sense to put anything in trunk like that? What >>> >> >>> >> > > > > > > might need more than a year of testing? Certainly not >>> changes >>> >> >>> >> > > > > > > to LocalFileSystem to use the new APIs. I also don't think >>> an >>> >> >>> >> > > > > > > upgrade >>> >> >>> >> > > > to >>> >> >>> >> > > > > > > various libraries qualifies. >>> >> >>> >> > > > > > > >>> >> >>> >> > > > > > > It might be best to shelve this for now, like we've done in >>> >> >>> >> > > > > > > the >>> >> >>> >> > > past, >>> >> >>> >> > > > > > > until we're ready to talk about a stable release that >>> requires >>> >> >>> >> > > JDK7+. >>> >> >>> >> > > > > > > At least that's my feeling. >>> >> >>> >> > > > > > > >>> >> >>> >> > > > > > > If we're really desperate for the new file APIs JDK7 >>> provides, >>> >> >>> >> > > > > > > we could consider using loadable modules for it in branch-2. >>> >> >>> >> > > > > > > This is similar to how we provide JNI versions of certain >>> >> >>> >> > > > > > > things on certain platforms, without dropping support for >>> the >>> >> other >>> >> >>> >> > platforms. >>> >> >>> >> > > > > > > >>> >> >>> >> > > > > > > best, >>> >> >>> >> > > > > > > Colin >>> >> >>> >> > > > > > > >>> >> >>> >> > > > > > > On Sun, Apr 13, 2014 at 10:39 AM, Raymie Stata < >>> >> >>> >> > > rst...@altiscale.com<mailto:rst...@altiscale.com> >>> >> >>> >> > > > > >>> >> >>> >> > > > > > > wrote: >>> >> >>> >> > > > > > > > There's an outstanding question addressed to me: "Are >>> there >>> >> >>> >> > > > > particular >>> >> >>> >> > > > > > > > features or new dependencies that you would like to >>> >> >>> >> > > > > > > > contribute >>> >> >>> >> > > (or >>> >> >>> >> > > > > see >>> >> >>> >> > > > > > > > contributed) that require using the Java 1.7 APIs?" The >>> >> >>> >> > > > > > > > question misses the point: We'd figure out how to write >>> >> >>> >> > > > > > > > something we >>> >> >>> >> > > wanted >>> >> >>> >> > > > to >>> >> >>> >> > > > > > > > contribute to Hadoop against the APIs of Java4 if that's >>> >> >>> >> > > > > > > > what it >>> >> >>> >> > > > took >>> >> >>> >> > > > > > > > to get them into a stable release. And at current course >>> >> >>> >> > > > > > > > and >>> >> >>> >> > > > speed, >>> >> >>> >> > > > > > > > that's how ridiculous things could get. >>> >> >>> >> > > > > > > > >>> >> >>> >> > > > > > > > To summarize, it seems like there's a vague consensus that >>> >> >>> >> > > > > > > > it >>> >> >>> >> > > might >>> >> >>> >> > > > > be >>> >> >>> >> > > > > > > > okay to eventually allow the use of Java7 in trunk, but >>> >> >>> >> > > > > > > > there's >>> >> >>> >> > > no >>> >> >>> >> > > > > > > > decision. And there's been no answer to the concern that >>> >> >>> >> > > > > > > > even if >>> >> >>> >> > > > > such >>> >> >>> >> > > > > > > > dependencies were allowed in Java7, the only people using >>> >> >>> >> > > > > > > > them >>> >> >>> >> > > > would >>> >> >>> >> > > > > > > > be people who uninterested in getting their patches into a >>> >> >>> >> > > > > > > > stable release of Hadoop on any knowable timeframe, which >>> >> >>> >> > > > > > > > doesn't bode >>> >> >>> >> > > > well >>> >> >>> >> > > > > > > > for the ability to stabilize that Java7 code when it comes >>> >> >>> >> > > > > > > > time >>> >> >>> >> > > to >>> >> >>> >> > > > > > > > attempt to. >>> >> >>> >> > > > > > > > >>> >> >>> >> > > > > > > > I don't have more to add, so I'll go back to lurking. >>> It'll >>> >> >>> >> > > > > > > > be interesting to see where we'll be standing a year from >>> >> now. >>> >> >>> >> > > > > > > > >>> >> >>> >> > > > > > > > On Sun, Apr 13, 2014 at 2:09 AM, Tsuyoshi OZAWA >>> >> >>> >> > > > > > > > <ozawa.tsuyo...@gmail.com<mailto:ozawa.tsuyo...@gmail.com >>> >> >>> >> wrote: >>> >> >>> >> > > > > > > >> Hi, >>> >> >>> >> > > > > > > >> >>> >> >>> >> > > > > > > >> +1 for Karthik's idea(non-binding). >>> >> >>> >> > > > > > > >> >>> >> >>> >> > > > > > > >> IMO, we should keep the compatibility between JDK 6 and >>> JDK >>> >> >>> >> > > > > > > >> 7 on >>> >> >>> >> > > > > both >>> >> >>> >> > > > > > > branch-1 >>> >> >>> >> > > > > > > >> and branch-2, because users can be using them. For future >>> >> >>> >> > > releases >>> >> >>> >> > > > > > that >>> >> >>> >> > > > > > > we can >>> >> >>> >> > > > > > > >> declare breaking compatibility(e.g. 3.0.0 release), we >>> can >>> >> >>> >> > > > > > > >> use >>> >> >>> >> > > > JDK 7 >>> >> >>> >> > > > > > > >> features if we >>> >> >>> >> > > > > > > >> can get benefits. However, it can increase maintenance >>> >> >>> >> > > > > > > >> costs and >>> >> >>> >> > > > > > > distributes the >>> >> >>> >> > > > > > > >> efforts of contributions to maintain branches. Then, I >>> >> >>> >> > > > > > > >> think it >>> >> >>> >> > > is >>> >> >>> >> > > > > > > >> reasonable approach >>> >> >>> >> > > > > > > >> that we use limited and minimum JDK-7 APIs when we have >>> >> >>> >> > > > > > > >> reasons >>> >> >>> >> > > we >>> >> >>> >> > > > > > need >>> >> >>> >> > > > > > > to use >>> >> >>> >> > > > > > > >> the features. >>> >> >>> >> > > > > > > >> By the way, if we start to use JDK 7 APIs, we should >>> >> >>> >> > > > > > > >> declare the >>> >> >>> >> > > > > basis >>> >> >>> >> > > > > > > >> when to use >>> >> >>> >> > > > > > > >> JDK 7 APIs on Wiki not to confuse contributors. >>> >> >>> >> > > > > > > >> >>> >> >>> >> > > > > > > >> Thanks, >>> >> >>> >> > > > > > > >> - Tsuyoshi >>> >> >>> >> > > > > > > >> >>> >> >>> >> > > > > > > >> On Wed, Apr 9, 2014 at 11:44 AM, Raymie Stata < >>> >> >>> >> > > > rst...@altiscale.com<mailto:rst...@altiscale.com> >>> >> >>> >> > > > > > >>> >> >>> >> > > > > > > wrote: >>> >> >>> >> > > > > > > >>>> It might make sense to try to enumerate the benefits of >>> >> >>> >> > > > switching >>> >> >>> >> > > > > to >>> >> >>> >> > > > > > > >>>> Java7 APIs and dependencies. >>> >> >>> >> > > > > > > >>> >>> >> >>> >> > > > > > > >>> - Java7 introduced a huge number of language, >>> byte-code, >>> >> >>> >> > > > > > > >>> API, >>> >> >>> >> > > > and >>> >> >>> >> > > > > > > >>> tooling enhancements! Just to name a few: >>> >> >>> >> > > > > > > >>> try-with-resources, >>> >> >>> >> > > > > newer >>> >> >>> >> > > > > > > >>> and stronger encyrption methods, more scalable >>> concurrency >>> >> >>> >> > > > > > primitives. >>> >> >>> >> > > > > > > >>> See >>> >> >>> >> > > > > > > >>> >>> http://www.slideshare.net/boulderjug/55-things-in-java-7 >>> >> >>> >> > > > > > > >>> >>> >> >>> >> > > > > > > >>> - We can't update current dependencies, and we can't >>> add >>> >> >>> >> > > > > > > >>> cool >>> >> >>> >> > > > new >>> >> >>> >> > > > > > > ones. >>> >> >>> >> > > > > > > >>> >>> >> >>> >> > > > > > > >>> - Putting language/APIs aside, don't forget that a >>> huge >>> >> >>> >> > > amount >>> >> >>> >> > > > of >>> >> >>> >> > > > > > > effort >>> >> >>> >> > > > > > > >>> goes into qualifying for Java6 (at least, I hope the >>> folks >>> >> >>> >> > > > claiming >>> >> >>> >> > > > > > to >>> >> >>> >> > > > > > > >>> support Java6 are putting in such an effort :-). >>> Wouldn't >>> >> >>> >> > > Hadoop >>> >> >>> >> > > > > > > >>> users/customers be better served if qualification effort >>> >> >>> >> > > > > > > >>> went >>> >> >>> >> > > > into >>> >> >>> >> > > > > > > >>> Java7/8 versus Java6/7? >>> >> >>> >> > > > > > > >>> >>> >> >>> >> > > > > > > >>> Getting to Java7 as a development env (and Java8 as a >>> >> >>> >> > > > > > > >>> runtime >>> >> >>> >> > > > env) >>> >> >>> >> > > > > > > >>> seems like a no-brainer. Question is: How? >>> >> >>> >> > > > > > > >>> >>> >> >>> >> > > > > > > >>> On Tue, Apr 8, 2014 at 10:21 AM, Sandy Ryza < >>> >> >>> >> > > > > sandy.r...@cloudera.com<mailto:sandy.r...@cloudera.com> >>> >> >>> >> > > > > > > >>> >> >>> >> > > > > > > wrote: >>> >> >>> >> > > > > > > >>>> It might make sense to try to enumerate the benefits of >>> >> >>> >> > > > switching >>> >> >>> >> > > > > to >>> >> >>> >> > > > > > > Java7 >>> >> >>> >> > > > > > > >>>> APIs and dependencies. IMO, the ones listed so far on >>> >> >>> >> > > > > > > >>>> this >>> >> >>> >> > > > thread >>> >> >>> >> > > > > > > don't >>> >> >>> >> > > > > > > >>>> make a compelling enough case to drop Java6 in branch-2 >>> >> >>> >> > > > > > > >>>> on any >>> >> >>> >> > > > > time >>> >> >>> >> > > > > > > frame, >>> >> >>> >> > > > > > > >>>> even if this means supporting Java6 through 2015. For >>> >> >>> >> > > example, >>> >> >>> >> > > > > the >>> >> >>> >> > > > > > > change >>> >> >>> >> > > > > > > >>>> in RawLocalFileSystem semantics might be an >>> incompatible >>> >> >>> >> > > change >>> >> >>> >> > > > > for >>> >> >>> >> > > > > > > >>>> branch-2 any way. >>> >> >>> >> > > > > > > >>>> >>> >> >>> >> > > > > > > >>>> >>> >> >>> >> > > > > > > >>>> On Tue, Apr 8, 2014 at 10:05 AM, Karthik Kambatla < >>> >> >>> >> > > > > > ka...@cloudera.com<mailto:ka...@cloudera.com> >>> >> >>> >> > > > > > > >wrote: >>> >> >>> >> > > > > > > >>>> >>> >> >>> >> > > > > > > >>>>> +1 to NOT breaking compatibility in branch-2. >>> >> >>> >> > > > > > > >>>>> >>> >> >>> >> > > > > > > >>>>> I think it is reasonable to require JDK7 for trunk, if >>> >> >>> >> > > > > > > >>>>> we >>> >> >>> >> > > limit >>> >> >>> >> > > > > use >>> >> >>> >> > > > > > > of >>> >> >>> >> > > > > > > >>>>> JDK7-only API to security fixes etc. If we make other >>> >> >>> >> > > > > optimizations >>> >> >>> >> > > > > > > (like >>> >> >>> >> > > > > > > >>>>> IO), it would be a pain to backport things to >>> branch-2. >>> >> >>> >> > > > > > > >>>>> I >>> >> >>> >> > > guess >>> >> >>> >> > > > > > this >>> >> >>> >> > > > > > > all >>> >> >>> >> > > > > > > >>>>> depends on when we see ourselves shipping Hadoop-3. >>> Any >>> >> >>> >> > > > > > > >>>>> ideas >>> >> >>> >> > > > on >>> >> >>> >> > > > > > > that? >>> >> >>> >> > > > > > > >>>>> >>> >> >>> >> > > > > > > >>>>> >>> >> >>> >> > > > > > > >>>>> On Tue, Apr 8, 2014 at 9:19 AM, Eli Collins < >>> >> >>> >> > > e...@cloudera.com<mailto:e...@cloudera.com>> >>> >> >>> >> > > > > > > wrote: >>> >> >>> >> > > > > > > >>>>> >>> >> >>> >> > > > > > > >>>>> > On Tue, Apr 8, 2014 at 2:00 AM, Ottenheimer, Davi >>> >> >>> >> > > > > > > >>>>> > <davi.ottenhei...@emc.com<mailto: >>> >> davi.ottenhei...@emc.com>> wrote: >>> >> >>> >> > > > > > > >>>>> > >> From: Eli Collins [mailto:e...@cloudera.com] >>> >> >>> >> > > > > > > >>>>> > >> Sent: Monday, April 07, 2014 11:54 AM >>> >> >>> >> > > > > > > >>>>> > >> >>> >> >>> >> > > > > > > >>>>> > >> >>> >> >>> >> > > > > > > >>>>> > >> IMO we should not drop support for Java 6 in a >>> >> >>> >> > > > > > > >>>>> > >> minor >>> >> >>> >> > > > update >>> >> >>> >> > > > > > of a >>> >> >>> >> > > > > > > >>>>> stable >>> >> >>> >> > > > > > > >>>>> > >> release (v2). I don't think the larger Hadoop >>> user >>> >> >>> >> > > > > > > >>>>> > >> base >>> >> >>> >> > > > > would >>> >> >>> >> > > > > > > find it >>> >> >>> >> > > > > > > >>>>> > >> acceptable that upgrading to a minor update >>> caused >>> >> >>> >> > > > > > > >>>>> > >> their >>> >> >>> >> > > > > > > systems to >>> >> >>> >> > > > > > > >>>>> stop >>> >> >>> >> > > > > > > >>>>> > >> working because they didn't upgrade Java. There >>> are >>> >> >>> >> > > people >>> >> >>> >> > > > > > still >>> >> >>> >> > > > > > > >>>>> getting >>> >> >>> >> > > > > > > >>>>> > >> support for Java 6. ... >>> >> >>> >> > > > > > > >>>>> > >> >>> >> >>> >> > > > > > > >>>>> > >> Thanks, >>> >> >>> >> > > > > > > >>>>> > >> Eli >>> >> >>> >> > > > > > > >>>>> > > >>> >> >>> >> > > > > > > >>>>> > > Hi Eli, >>> >> >>> >> > > > > > > >>>>> > > >>> >> >>> >> > > > > > > >>>>> > > Technically you are correct those with extended >>> >> >>> >> > > > > > > >>>>> > > support >>> >> >>> >> > > get >>> >> >>> >> > > > > > > critical >>> >> >>> >> > > > > > > >>>>> > security fixes for 6 until the end of 2016. I am >>> >> >>> >> > > > > > > >>>>> > curious >>> >> >>> >> > > > > whether >>> >> >>> >> > > > > > > many of >>> >> >>> >> > > > > > > >>>>> > those are in the Hadoop user base. Do you know? My >>> >> >>> >> > > > > > > >>>>> > guess is >>> >> >>> >> > > > the >>> >> >>> >> > > > > > > vast >>> >> >>> >> > > > > > > >>>>> > majority are within Oracle's official public end of >>> >> >>> >> > > > > > > >>>>> > life, >>> >> >>> >> > > > which >>> >> >>> >> > > > > > > was over >>> >> >>> >> > > > > > > >>>>> 12 >>> >> >>> >> > > > > > > >>>>> > months ago. Even Premier support ended Dec 2013: >>> >> >>> >> > > > > > > >>>>> > > >>> >> >>> >> > > > > > > >>>>> > > http://www.oracle.com/technetwork/java/eol-< >>> >> http://www.oracle.com/technetwork/java/eol-135779.ht> >>> >> >>> >> > 135779.ht<http://www.oracle.com/technetwork/java/eol-135779.ht> >>> >> >>> >> > > > > > > >>>>> > > ml >>> >> >>> >> > > > > > > >>>>> > > >>> >> >>> >> > > > > > > >>>>> > > The end of Java 6 support carries much risk. It >>> has >>> >> >>> >> > > > > > > >>>>> > > to be >>> >> >>> >> > > > > > > considered in >>> >> >>> >> > > > > > > >>>>> > terms of serious security vulnerabilities such as >>> >> >>> >> > > > CVE-2013-2465 >>> >> >>> >> > > > > > > with CVSS >>> >> >>> >> > > > > > > >>>>> > score 10.0. >>> >> >>> >> > > > > > > >>>>> > > >>> >> >>> >> > > > > > > >>>>> > > http://www.cvedetails.com/cve/CVE-2013-2465/ >>> >> >>> >> > > > > > > >>>>> > > >>> >> >>> >> > > > > > > >>>>> > > Since you mentioned "caused systems to stop" as an >>> >> >>> >> > > example >>> >> >>> >> > > > of >>> >> >>> >> > > > > > > what >>> >> >>> >> > > > > > > >>>>> would >>> >> >>> >> > > > > > > >>>>> > be a concern to Hadoop users, please note the >>> >> >>> >> > > > > > > >>>>> > CVE-2013-2465 >>> >> >>> >> > > > > > > availability >>> >> >>> >> > > > > > > >>>>> > impact: >>> >> >>> >> > > > > > > >>>>> > > >>> >> >>> >> > > > > > > >>>>> > > "Complete (There is a total shutdown of the >>> affected >>> >> >>> >> > > > > resource. >>> >> >>> >> > > > > > > The >>> >> >>> >> > > > > > > >>>>> > attacker can render the resource completely >>> >> unavailable.)" >>> >> >>> >> > > > > > > >>>>> > > >>> >> >>> >> > > > > > > >>>>> > > This vulnerability was patched in Java 6 Update >>> 51, >>> >> >>> >> > > > > > > >>>>> > > but >>> >> >>> >> > > > post >>> >> >>> >> > > > > > end >>> >> >>> >> > > > > > > of >>> >> >>> >> > > > > > > >>>>> > life. Apple pushed out the update specifically >>> because >>> >> >>> >> > > > > > > >>>>> > of >>> >> >>> >> > > > this >>> >> >>> >> > > > > > > >>>>> > vulnerability (http://support.apple.com/kb/HT5717) >>> as >>> >> >>> >> > > > > > > >>>>> > did >>> >> >>> >> > > > some >>> >> >>> >> > > > > > > other >>> >> >>> >> > > > > > > >>>>> > vendors privately, but for the majority of people >>> >> >>> >> > > > > > > >>>>> > using >>> >> >>> >> > > Java >>> >> >>> >> > > > 6 >>> >> >>> >> > > > > > > means they >>> >> >>> >> > > > > > > >>>>> > have a ticking time bomb. >>> >> >>> >> > > > > > > >>>>> > > >>> >> >>> >> > > > > > > >>>>> > > Allowing it to stay should be considered in terms >>> of >>> >> >>> >> > > > > accepting >>> >> >>> >> > > > > > > the >>> >> >>> >> > > > > > > >>>>> whole >>> >> >>> >> > > > > > > >>>>> > risk posture. >>> >> >>> >> > > > > > > >>>>> > > >>> >> >>> >> > > > > > > >>>>> > >>> >> >>> >> > > > > > > >>>>> > There are some who get extended support, but I >>> suspect >>> >> >>> >> > > > > > > >>>>> > many >>> >> >>> >> > > > > just >>> >> >>> >> > > > > > > have >>> >> >>> >> > > > > > > >>>>> > a if-it's-not-broke mentality when it comes to >>> >> >>> >> > > > > > > >>>>> > production >>> >> >>> >> > > > > > > deployments. >>> >> >>> >> > > > > > > >>>>> > The current code supports both java6 and java7 and >>> so >>> >> >>> >> > > allows >>> >> >>> >> > > > > > these >>> >> >>> >> > > > > > > >>>>> > people to remain compatible, while enabling others >>> to >>> >> >>> >> > > upgrade >>> >> >>> >> > > > > to >>> >> >>> >> > > > > > > the >>> >> >>> >> > > > > > > >>>>> > java7 runtime. This seems like the right compromise >>> >> >>> >> > > > > > > >>>>> > for a >>> >> >>> >> > > > > stable >>> >> >>> >> > > > > > > >>>>> > release series. Again, absolutely makes sense for >>> >> >>> >> > > > > > > >>>>> > trunk (ie >>> >> >>> >> > > > v3) >>> >> >>> >> > > > > > to >>> >> >>> >> > > > > > > >>>>> > require java7 or greater. >>> >> >>> >> > > > > > > >>>>> > >>> >> >>> >> > > > > > > >>>>> >>> >> >>> >> > > > > > > >> >>> >> >>> >> > > > > > > >> >>> >> >>> >> > > > > > > >> >>> >> >>> >> > > > > > > >> -- >>> >> >>> >> > > > > > > >> - Tsuyoshi >>> >> >>> >> > > > > > > >>> >> >>> >> > > > > > >>> >> >>> >> > > > > > >>> >> >>> >> > > > > > >>> >> >>> >> > > > > > -- >>> >> >>> >> > > > > > Best regards, >>> >> >>> >> > > > > > >>> >> >>> >> > > > > > - Andy >>> >> >>> >> > > > > > >>> >> >>> >> > > > > > Problems worthy of attack prove their worth by hitting back. - >>> >> >>> >> > > > > > Piet >>> >> >>> >> > > > Hein >>> >> >>> >> > > > > > (via Tom White) >>> >> >>> >> > > > > > >>> >> >>> >> > > > > >>> >> >>> >> > > > > -- >>> >> >>> >> > > > > CONFIDENTIALITY NOTICE >>> >> >>> >> > > > > NOTICE: This message is intended for the use of the individual >>> or >>> >> >>> >> > > entity >>> >> >>> >> > > > to >>> >> >>> >> > > > > which it is addressed and may contain information that is >>> >> >>> >> > > > > confidential, privileged and exempt from disclosure under >>> >> >>> >> > > > > applicable law. If the >>> >> >>> >> > > reader >>> >> >>> >> > > > > of this message is not the intended recipient, you are hereby >>> >> >>> >> > > > > notified >>> >> >>> >> > > > that >>> >> >>> >> > > > > any printing, copying, dissemination, distribution, disclosure >>> or >>> >> >>> >> > > > > forwarding of this communication is strictly prohibited. If you >>> >> >>> >> > > > > have received this communication in error, please contact the >>> >> >>> >> > > > > sender >>> >> >>> >> > > > immediately >>> >> >>> >> > > > > and delete it from your system. Thank You. >>> >> >>> >> > > > > >>> >> >>> >> > > > >>> >> >>> >> > > >>> >> >>> >> > > -- >>> >> >>> >> > > CONFIDENTIALITY NOTICE >>> >> >>> >> > > NOTICE: This message is intended for the use of the individual or >>> >> >>> >> > > entity to which it is addressed and may contain information that is >>> >> >>> >> > > confidential, privileged and exempt from disclosure under applicable >>> >> >>> >> > > law. If the reader of this message is not the intended recipient, >>> you >>> >> >>> >> > > are hereby notified that any printing, copying, dissemination, >>> >> >>> >> > > distribution, disclosure or forwarding of this communication is >>> >> >>> >> > > strictly prohibited. If you have received this communication in >>> error, >>> >> >>> >> > > please contact the sender immediately and delete it from your >>> system. >>> >> >>> >> > Thank You. >>> >> >>> >> > > >>> >> >>> >> >> >> >> -- >> Best regards, >> >> - Andy >> >> Problems worthy of attack prove their worth by hitting back. - Piet Hein >> (via Tom White)
