RE: Plans of moving towards JDK7 in trunk

2014-06-18 Thread Ottenheimer, Davi
?  The

   question misses the point: We'd figure out how to write

   something we

  wanted

   to

   contribute to Hadoop against the APIs of Java4 if that's

   what it

   took

   to get them into a stable release.  And at current course

   and

   speed,

   that's how ridiculous things could get.

  

   To summarize, it seems like there's a vague consensus that

   it

  might

be

   okay to eventually allow the use of Java7 in trunk, but

   there's

  no

   decision.  And there's been no answer to the concern that

   even if

such

   dependencies were allowed in Java7, the only people using

   them

   would

   be people who uninterested in getting their patches into a

   stable release of Hadoop on any knowable timeframe, which

   doesn't bode

   well

   for the ability to stabilize that Java7 code when it comes

   time

  to

   attempt to.

  

   I don't have more to add, so I'll go back to lurking.  It'll

   be interesting to see where we'll be standing a year from now.

  

   On Sun, Apr 13, 2014 at 2:09 AM, Tsuyoshi OZAWA

   ozawa.tsuyo...@gmail.commailto:ozawa.tsuyo...@gmail.com wrote:

   Hi,

  

   +1 for Karthik's idea(non-binding).

  

   IMO, we should keep the compatibility between JDK 6 and JDK

   7 on

both

  branch-1

   and branch-2, because users can be using them. For future

  releases

 that

  we can

   declare breaking compatibility(e.g. 3.0.0 release), we can

   use

   JDK 7

   features if we

   can get benefits. However, it can increase maintenance

   costs and

  distributes the

   efforts of contributions to maintain branches. Then, I

   think it

  is

   reasonable approach

   that we use limited and minimum JDK-7 APIs when we have

   reasons

  we

 need

  to use

   the features.

   By the way, if we start to use JDK 7 APIs, we should

   declare the

basis

   when to use

   JDK 7 APIs on Wiki not to confuse contributors.

  

   Thanks,

   - Tsuyoshi

  

   On Wed, Apr 9, 2014 at 11:44 AM, Raymie Stata 

   rst...@altiscale.commailto:rst...@altiscale.com



  wrote:

   It might make sense to try to enumerate the benefits of

   switching

to

   Java7 APIs and dependencies.

  

 - Java7 introduced a huge number of language, byte-code,

   API,

   and

   tooling enhancements!  Just to name a few:

   try-with-resources,

newer

   and stronger encyrption methods, more scalable concurrency

 primitives.

See

   http://www.slideshare.net/boulderjug/55-things-in-java-7

  

 - We can't update current dependencies, and we can't add

   cool

   new

  ones.

  

 - Putting language/APIs aside, don't forget that a huge

  amount

   of

  effort

   goes into qualifying for Java6 (at least, I hope the folks

   claiming

 to

   support Java6 are putting in such an effort :-).  Wouldn't

  Hadoop

   users/customers be better served if qualification effort

   went

   into

   Java7/8 versus Java6/7?

  

   Getting to Java7 as a development env (and Java8 as a

   runtime

   env)

   seems like a no-brainer.  Question is: How?

  

   On Tue, Apr 8, 2014 at 10:21 AM, Sandy Ryza 

sandy.r...@cloudera.commailto:sandy.r...@cloudera.com

 

  wrote:

   It might make sense to try to enumerate the benefits of

   switching

to

  Java7

   APIs and dependencies.  IMO, the ones listed so far on

   this

   thread

  don't

   make a compelling enough case to drop Java6 in branch-2

   on any

time

  frame,

   even if this means supporting Java6 through 2015.  For

  example,

the

  change

   in RawLocalFileSystem semantics might be an incompatible

  change

for

   branch-2 any way.

  

  

   On Tue, Apr 8, 2014 at 10:05 AM, Karthik Kambatla 

 ka...@cloudera.commailto:ka...@cloudera.com

  wrote:

  

   +1 to NOT breaking compatibility in branch-2.

  

   I think it is reasonable to require JDK7 for trunk, if

   we

  limit

use

  of

   JDK7-only API to security fixes etc. If we make other

optimizations

  (like

   IO), it would be a pain to backport things to branch-2.

   I

  guess

 this

  all

   depends on when we see ourselves shipping Hadoop-3. Any

   ideas

   on

  that?

  

  

   On Tue, Apr 8, 2014 at 9:19 AM, Eli Collins 

  e...@cloudera.commailto:e...@cloudera.com

  wrote:

  

On Tue, Apr 8, 2014 at 2:00 AM, Ottenheimer, Davi

davi.ottenhei...@emc.commailto:davi.ottenhei...@emc.com 
wrote

RE: Plans of moving towards JDK7 in trunk

2014-04-08 Thread Ottenheimer, Davi
 From: Eli Collins [mailto:e...@cloudera.com]
 Sent: Monday, April 07, 2014 11:54 AM
 
 
 IMO we should not drop support for Java 6 in a minor update of a stable
 release (v2).  I don't think the larger Hadoop user base would find it
 acceptable that upgrading to a minor update caused their systems to stop
 working because they didn't upgrade Java. There are people still getting
 support for Java 6. ...
 
 Thanks,
 Eli

Hi Eli, 

Technically you are correct those with extended support get critical security 
fixes for 6 until the end of 2016. I am curious whether many of those are in 
the Hadoop user base. Do you know? My guess is the vast majority are within 
Oracle's official public end of life, which was over 12 months ago. Even 
Premier support ended Dec 2013:

http://www.oracle.com/technetwork/java/eol-135779.html

The end of Java 6 support carries much risk. It has to be considered in terms 
of serious security vulnerabilities such as CVE-2013-2465 with CVSS score 10.0. 

http://www.cvedetails.com/cve/CVE-2013-2465/

Since you mentioned caused systems to stop as an example of what would be a 
concern to Hadoop users, please note the CVE-2013-2465 availability impact:

Complete (There is a total shutdown of the affected resource. The attacker can 
render the resource completely unavailable.)

This vulnerability was patched in Java 6 Update 51, but post end of life. Apple 
pushed out the update specifically because of this vulnerability 
(http://support.apple.com/kb/HT5717) as did some other vendors privately, but 
for the majority of people using Java 6 means they have a ticking time bomb. 

Allowing it to stay should be considered in terms of accepting the whole risk 
posture.

Davi


RE: Coverity Scan (MAPREDUCE-5032)

2013-08-26 Thread Ottenheimer, Davi
Perhaps open the JIRA with only a reference/link to the Coverity report, and 
limit access to only those working on the issues. 

Full disclosure, update the JIRA, after fix.

--
Davi Ottenheimer
Senior Director of Trust
EMC Corporation
davi.ottenhei...@emc.com | @daviottenheimer | +1-415-271-6259
blog: http://www.flyingpenguin.com/


 -Original Message-
 From: shaposh...@gmail.com [mailto:shaposh...@gmail.com] On Behalf Of
 Roman Shaposhnik
 Sent: Monday, August 26, 2013 10:50 AM
 To: common-dev@hadoop.apache.org
 Subject: Re: Coverity Scan (MAPREDUCE-5032)
 
 On Mon, Aug 26, 2013 at 10:43 AM, Vinod Kumar Vavilapalli
 vino...@apache.org wrote:
 
  Can you file a JIRA and attach the report there? That is the best way to
 move this forward.
 
 Last time I was involved in a Coverity scan was when they scanned another
 project I'm committer on (FFmpeg). The lesson there was that the value you
 get out of browsing on their site https://scan.coverity.com is immeasurably
 higher than from any static report that can be attached to a JIRA.
 
 Also, at least in FFmpeg's case, Coverity identified a few things that 
 could've
 been used as potential exploits so it made perfect sense to have a white-list
 of project members who could get access to the initial report instead of going
 all public with it to begin with (which would happen if it just gets attached 
 to
 a JIRA in its entirety).
 
 Just my 2c worth of working with them in the past.
 
 Thanks,
 Roman.