Re: Thinking ahead to hadoop-2.6

2014-11-05 Thread Arun C Murthy
>>>>> >>>>>>>> >>>>>>>> >>>>>>>> I actually would like to see both archival storage and single replica >>>>>>>> memory writes to be in 2.6 release. Archival storage is in the

Re: Thinking ahead to hadoop-2.6

2014-10-23 Thread Yongjun Zhang
t;>>>> in the final stages to be merged to branch-2. >> >>>>>> >> >>>>>> Regards, >> >>>>>> Tsz-Wo >> >>>>>> >> >>>>>> >> >>>>>> >> >>>&g

Re: Thinking ahead to hadoop-2.6

2014-10-19 Thread Yongjun Zhang
00 AM, Suresh Srinivas < > >>>>>> sur...@hortonworks.com> wrote: > >>>>>> > >>>>>> > >>>>>>> > >>>>>>> > >>>>>>> I actually would like to see b

Re: Thinking ahead to hadoop-2.6

2014-10-15 Thread Vinod Kumar Vavilapalli
rks.com> wrote: > >>>>>> > >>>>>> > >>>>>>> > >>>>>>> > >>>>>>> I actually would like to see both archival storage and single > replica > >>>>>>

Re: Thinking ahead to hadoop-2.6

2014-10-14 Thread Arun C Murthy
6581 gets ready sooner. Both of >>>>> these >>>>>>> features are being in development for sometime. >>>>>>> >>>>>>> On Tue, Sep 23, 2014 at 3:27 PM, Andrew Wang < >>>> andrew.w...@cloudera.com> >&

Re: Thinking ahead to hadoop-2.6

2014-10-02 Thread Arun C Murthy
; stages >>>>>>> of getting ready for branch-2 merge as Nicholas has already indicated >>>> on >>>>>>> the dev mailing list. Hopefully HDFS-6581 gets ready sooner. Both of >>>>> these >>>>>>> features are bei

Re: Thinking ahead to hadoop-2.6

2014-09-26 Thread Arun Murthy
ngly? >>>>>> >>>>>> I think the KMS and transparent encryption are ready to go. We've >> got >>> a >>>>>> very few further bug fixes pending, but that's it. >>>>>> >>>>>> Two

Re: Thinking ahead to hadoop-2.6

2014-09-24 Thread Vinod Kumar Vavilapalli
eds a little more work before it goes into > > branch-2. > > > >> HDFS-6581 hasn't even been merged to trunk yet, so seems a bit > further > > > off > > > >> yet. > > > >> > > > >&g

Re: Thinking ahead to hadoop-2.6

2014-09-24 Thread Vinod Kumar Vavilapalli
it - hopefully in the next few weeks. > > Thoughts? > > thanks, > Arun > > On Tue, Aug 12, 2014 at 1:34 PM, Arun C Murthy > wrote: > > > Folks, > > > > With hadoop-2.5 nearly done, it's time to start thinking ahead to > > hadoop-

Re: Thinking ahead to hadoop-2.6

2014-09-24 Thread Jitendra Pandey
ingle replica memory writes > > > (HDFS-6581), > > > >> which I believe are under the HSM banner. HDFS-6484 was just merged > to > > > >> trunk and I think needs a little more work before it goes into > > branch-2. > > > >> HDFS-6581 hasn'

Re: Thinking ahead to hadoop-2.6

2014-09-24 Thread Suresh Srinivas
work directly on these features. I just > > generally > > >> shy away from shipping bits quite this fresh. > > >> > > >> Thanks, > > >> Andrew > > >> > > >> On Tue, Sep 23, 2014 at 3:03 PM,

Re: Thinking ahead to hadoop-2.6

2014-09-24 Thread Andrew Wang
t;> Andrew > >> > >> On Tue, Sep 23, 2014 at 3:03 PM, Arun Murthy > wrote: > >> > >> > Looks like most of the content is in and hadoop-2.6 is shaping up > nicely. > >> > > >> > I'll create branch-2.6 by end of the week

Re: Thinking ahead to hadoop-2.6

2014-09-23 Thread Tsz Wo (Nicholas), Sze
n Tue, Sep 23, 2014 at 3:03 PM, Arun Murthy wrote: >> >> > Looks like most of the content is in and hadoop-2.6 is shaping up nicely. >> > >> > I'll create branch-2.6 by end of the week and we can go from there to >> > stabilize it - hopefully in the n

Re: Thinking ahead to hadoop-2.6

2014-09-23 Thread Sangjin Lee
did not work directly on these features. I just > > > generally > > > > > shy away from shipping bits quite this fresh. > > > > > > > > > > Thanks, > > > > > Andrew > > > > > > > > > > On Tue, Sep 2

Re: Thinking ahead to hadoop-2.6

2014-09-23 Thread Subramaniam V K
t; > generally > > > > shy away from shipping bits quite this fresh. > > > > > > > > Thanks, > > > > Andrew > > > > > > > > On Tue, Sep 23, 2014 at 3:03 PM, Arun Murthy > > > wrote: > > > > > > &g

Re: Thinking ahead to hadoop-2.6

2014-09-23 Thread Karthik Kambatla
he content is in and hadoop-2.6 is shaping up > > nicely. > > > > > > > > I'll create branch-2.6 by end of the week and we can go from there to > > > > stabilize it - hopefully in the next few weeks. > > > > > > > > Thoughts? &g

Re: Thinking ahead to hadoop-2.6

2014-09-23 Thread Andrew Wang
; > > > > Looks like most of the content is in and hadoop-2.6 is shaping up > nicely. > > > > > > I'll create branch-2.6 by end of the week and we can go from there to > > > stabilize it - hopefully in the next few weeks. > > > > > >

Re: Thinking ahead to hadoop-2.6

2014-09-23 Thread Chris Trezzo
run Murthy > wrote: > > > > > Looks like most of the content is in and hadoop-2.6 is shaping up > nicely. > > > > > > I'll create branch-2.6 by end of the week and we can go from there to > > > stabilize it - hopefully in the next few weeks. > &g

Re: Thinking ahead to hadoop-2.6

2014-09-23 Thread Suresh Srinivas
to > > stabilize it - hopefully in the next few weeks. > > > > Thoughts? > > > > thanks, > > Arun > > > > On Tue, Aug 12, 2014 at 1:34 PM, Arun C Murthy > > wrote: > > > > > Folks, > > > > > > With hadoop-2.5

Re: Thinking ahead to hadoop-2.6

2014-09-23 Thread Andrew Wang
; > I'll create branch-2.6 by end of the week and we can go from there to > stabilize it - hopefully in the next few weeks. > > Thoughts? > > thanks, > Arun > > On Tue, Aug 12, 2014 at 1:34 PM, Arun C Murthy > wrote: > > > Folks, > > > >

Re: Thinking ahead to hadoop-2.6

2014-09-23 Thread Arun Murthy
gt; > With hadoop-2.5 nearly done, it's time to start thinking ahead to > hadoop-2.6. > > Currently, here is the Roadmap per the wiki: > > • HADOOP > • Credential provider HADOOP-10607 > • HDFS > • Heterogeneous storage (

Re: In hindsight... Re: Thinking ahead to hadoop-2.6

2014-09-23 Thread Arun Murthy
Sorry, coming to discussion late. We all agreed that 2.6 would the *last* release supporting JDK6 and hadoop-2.7 would drop support for JDK6. We could easily do 2.7 right after 2.6 (maybe with few critical bug-fixes) with the defining feature of 2.7 being *JDK7 only*. I've checked with HBase, Pig

Re: In hindsight... Re: Thinking ahead to hadoop-2.6

2014-09-17 Thread Travis Thompson
There's actually an umbrella JIRA to track issues with JDK8 (HADOOP-11090), in case anyone missed it. At LinkedIn we've been running our Hadoop 2.3 deployment on JDK8 for about a month now with some mixed results. It definitely works but there are issues, mostly around virtual memory exploding.

Re: In hindsight... Re: Thinking ahead to hadoop-2.6

2014-09-17 Thread Allen Wittenauer
On Sep 17, 2014, at 2:47 AM, Steve Loughran wrote: > > I don't agree. Certainly the stuff I got into Hadoop 2.5 nailed down the > filesystem binding with more tests than ever before. FWIW, based upon my survey of JIRA, there are a lot of unit test fixes that are only in trunk. > But

Re: In hindsight... Re: Thinking ahead to hadoop-2.6

2014-09-17 Thread Steve Loughran
gt; to > > the list? We have most of the patches for the sub-JIRAs under review and > > have committed a couple. > > > > -Subru > > > > -- Forwarded message -- > > > > From: Arun C Murthy > > > > Date: Tue, Aug 12, 2014 a

Re: In hindsight... Re: Thinking ahead to hadoop-2.6

2014-09-15 Thread Allen Wittenauer
On Sep 15, 2014, at 11:17 AM, Colin McCabe wrote: > On Mon, Sep 15, 2014 at 10:48 AM, Allen Wittenauer wrote: >> >>It’s now September. With the passage of time, I have a lot of doubts >> about this plan and where that trajectory takes us. >> >> * The list of changes that are already

Re: In hindsight... Re: Thinking ahead to hadoop-2.6

2014-09-15 Thread Colin McCabe
e patches for the sub-JIRAs under review and >> have committed a couple. >> >> -Subru >> >> -- Forwarded message -- >> >> From: Arun C Murthy >> >> Date: Tue, Aug 12, 2014 at 1:34 PM >> >> Subject: Thinking ahead to ha

In hindsight... Re: Thinking ahead to hadoop-2.6

2014-09-15 Thread Allen Wittenauer
he sub-JIRAs under review and > have committed a couple. > > -Subru > > -- Forwarded message -- > > From: Arun C Murthy > > Date: Tue, Aug 12, 2014 at 1:34 PM > > Subject: Thinking ahead to hadoop-2.6 > > To: "common-...@hadoop.apa

Re: Thinking ahead to hadoop-2.6

2014-08-15 Thread Subramaniam Krishnan
e, Aug 12, 2014 at 1:34 PM Subject: Thinking ahead to hadoop-2.6 To: "common-...@hadoop.apache.org" , " hdfs-...@hadoop.apache.org" , " mapreduce-dev@hadoop.apache.org" , "yarn-...@hadoop.apache.org" Folks, With hadoop-2.5 nearly done, i

Thinking ahead to hadoop-2.6

2014-08-12 Thread Arun C Murthy
Folks, With hadoop-2.5 nearly done, it's time to start thinking ahead to hadoop-2.6. Currently, here is the Roadmap per the wiki: • HADOOP • Credential provider HADOOP-10607 • HDFS • Heterogeneous storage (Phase 2) - Support APIs for