Re: Heads up: next hadoop-2 release

2012-09-01 Thread Todd Lipcon
On Fri, Aug 31, 2012 at 1:15 PM, Eli Collins e...@cloudera.com wrote: Yea, I think we should nuke 2.1.0-alpha and re-create when we're actually going to do a release. On the HDFS side there's quite a few things already waiting to get out, if it's going to take another 4 or so weeks then would

Re: Heads up: next hadoop-2 release

2012-09-01 Thread Andrew Purtell
Hi Arun et.al. I'd still be grateful if you could share what candidates from 0.23 you would have cherry picked into 2.1.0, maybe the critical ones? I will need to make this analysis at this point. Pointers would be appreciated very much. On Friday, August 31, 2012, Vinod Kumar Vavilapalli wrote:

Re: Large feature development

2012-09-01 Thread Todd Lipcon
Thanks for starting this thread, Steve. I think your points below are good. I've snipped most of your comment and will reply inline to one bit below: On Fri, Aug 31, 2012 at 10:07 AM, Steve Loughran steve.lough...@gmail.com wrote: Of the big changes that have worked, they are 1. HDFS 2's

Re: [DISCUSS] Spin out MR, HDFS and YARN as their own TLPs and disband Hadoop umbrella project

2012-09-01 Thread Sharad Agarwal
Hadoop 2.0 has been years of work in the making and is finally relatively close. I think it would be a mistake to throw another impediment in the way of getting a stable version of 2.0 done, as many folks have pointed out. So I'd suggest that we plan to do a split once there is a broad

Re: [DISCUSS] Spin out MR, HDFS and YARN as their own TLPs and disband Hadoop umbrella project

2012-09-01 Thread Andrew Purtell
As a downstream user of and contributor to BigTop (though only once, I know we need to do better, Roman), it would be awesome to see the community rally around it as an integration point if the project splits into finer grained components yet. On Friday, August 31, 2012, Roman Shaposhnik wrote:

Re: [DISCUSS] Spin out MR, HDFS and YARN as their own TLPs and disband Hadoop umbrella project

2012-09-01 Thread Andrew Purtell
Or resurrect MR(v1) in Apache Hadoop as Apache YARN becomes a TLP, and let the new YARN TLP decide if they want to use the Hadoop MR artifacts and/or contribute patches that harmonize the implementation with theirs, or pursue an alternate MR implementation within their larger framework. I'd

Re: Large feature development

2012-09-01 Thread Arun C Murthy
Todd, On Sep 1, 2012, at 1:20 AM, Todd Lipcon wrote: I'd actually contend that YARN was merged too early. I have yet to see anyone running YARN in production, and it's holding up the Stable moniker for Hadoop 2.0 -- HDFS-wise we are already quite stable and I'm seeing fewer issues in our

Re: Large feature development

2012-09-01 Thread Rajiv Chittajallu
Its unfortunate that certain work, an year after accepted in to the main line, being attributed to a single person. There is significant amount of work done by people who are not in the PMC or a commiter, especially to get it running in production. For those who have been associated with

Re: Large feature development

2012-09-01 Thread Arun Murthy
Rajiv, I'm pretty sure you mean '*blame* for certain work, [...], being attributed' ... :) I certainly find blame for failures much more palatable than credit for collective successes. IAC, thanks for chiming in, Hadoop will be better with you being more present at the forefront. Arun On Sep