Re: [DISCUSS] Move common, hdfs, mapreduce contrib components to apache-extras.org or elsewhere

2011-02-11 Thread Bernd Fondermann
-1. Move it away from TRUNK so it doesn't affect builds is a much better (and simpler) solution. If someone wants to revive it, he can within the bounds of Apache Hadoop and will become a part of the Hadoop community, which would be good. If you'd move it off-site, if the code ever gets new

Re: [VOTE] Abandon hdfsproxy HDFS contrib

2011-02-11 Thread Bernd Fondermann
On Fri, Feb 11, 2011 at 07:33, Ian Holsman had...@holsman.net wrote: On Feb 11, 2011, at 5:11 PM, Nigel Daley wrote: On Feb 10, 2011, at 9:24 PM, Owen O'Malley wrote: On Thu, Feb 10, 2011 at 7:40 PM, Nigel Daley nda...@mac.com wrote: I think the PMC should abandon the hdfsproxy HDFS

Re: [VOTE] Abandon hod Common contrib

2011-02-11 Thread Steve Loughran
On 11/02/2011 02:51, Nigel Daley wrote: I think the PMC should abandon the hod common contrib component. It's last meaningful contribution was February 2009: HADOOP-2898. Provide an option to specify a port range for Hadoop services provisioned by HOD. Contributed by Peeyush Bishnoi. There

Re: Abandon fuse-dfs HDFS contrib

2011-02-11 Thread Brian Bockelman
(Sorry, I apparently wasn't subscribed to general@hadoop... so this response is going to muck up your email client's threading) We use fuse-dfs heavily at approximately 10 sites for the LHC; at least 3 of them at the 1 petabyte level to deliver multiple terabytes of data per day. However, as

Re: [VOTE] Abandon fuse-dfs HDFS contrib

2011-02-11 Thread Owen O'Malley
On Thu, Feb 10, 2011 at 9:57 PM, Nigel Daley nda...@mac.com wrote: If a reasonable replacement was developed elsewhere, would you change your vote? I believe some folks were interested in developing this elsewhere. It would depend. As I have said before, our goal as a PMC is to produce useful

Re: [VOTE] Abandon mrunit MapReduce contrib

2011-02-11 Thread Eric Sammer
Owen: I think you make a fair point. The reason I think it still makes sense to bring mrunit out of Hadoop contrib is to: - start to simplify the build by breaking projects that are only clients of Hadoop libs out of contrib. - allow mrunit to have its own release cycle. This is, I think, the

Re: [VOTE] Abandon mrunit MapReduce contrib

2011-02-11 Thread Owen O'Malley
On Feb 11, 2011, at 8:02 AM, Eric Sammer wrote: - allow mrunit to have its own release cycle. This is, I think, the most important. If you submit your work to Apache we can evaluate it for inclusion in the 0.20.100 branch to get your changes released in a timely manner. I would

Re: [VOTE] Abandon fuse-dfs HDFS contrib

2011-02-11 Thread Nigel Daley
On Feb 11, 2011, at 7:33 AM, Owen O'Malley wrote: On Thu, Feb 10, 2011 at 9:57 PM, Nigel Daley nda...@mac.com wrote: If a reasonable replacement was developed elsewhere, would you change your vote? I believe some folks were interested in developing this elsewhere. It would depend. As

Re: [VOTE] Abandon fuse-dfs HDFS contrib

2011-02-11 Thread M. C. Srivas
On Thu, Feb 10, 2011 at 9:57 PM, Nigel Daley nda...@mac.com wrote: On Feb 10, 2011, at 9:22 PM, Owen O'Malley wrote: On Thu, Feb 10, 2011 at 6:58 PM, Nigel Daley nda...@mac.com wrote: I think the PMC should abandon the fuse-dfs HDFS contrib component. It's last meaningful

Re: [VOTE] Abandon mrunit MapReduce contrib

2011-02-11 Thread Eric Sammer
On Fri, Feb 11, 2011 at 11:48 AM, Owen O'Malley omal...@apache.org wrote: On Feb 11, 2011, at 8:02 AM, Eric Sammer wrote: - allow mrunit to have its own release cycle. This is, I think, the most important. If you submit your work to Apache we can evaluate it for inclusion in the

Re: [VOTE] Abandon fuse-dfs HDFS contrib

2011-02-11 Thread Brian Bockelman
On Feb 11, 2011, at 10:59 AM, M. C. Srivas wrote: On Thu, Feb 10, 2011 at 9:57 PM, Nigel Daley nda...@mac.com wrote: On Feb 10, 2011, at 9:22 PM, Owen O'Malley wrote: On Thu, Feb 10, 2011 at 6:58 PM, Nigel Daley nda...@mac.com wrote: I think the PMC should abandon the fuse-dfs HDFS

Re: Hadoop 0.22 Blockers

2011-02-11 Thread Nigel Daley
On Feb 11, 2011, at 9:41 AM, Allen Wittenauer wrote: On Feb 10, 2011, at 11:33 PM, Nigel Daley wrote: Tom has created a public Jira filter for 0.22 blockers (thanks Tom!): https://issues.apache.org/jira/secure/IssueNavigator.jspa?mode=hiderequestId=12313687 With the new version of Jira,

Re: [VOTE] Abandon mrunit MapReduce contrib

2011-02-11 Thread Mattmann, Chris A (388J)
Guys, BTW, if you need help or a mentor in Apache Incubator-ville for MRUnit, I would be happy to help. Cheers, Chris On Feb 11, 2011, at 9:04 AM, Eric Sammer wrote: On Fri, Feb 11, 2011 at 11:48 AM, Owen O'Malley omal...@apache.org wrote: On Feb 11, 2011, at 8:02 AM, Eric Sammer wrote:

Re: [VOTE] Abandon hod Common contrib

2011-02-11 Thread Allen Wittenauer
On Feb 10, 2011, at 6:51 PM, Nigel Daley wrote: I think the PMC should abandon the hod common contrib component. It's last meaningful contribution was February 2009: HADOOP-2898. Provide an option to specify a port range for Hadoop services provisioned by HOD. Contributed by Peeyush

Re: [DISCUSS] Move common, hdfs, mapreduce contrib components to apache-extras.org or elsewhere

2011-02-11 Thread Tom White
For contrib components that we elect not to remove, I suggest that we remove them from the main build, so that failures in a contrib component don't hinder the main build and release. This is what ZooKeeper does, for example. Tom On Fri, Feb 11, 2011 at 2:03 AM, Bernd Fondermann

Re: [VOTE] Abandon mrunit MapReduce contrib

2011-02-11 Thread Garrett Wu
On Fri, Feb 11, 2011 at 9:04 AM, Eric Sammer esam...@cloudera.com wrote: On Fri, Feb 11, 2011 at 11:48 AM, Owen O'Malley omal...@apache.org wrote: On Feb 11, 2011, at 8:02 AM, Eric Sammer wrote: - allow mrunit to have its own release cycle. This is, I think, the most important.

Re: [VOTE] Abandon fuse-dfs HDFS contrib

2011-02-11 Thread Grant Mackey
As a university researcher for file systems, having the fuse-dfs code makes life easy when trying to run i/o benchmark apps. I would hate to see it go. Here's my -1 - Grant Quoting Nigel Daley nda...@mac.com: I think the PMC should abandon the fuse-dfs HDFS contrib component. It's last

Re: [VOTE] Abandon fuse-dfs HDFS contrib

2011-02-11 Thread Mattmann, Chris A (388J)
-1 (not binding) as well. Cheers, Chris On Feb 11, 2011, at 11:37 AM, Grant Mackey wrote: As a university researcher for file systems, having the fuse-dfs code makes life easy when trying to run i/o benchmark apps. I would hate to see it go. Here's my -1 - Grant Quoting Nigel

Re: [VOTE] Abandon mrunit MapReduce contrib

2011-02-11 Thread Patrick Hunt
On Fri, Feb 11, 2011 at 9:44 AM, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Guys, BTW, if you need help or a mentor in Apache Incubator-ville for MRUnit, I would be happy to help. I was going to suggest the same thing (mrunit to incubator). I would also be happy to be a

Re: [VOTE] Abandon mrunit MapReduce contrib

2011-02-11 Thread Mattmann, Chris A (388J)
Awesome Patrick, we'd probably need one more active mentor. Any takers? After we get that, then we cook up a proposal on the Incubator wiki here [1], and follow the process here [2] to get started... Cheers, Chris [1] http://wiki.apache.org/incubator/MRUnitProposal [2]

Re: [DISCUSS] Move common, hdfs, mapreduce contrib components to apache-extras.org or elsewhere

2011-02-11 Thread Aaron Kimball
Tom, How do these contrib components get released then? If the intent of having the code is to eventually produce release artifacts that people can use, then allowing them to further degrade in releasability seems antithetical to the point of keeping the source around. I think users who download

Re: [VOTE] Abandon mrunit MapReduce contrib

2011-02-11 Thread Aaron Kimball
The main reason I am interested in removing MRUnit from Hadoop is that I believe that MRUnit deserves its own release cycle. I think this is in the best interest of its users. MRUnit is valuable to users of several different versions of Hadoop. But MRUnit has only ever been committed to version

Re: [VOTE] Abandon mrunit MapReduce contrib

2011-02-11 Thread Eric Sammer
Just to add to the option of going to incubator, I'm fine with that as well. Github was an easy thing to get started and I was under the impression we needed some greater degree of committer diversity and, frankly, a bigger project. If mrunit is a candidate, keeping this under the ASF umbrella is

SF Hadoop meetup report

2011-02-11 Thread Aaron Kimball
Hadoop fans, This week we held the second SF Hadoop meetup. About 70 individuals attended, and we again had several great discussions in parallel, using the same unconference format as the first meetup in January. The following break-out sessions were held: * Hadoop and Workflow Systems *

Re: [ANNOUNCEMENT] Yahoo focusing on Apache Hadoop, discontinuing The Yahoo Distribution of Hadoop

2011-02-11 Thread Owen O'Malley
On Jan 31, 2011, at 7:27 PM, Eric Baldeschwieler wrote: Unfortunately, we now have to sort out how to contribute several person-years worth of work to Apache to let us unwind the Yahoo! git repositories. We currently run two lines of Hadoop development, our sustaining program

Re: [DISCUSS] Move common, hdfs, mapreduce contrib components to apache-extras.org or elsewhere

2011-02-11 Thread Tom White
On Fri, Feb 11, 2011 at 1:03 PM, Aaron Kimball akimbal...@gmail.com wrote: Tom, How do these contrib components get released then? In source form - this is what ZooKeeper does. If the intent of having the code is to eventually produce release artifacts that people can use, then allowing

Re: [VOTE] Abandon failmon Common contrib

2011-02-11 Thread Nigel Daley
This component has been removed with Jira HADOOP-7136. Thanks, Nige On Feb 9, 2011, at 9:18 AM, Nigel Daley wrote: I think the PMC should abandon the failmon common contrib component. It's last meaningful contribution was it's original commit in August of 2008: HADOOP-3585. FailMon

Re: [VOTE] Abandon hod Common contrib

2011-02-11 Thread Nigel Daley
On Feb 11, 2011, at 9:48 AM, Allen Wittenauer wrote: On Feb 10, 2011, at 6:51 PM, Nigel Daley wrote: I think the PMC should abandon the hod common contrib component. It's last meaningful contribution was February 2009: HADOOP-2898. Provide an option to specify a port range for Hadoop

Re: [VOTE] Abandon thriftfs HDFS contrib

2011-02-11 Thread Dhruba Borthakur
+0. we are using this in production, but sadly I never had any time to keep it updated. I have a plan of integrating the generated Thrift server-stubs with the namenode itself, so that the NameNode can expose Hadoop RPC as well as Thrift RPC. But this is something for the future, so I am fine

Re: [VOTE] Abandon hod Common contrib

2011-02-11 Thread Owen O'Malley
On Feb 11, 2011, at 6:17 PM, Nigel Daley wrote: a) I don't think hod is actually part of any unit tests, so including it would likely only be a burden on the tarball size. Not true. HOD has python unit tests and is the reason our builds have dependencies on python. But Allen's point is

Re: [VOTE] Abandon hod Common contrib

2011-02-11 Thread Nigel Daley
On Feb 11, 2011, at 9:15 PM, Owen O'Malley wrote: On Feb 11, 2011, at 6:17 PM, Nigel Daley wrote: a) I don't think hod is actually part of any unit tests, so including it would likely only be a burden on the tarball size. Not true. HOD has python unit tests and is the reason our

Re: [VOTE] Abandon mrunit MapReduce contrib

2011-02-11 Thread Nigel Daley
This is great! So we'll leave mrunit in contrib until it can be moved to incubator. Nige On Feb 11, 2011, at 2:26 PM, Eric Sammer wrote: Just to add to the option of going to incubator, I'm fine with that as well. Github was an easy thing to get started and I was under the impression we

Re: [DISCUSS] Move common, hdfs, mapreduce contrib components to apache-extras.org or elsewhere

2011-02-11 Thread Nigel Daley
+1. And only include them as source in releases. Nige On Feb 11, 2011, at 10:12 AM, Tom White wrote: For contrib components that we elect not to remove, I suggest that we remove them from the main build, so that failures in a contrib component don't hinder the main build and release. This