Re: Hadoop support for hbase

2010-05-10 Thread Allen Wittenauer
Let me understand this: a) the hbase folks have been required to patch hadoop due to bugs b) they have been doing this for X months now c) we finally have momentum on getting 0.21 out the door d) hey, let's make their life easier and take resources out of 0.21 by creating a branch Are we

Re: Hadoop support for hbase

2010-05-10 Thread Stack
On Mon, May 10, 2010 at 9:58 AM, Allen Wittenauer awittena...@linkedin.com wrote: Let me understand this: a) the hbase folks have been required to patch hadoop due to bugs The branch is to work on adding a feature to 0.20, not for fixing bugs. c) we finally have momentum on getting 0.21 out

Re: Hadoop support for hbase

2010-05-10 Thread Allen Wittenauer
On May 10, 2010, at 10:18 AM, Stack wrote: The above is a fallacious setup. How does a branch in 0.20 detract from the 0.21 momentum (The append feature that we'd work on in 0.20 branch has little relation to how append works in 0.21). There are X amount of hours that people can put into

Re: Hadoop support for hbase

2010-05-10 Thread Ryan Rawson
On Mon, May 10, 2010 at 10:24 AM, Allen Wittenauer awittena...@linkedin.com wrote: On May 10, 2010, at 10:18 AM, Stack wrote: The above is a fallacious setup.  How does a branch in 0.20 detract from the 0.21 momentum (The append feature that we'd work on in 0.20 branch has little relation to

Re: Hadoop support for hbase

2010-05-10 Thread Allen Wittenauer
On May 10, 2010, at 11:05 AM, Ryan Rawson wrote: That's not how it works though - people have adopted and use Hadoop 0.20 because of the fact that people like Yahoo, Facebook, etc run it on multi-thousand node clusters and have done so for months (or soon to be years now). If you look

RE: Hadoop support for hbase

2010-05-10 Thread Michael Segel
From: t...@cloudera.com Date: Mon, 10 May 2010 10:45:13 -0700 Subject: Re: Hadoop support for hbase To: general@hadoop.apache.org The above is a fallacious setup. How does a branch in 0.20 detract from the 0.21 momentum (The append feature that we'd work on in 0.20 branch has

Re: Hadoop support for hbase

2010-05-10 Thread Jay Booth
PM, Michael Segel michael_se...@hotmail.com wrote: From: t...@cloudera.com Date: Mon, 10 May 2010 10:45:13 -0700 Subject: Re: Hadoop support for hbase To: general@hadoop.apache.org The above is a fallacious setup.  How does a branch in 0.20 detract from the 0.21 momentum (The append

Re: Hadoop support for hbase

2010-05-09 Thread Stack
On Fri, May 7, 2010 at 10:34 AM, Dhruba Borthakur dhr...@gmail.com wrote: My proposal is to create a new branch off the hadoop 0.20 branch and name it branch-0.20-hbase. It will have support for append/sync and will be API compatible with the hadoop 0.20 branch. However, this branch will be

Re: Hadoop support for hbase

2010-05-09 Thread Stack
On Fri, May 7, 2010 at 10:34 AM, Dhruba Borthakur dhr...@gmail.com wrote: This branch will contain all of hdfs/mapreduce/core. Should it be all of hadoop? Could it be hdfs only? St.Ack

Re: Hadoop support for hbase

2010-05-09 Thread Stack
On Sun, May 9, 2010 at 3:15 PM, Stack st...@duboce.net wrote: Should it be all of hadoop?  Could it be hdfs only? Please ignore the above question (I just took a look at 0.20 repo). St.Ack

Re: Hadoop support for hbase

2010-05-09 Thread Dhruba Borthakur
I will try to explain my opinion on some of the questions being raised so far. Of course all these are open for discussion and nothing is final at this moment. 1. This code base is primarily targeted for usage by HBase and Scribe. Both of these are Apache open source projects. 2. Accordingly, my

Re: Hadoop support for hbase

2010-05-08 Thread Todd Lipcon
On Sat, May 8, 2010 at 9:59 AM, Thomas Koch tho...@koch.ro wrote: I'm a little confused and concerned now that I learn that hbase uses a patches hadoop. For Debian I use plain hadoop under hbase and it seems to work in testing environments. - Are these patches necessary to run HBase? It

Hadoop support for hbase

2010-05-07 Thread Dhruba Borthakur
Hi folks, I would like to open a discussion on how we can make HBase work well with a supported/released version of Hadoop. HBase currently ships with a hadoop jar and that hadoop jar is from hadoop 0.20 + a set of ten/twenty patches. Most of these patches are focussed on HDFS append support in

Re: Hadoop support for hbase

2010-05-07 Thread Todd Lipcon
I have a few questions about this proposal: 1) Will we open new JIRAs separately for each change we want to commit, and go through the normal review process? Currently the 20-append work has been mostly going on under HDFS-142 for whatever reason, with ancillary issues only for bugs that also

Re: Hadoop support for hbase

2010-05-07 Thread Doug Cutting
On 05/07/2010 10:57 AM, Todd Lipcon wrote: 1) Will we open new JIRAs separately for each change we want to commit, and go through the normal review process? Currently the 20-append work has been mostly going on under HDFS-142 for whatever reason, with ancillary issues only for bugs that also