Great work, guys! -----Original Message----- From: saint....@gmail.com [mailto:saint....@gmail.com] On Behalf Of Stack Sent: Thursday, June 09, 2011 5:03 PM To: dev@hbase.apache.org Cc: apurt...@apache.org Subject: Re: HDFS-1599 status? (HDFS tickets to improve HBase)
HDFS-941 just got committed to TRUNK in a coordinated effort led by our man Todd (Hopefully it makes it into 0.22!). HDFS-1148 is next! St.Ack On Sun, Jun 5, 2011 at 10:09 PM, Todd Lipcon <t...@cloudera.com> wrote: > OK guys, I did my part: I rebased HDFS-941 and HDFS-1148 to trunk. > Also attempted to rebase HDFS-1323 (nee 918) but it's failing tests. > > If you want to help, please take a look at the failing tests on > HDFS-1323 and see if you can understand what might be going wrong. > > Unfortunately this isn't my top priority at work right now (HA for the > namenode is), but I'm happy to spend some nights and weekends to help > push these through if they really work. > > -Todd > > On Sun, Jun 5, 2011 at 3:40 PM, Todd Lipcon <t...@cloudera.com> wrote: > >> On Sat, Jun 4, 2011 at 1:46 AM, Andrew Purtell <apurt...@apache.org>wrote: >> >>> This is not discouraging. :-) >>> >>> HBasers patch CDH because trunk -- anything > 0.20 actually -- is >>> not trusted by consensus if you look at all of the production >>> deployments. Does ANYONE run trunk under anything approaching >>> "production"? And trunk/upstream has a history of ignoring any HBase >>> specific concern. So the use of and trading of patches will probably >>> continue for a while, maybe forever. >>> >> >> Right - I wasn't suggesting that you run trunk in production as of >> yet. But there has been very little activity in terms of HBase people >> running trunk in dev/test clusters in the past. Stack has done some >> awesome work here in the last few weeks, so that should open it up >> for some more people to jump on board. >> >> I agree that HBase has been treated as a second-class citizen in >> recent years from HDFS's performance, but I think that has changed. >> All of the major HDFS contributors now have serious stakes in HBase, >> and so long as there are tests with sufficient testing that apply >> against trunk, I don't see a reason they wouldn't be included. >> >> >>> >>> Part of the problem is the expectation that any patch provided >>> against trunk may generate months of back and forth, as we have >>> seen, which presents difficulities to a potential contributor who >>> does not work on e.g. HDFS matters full time. Alternatively it may >>> pick up a committer as sponsor and then be vetoed by Yahoo because >>> they're mad at Cloudera over some unrelated issue and a patch appears to >>> have a Cloudera sponsor and/or or vice versa. >>> Now, that situation I describe _is_ discouraging. It's not enough to >>> say that we must contribute through trunk. Trunk needs to earn back our >>> trust. >>> >> >> Yes, there have been some unfortunate things in the past. There have >> also been some half-finished or untested patches proposed, and you >> can't blame HDFS folks for not taking a big patch that doesn't have a >> lot of confidence behind it. >> >> I've been thinking about this this afternoon, and have an idea. It >> may prove to be an awful one, but maybe it's a good one, only time >> will tell :) I'll create a branch off of HDFS trunk specifically for >> HBase performance work. We can commit these "90% done" patches there, >> which will make it easier for others to test and gain confidence. >> Branches also can make it easier to maintain patches over time with a >> changing trunk. >> >> How does this sound to the HBase community? If it seems like a good >> idea, >> *and* there are some people who would be willing to set it up on some >> small dev clusters and run load tests, I'll move forward with it. >> >> >>> I believe I recently saw discussion that append should be removed or >>> disabled by default on 0.22 or trunk. Did you see anything like >>> this? If I am mistaken, fine. If not, this is going in the wrong >>> direction, for example. >>> >> >> Not sure what you're referring to - I don't remember any discussion >> like this. >> >> -Todd >> -- >> Todd Lipcon >> Software Engineer, Cloudera >> > > > > -- > Todd Lipcon > Software Engineer, Cloudera >
