On Sat, May 8, 2010 at 9:59 AM, Thomas Koch <tho...@koch.ro> wrote: > I'm a little confused and concerned now that I learn that hbase uses a > patches > hadoop. For Debian I use plain hadoop under hbase and it seems to work in > testing environments.
> - Are these patches necessary to run HBase? > It will work unless you have failures, in which case it will lose edits. HBase relies on the "hflush" API (called "sync" in 0.20) which does not work properly in 0.20 without significant patching. Without this patch series, HBase will certainly run, but I could never recommend running it in a production environment where data loss is a show-stopper. > - Where can I find these patches? > Currently they're in various places on the JIRA - HDFS-200, HDFS-142, HDFS-826, HDFS-561, etc. I have a github branch up which contains them all applied, but I haven't tested it beyond unit tests - my testing is all happening in our CDH3 tree, and afaik Dhruba's testing is on their FB internal tree. > - Why aren't these patches included in hadoop? Are they too unstable? > Yes, the policy is not to make such significant changes in patch releases, so they would need to be voted into the 0.20 series. It's not that they're entirely unstable, it's just that the code is very tricky and still under development. The upcoming 0.21 release has a *different* implementation of Append which also hasn't been tested significantly in real life failure scenarios, but it's important that we keep the stable release stable. > - If they're unstable, does this mean, HBase is unstable? > > Again I would not say it's terribly unstable - but it's nowhere near the level of stability that Hadoop is > Should I worry at all about these patches for the Debian packages? If you expect that people might actually want to run a production HBase, they should have the patches. If you expect people to just be playing around on single node "clusters" where failures aren't an issue, best to skip. Of course, even for production usage, I wouldn't recommend running what we've got now - wait a month or two and it should be one more notch up the stability/testing scale. -Todd -- Todd Lipcon Software Engineer, Cloudera
