Lars, so it worked for me, and I'm more than happy for anyone to use/adapt it as necessary for hbase proper. But I'm not sure it's anywhere near production ready, and I don't have the time to work on it more right now. Perhaps someone with more knowledge of region internals could vet it and add relevant tests. We could enter a JIRA, and if I find time in the future I can take a look.
And yes, @JM, my gist was specific to an online migration (cluster is active, but table is disabled). Offline did not meet our requirements at the time, so I never tried it. On Thu, Aug 28, 2014 at 5:04 PM, lars hofhansl <la...@apache.org> wrote: > Agreed. > > Bryan, we should pull in your code if that works better. > > -- Lars > > > > ________________________________ > From: Andrew Purtell <apurt...@apache.org> > To: "user@hbase.apache.org" <user@hbase.apache.org> > Cc: Development <developm...@mentacapital.com> > Sent: Thursday, August 28, 2014 12:12 PM > Subject: Re: state-of-the-art method for merging regions on v0.94 > > > If the 0.94 merge code doesn't work out the box we should fix that. > > > > > > On Thu, Aug 28, 2014 at 11:26 AM, Bryan Beaudreault < > bbeaudrea...@hubspot.com> wrote: > > > I've done it. This is the code I used: > > https://gist.github.com/bbeaudreault/7567385 > > > > It comes from the hbase source, but is modified to actually work (the > class > > provided in hbase is private and does not work out of the box). There is > a > > readme at the bottom of the gist with my process. One important note > > though, I did this with a deep understanding (after hours of reading > hbase > > code and doing tests on a test cluster) of how it all works. And even > then > > I felt nervous to do it in prod. Hence why I went the snapshot/compact > > route. > > > > I would definitely test it on a test cluster and get some familiarity > > before getting close to a production table. That said, I've run this on > > 8-10 production tables a few months ago, reducing in size from 10-20x in > > some cases. > > > > > > On Thu, Aug 28, 2014 at 2:19 PM, Ted Tuttle <t...@mentacapital.com> > wrote: > > > > > Hello- > > > > > > We recently realized our region size is 1G and need to increase it to > get > > > our region count under control. I've done some research on merging > > regions > > > and have come away confused. > > > > > > There is the ops handbook: > > > > > > http://hbase.apache.org/book/ops.regionmgt.html > > > > > > And then there is this horror story: > > > > > > http://metabroadcast.com/blog/so-you-broke-hbase > > > > > > Is there someone out there that has done a large scale (i.e. 10:1 > > > reduction on 10k's of regions) merge successfully on HBase 0.94? If > so, > > > how did you do it? > > > > > > Thanks, > > > Ted > > > > > > > > > > > > -- > Best regards, > > - Andy > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > (via Tom White) >
