Can you file a JIRA ? Thanks
On Jul 2, 2011, at 10:36 AM, Lars George <[email protected]> wrote: > Hi, > > These two seem both in a bit of a weird state: HMerge is scoped package > local, therefore no one but the package can call the merge() functions... and > no one does that but the unit test. But it would be good to have this on the > CLI and shell as a command (and in the shell maybe with a confirmation > message?), but it is not available AFAIK. > > HMerge can merge regions of tables that are disabled. It also merges all that > qualify, i.e. where the merged region is less than or equal of half the > configured max file size. > > Merge on the other hand does have a main(), so can be invoked: > > $ hbase org.apache.hadoop.hbase.util.Merge > Usage: bin/hbase merge <table-name> <region-1> <region-2> > > Note how the help insinuates that you can use it as a tool, but that is not > correct. Also, it only merges two given regions, and the cluster must be shut > down (only the HBase daemons). So that is a step back. > > What is worse is that I cannot get it to work. I tried in the shell: > > hbase(main):001:0> create 'testtable', 'colfam1', {SPLITS => > ['row-10','row-20','row-30','row-40','row-50']} > 0 row(s) in 0.2640 seconds > > hbase(main):002:0> for i in '0'..'9' do for j in '0'..'9' do put 'testtable', > "row-#{i}#{j}", "colfam1:#{j}", "#{j}" end end > 0 row(s) in 1.0450 seconds > > hbase(main):003:0> flush 'testtable' > 0 row(s) in 0.2000 seconds > > hbase(main):004:0> scan '.META.', { COLUMNS => ['info:regioninfo']} > ROW COLUMN+CELL > testtable,,1309614509037.612d1e0112 column=info:regioninfo, timestamp=130... > 406e6c2bb482eeaec57322. STARTKEY => '', ENDKEY => 'row-10' > testtable,row-10,1309614509040.2fba column=info:regioninfo, timestamp=130... > fcc9bc6afac94c465ce5dcabc5d1. STARTKEY => 'row-10', ENDKEY => 'row-20' > testtable,row-20,1309614509041.e7c1 column=info:regioninfo, timestamp=130... > 6267eb30e147e5d988c63d40f982. STARTKEY => 'row-20', ENDKEY => 'row-30' > testtable,row-30,1309614509041.a9cd column=info:regioninfo, timestamp=130... > e1cbc7d1a21b1aca2ac7fda30ad8. STARTKEY => 'row-30', ENDKEY => 'row-40' > testtable,row-40,1309614509041.d458 column=info:regioninfo, timestamp=130... > 236feae097efcf33477e7acc51d4. STARTKEY => 'row-40', ENDKEY => 'row-50' > testtable,row-50,1309614509041.74a5 column=info:regioninfo, timestamp=130... > 7dc7e3e9602d9229b15d4c0357d1. STARTKEY => 'row-50', ENDKEY => '' > 6 row(s) in 0.0440 seconds > > hbase(main):005:0> exit > > $ ./bin/stop-hbase.sh > > $ hbase org.apache.hadoop.hbase.util.Merge testtable \ > testtable,row-20,1309614509041.e7c16267eb30e147e5d988c63d40f982. \ > testtable,row-30,1309614509041.a9cde1cbc7d1a21b1aca2ac7fda30ad8. > > But I get consistently errors: > > 11/07/02 07:20:49 INFO util.Merge: Merging regions > testtable,row-20,1309613053987.23a35ac696bdf4a8023dcc4c5b8419e0. and > testtable,row-30,1309613053987.3664920956c30ac5ff2a7726e4e6 in table testtable > 11/07/02 07:20:49 INFO wal.HLog: HLog configuration: blocksize=32 MB, > rollsize=30.4 MB, enabled=true, optionallogflushinternal=1000ms > 11/07/02 07:20:49 INFO wal.HLog: New hlog > /Volumes/Macintosh-HD/Users/larsgeorge/.logs_1309616449171/hlog.1309616449181 > 11/07/02 07:20:49 INFO wal.HLog: getNumCurrentReplicas--HDFS-826 not > available; hdfs_out=org.apache.hadoop.fs.FSDataOutputStream@25961581, > exception=org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.getNumCurrentReplicas() > 11/07/02 07:20:49 INFO regionserver.HRegion: Setting up tabledescriptor > config now ... > 11/07/02 07:20:49 INFO regionserver.HRegion: Onlined -ROOT-,,0.70236052; next > sequenceid=1 > info: null > region1: [B@48fd918a > region2: [B@7f5e2075 > 11/07/02 07:20:49 FATAL util.Merge: Merge failed > java.io.IOException: Could not find meta region for > testtable,row-20,1309613053987.23a35ac696bdf4a8023dcc4c5b8419e0. > at org.apache.hadoop.hbase.util.Merge.mergeTwoRegions(Merge.java:211) > at org.apache.hadoop.hbase.util.Merge.run(Merge.java:111) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) > at org.apache.hadoop.hbase.util.Merge.main(Merge.java:386) > 11/07/02 07:20:49 INFO regionserver.HRegion: Setting up tabledescriptor > config now ... > 11/07/02 07:20:49 INFO regionserver.HRegion: Onlined .META.,,1.1028785192; > next sequenceid=1 > 11/07/02 07:20:49 INFO regionserver.HRegion: Closed -ROOT-,,0.70236052 > 11/07/02 07:20:49 INFO wal.HLog: main.logSyncer exiting > 11/07/02 07:20:49 ERROR util.Merge: exiting due to error > java.lang.NullPointerException > at org.apache.hadoop.hbase.util.Merge$1.processRow(Merge.java:119) > at > org.apache.hadoop.hbase.util.MetaUtils.scanMetaRegion(MetaUtils.java:229) > at > org.apache.hadoop.hbase.util.MetaUtils.scanMetaRegion(MetaUtils.java:258) > at org.apache.hadoop.hbase.util.Merge.run(Merge.java:116) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) > at org.apache.hadoop.hbase.util.Merge.main(Merge.java:386) > > After which I most of the times have shot .META. with an error > > 2011-07-02 06:42:10,763 WARN org.apache.hadoop.hbase.master.HMaster: Failed > getting all descriptors > java.io.FileNotFoundException: No status for > hdfs://localhost:8020/hbase/.corrupt > at > org.apache.hadoop.hbase.util.FSUtils.getTableInfoModtime(FSUtils.java:888) > at > org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:122) > at > org.apache.hadoop.hbase.util.FSTableDescriptors.getAll(FSTableDescriptors.java:149) > at > org.apache.hadoop.hbase.master.HMaster.getHTableDescriptors(HMaster.java:1429) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:312) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1065) > > Lars
