I like this one: >> like making a malnourished child work in a coal mine
On Thu, Jun 23, 2011 at 10:03 AM, Jean-Daniel Cryans <[email protected]>wrote: > Getting RetriesExhaustedWithDetailsException due to NSRE means that it > took forever for a region server to close or split a region, what you > pasted from the region server talks a region closing but that also > happens during split. > > I'd suggest digging more in those region server logs using this guide: > http://hbase.apache.org/book/trouble.html > > Also make sure you review this > http://hbase.apache.org/book/performance.html > > Finally giving a 1GB heap to HBase while inserting a lot of data is > like making a malnourished child work in a coal mine, it's not very > nice of you. > > J-D > > On Wed, Jun 22, 2011 at 11:06 PM, Sam Seigal <[email protected]> wrote: > > Hi, > > > > I am loading data into my HBase cluster and running into two issues - > > > > During my import, I received the following exception -> > > > > org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: > Failed > > 53484 actions: servers with issues: spock7001:60020, > > at > > > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1220) > > at > > > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchOfPuts(HConnectionManager.java:1234) > > at > > org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:819) > > at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:675) > > at org.apache.hadoop.hbase.client.HTable.put(HTable.java:660) > > > > May have cluster issues => true > > Cause 0 > > > > When I check the logs on the regions server, the last thrown exception is > > the following => > > > > Thu Jun 23 05:16:18 2011 GMT regionserver 10460-0@spock7001:0 [DEBUG] > (IPC > > Server handler 7 on 60020) > > { org.apache.hadoop.hbase.NotServingRegionException: > > hbaseTable,,1308805558566.5aefc6c2b9599f55f8b40351a61db03c. is closing > > Thu Jun 23 05:22:18 2011 GMT regionserver 10460-0@spock7001:0 [DEBUG] > > (regionserver60020.logRoller) org.apache.hadoop.conf.Configuration: > > java.io.IOException: config() > > > > On running status 'detailed' in the shell , I get => > > > > 0 regionsInTransition > > 3 live servers > > spock7001:60020 1308805454136 > > requests=0, regions=0, usedHeap=470, maxHeap=910 > > spock6002:60020 1308805434201 > > requests=0, regions=1, usedHeap=550, maxHeap=910 > > hbaseTable,,1308805558566.5aefc6c2b9599f55f8b40351a61db03c. > > stores=1, storefiles=2, storefileSizeMB=383, memstoreSizeMB=0, > > storefileIndexSizeMB=1 > > spock6001:60020 1308805268507 > > requests=0, regions=2, usedHeap=90, maxHeap=910 > > -ROOT-,,0 > > stores=1, storefiles=1, storefileSizeMB=0, memstoreSizeMB=0, > > storefileIndexSizeMB=0 > > .META.,,1 > > stores=1, storefiles=0, storefileSizeMB=0, memstoreSizeMB=0, > > storefileIndexSizeMB=0 > > 0 dead servers > > > > > > I am issuing a checkAndPut() to insert records into HBase. Is this a bug > ? > > > > Secondly, I have followed the instructions in the HBase book to increase > > write throughput. I have the following settings for my hbase table: > > > > config = HBaseConfiguration.create(); > > table = new HTable (config, "hbaseTable"); > > table.setAutoFlush(false); > > table.setWriteBufferSize(104857600); > > > > However, according to my logs, each checkAndPut() call takes on an > average > > of 5 milliseconds. Is this unavoidable overhead due to locking ? > > > > All of my HBase daemons are running with -Xmx1g of heapsize. > > > > Any help is appreciated. > > > > Thank you, > > > > Sam > > >
