after server restart - getting exception - java.io.IOException: Timed out waiting for lock for row
need some help. this has happened for 2 of my servers - *[B.defaultRpcServer.handler=2,queue=2,port=16020] regionserver.HRegion: Failed getting lock in batch put, row=a\xF7\x1D\xCBdR\xBC\xEC_\x18D>\xA2\xD0\x95\xFF* *java.io.IOException: Timed out waiting for lock for row: a\xF7\x1D\xCBdR\xBC\xEC_\x18D>\xA2\xD0\x95\xFF* at org.apache.hadoop.hbase.regionserver.HRegion.getRowLockInternal(HRegion.java:5051) at org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:2944) at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2801) at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2743) at org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:692) at org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:654) at org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2031) at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32213) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2114) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:101) at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130) at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107) at java.lang.Thread.run(Thread.java:745) -- with regards, ch Vishnu mash213.wordpress.com doodle-vishnu.blogspot.in
hbase get big table problem
I got a cluster of 200 regionserver, and one of the tables is about 3T and 5 billion lines. Is it possible to get about 8000 Gets per second(about 100,000 lines)? I found YOUNG GC occurs every several senconds, each GC cost about 1second. if I set -Xmn bigger, the GC occurs every several minutes, but each GC cost more time. Any suggetion? thanks. = Thanks, lujinhong
RE: HBase number of columns
Thanks Saad! -Original Message- From: Saad Mufti [mailto:saad.mu...@gmail.com] Sent: Thursday, June 16, 2016 10:30 PM To: user@hbase.apache.org Subject: Re: HBase number of columns There is no real column schema in HBase other than defining the column family, each write to a column writes a cell with the column name plus value, so in theory number of columns doesn't really matter. What matters is how much data you read and write. That said there are settings in the column family schema for DATA_BLOCK_ENCODING that affect how much actual space each column/cell takes, FAST_DIFF is a decent choice to make sure there is not too much redundancy by writing the same column name over and over again if lots of rows have the same column name. There are also compression settings of course. Hope that helps. Saad On Wed, Jun 15, 2016 at 7:11 AM, Siddharth Ubale < siddharth.ub...@syncoms.com> wrote: > Hi, > > As per the official documentation of HBase it is mentioned that HBase > typical schema should contain 1 to 3 column families per table ( > https://hbase.apache.org/book.html#table_schema_rules_of_thumb ) . > However there is no mention of how many column qualifiers should a row > contain for each column family to see good read & write performance. > Could anybody let us know their input on how many columns per row is > desirable in HBase or how many column qualifiers per column family > would be desirable. > Thanks, > Siddharth Ubale, > >
