Did you try to use REQUEST_NUMBER as throttle type? 2016-09-22 11:33 GMT+08:00 Sumit Nigam <sumit_o...@yahoo.com.invalid>:
> Hi Guanghao, > > This throttling error shows up as soon as I start hbase. So, ideally there > shouldn't be too many prior operations in play here. Plus, the error shows > up even when my table has hardly any data (possibly in kbs) and I have set > throttling limit to be ~10GB. > > Thanks,Sumit > > From: Guanghao Zhang <zghao...@gmail.com> > To: user@hbase.apache.org; Sumit Nigam <sumit_o...@yahoo.com> > Sent: Thursday, September 22, 2016 7:37 AM > Subject: Re: Hbase throttling issues > > All scan operations were throttled? Now it use the avg size of all previous > operations to check quota. Maybe the previous scan operation read too much > data. > > 2016-09-22 1:18 GMT+08:00 Sumit Nigam <sumit_o...@yahoo.com.invalid>: > > > Hi Hbase experts, > > > > Our application is unable to scan or read from hbase tables, when > > throttling is set. We are getting ThrottlingException every time. The > error > > is seen more frequently when the number of hbase pre splits is > > increased. The size tables for which this error is showing is empty ( > > during some runs it was very low in the order of few kbs). Have tried > both > > rate limiters already - average and fixed. Can't understand why read rate > > limit exceeds when there is hardly any data in hbase. Anyone has faced > this > > issue before? > > > > Setup Details: > > > > Hbase version : 1.1.2 > > Number of region servers :4 > > Number of regions : 116 > > HeapMemory for Region Server : 2GB > > > > Quotas set : > > TABLE => ns1:table1 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE, > LIMIT > > => 10G/sec, SCOPE => MACHINE > > TABLE => ns2:table2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE, > LIMIT > > => 10G/sec, SCOPE => MACHINE > > > > Following is the error we faced. Pasting debug logs of regions server: > > > > 2016-09-17 22:35:40,674 DEBUG [B.defaultRpcServer.handler= > 55,queue=1,port=58526] > > quotas.RegionServerQuotaManager: Throttling exception for user=root > > table=ns1:table1 numWrites=0 numReads=0 numScans=1: read size limit > > exceeded - wait 0.00sec > > > > 2016-09-17 22:35:40,676 DEBUG [B.defaultRpcServer.handler= > 55,queue=1,port=58526] > > ipc.RpcServer: B.defaultRpcServer.handler=55,queue=1,port=58526: callId: > > 52 service: ClientService methodName: Scan size: 28 connection: > > 10.65.141.170:42806 > > > > org.apache.hadoop.hbase.quotas.ThrottlingException: read size limit > > exceeded - wait 0.00sec > > > > at org.apache.hadoop.hbase.quotas.ThrottlingException. > > throwThrottlingException(ThrottlingException.java:107) > > > > at org.apache.hadoop.hbase.quotas.ThrottlingException. > > throwReadSizeExceeded(ThrottlingException.java:101) > > > > at org.apache.hadoop.hbase.quotas.TimeBasedLimiter. > > checkQuota(TimeBasedLimiter.java:139) > > > > at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota( > > DefaultOperationQuota.java:59) > > > > at org.apache.hadoop.hbase.quotas.RegionServerQuotaManager.checkQuota( > > RegionServerQuotaManager.java:180) > > > > at org.apache.hadoop.hbase.quotas.RegionServerQuotaManager.checkQuota( > > RegionServerQuotaManager.java:125) > > > > at org.apache.hadoop.hbase.regionserver.RSRpcServices. > > scan(RSRpcServices.java:2265) > > > > at org.apache.hadoop.hbase.protobuf.generated. > > ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32205) > > > > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2114) > > > > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:101) > > > > at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop( > > RpcExecutor.java:130) > > > > at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107) > > > > at java.lang.Thread.run(Thread.java:745) > > > > > > Thanks, > > > > Sumit > > > >
