Did you set any other quotas? The exception ThrottlingException.throwReadSizeExceeded shows that the throttle type is READ_SIZE, but your quota's throttle type is REQUEST_SIZE. You can use list_quota shell cmd to check whether there are READ_SIZE quota.
2016-09-22 12:06 GMT+08:00 Sumit Nigam <[email protected]>: > No, did not try request number. I want to use size as my throttling > factor. > > Thanks again! > > From: Guanghao Zhang <[email protected]> > To: [email protected]; Sumit Nigam <[email protected]> > Sent: Thursday, September 22, 2016 9:28 AM > Subject: Re: Hbase throttling issues > > Did you try to use REQUEST_NUMBER as throttle type? > > 2016-09-22 11:33 GMT+08:00 Sumit Nigam <[email protected]>: > > > Hi Guanghao, > > > > This throttling error shows up as soon as I start hbase. So, ideally > there > > shouldn't be too many prior operations in play here. Plus, the error > shows > > up even when my table has hardly any data (possibly in kbs) and I have > set > > throttling limit to be ~10GB. > > > > Thanks,Sumit > > > > From: Guanghao Zhang <[email protected]> > > To: [email protected]; Sumit Nigam <[email protected]> > > Sent: Thursday, September 22, 2016 7:37 AM > > Subject: Re: Hbase throttling issues > > > > All scan operations were throttled? Now it use the avg size of all > previous > > operations to check quota. Maybe the previous scan operation read too > much > > data. > > > > 2016-09-22 1:18 GMT+08:00 Sumit Nigam <[email protected]>: > > > > > Hi Hbase experts, > > > > > > Our application is unable to scan or read from hbase tables, when > > > throttling is set. We are getting ThrottlingException every time. The > > error > > > is seen more frequently when the number of hbase pre splits is > > > increased. The size tables for which this error is showing is empty ( > > > during some runs it was very low in the order of few kbs). Have tried > > both > > > rate limiters already - average and fixed. Can't understand why read > rate > > > limit exceeds when there is hardly any data in hbase. Anyone has faced > > this > > > issue before? > > > > > > Setup Details: > > > > > > Hbase version : 1.1.2 > > > Number of region servers :4 > > > Number of regions : 116 > > > HeapMemory for Region Server : 2GB > > > > > > Quotas set : > > > TABLE => ns1:table1 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE, > > LIMIT > > > => 10G/sec, SCOPE => MACHINE > > > TABLE => ns2:table2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE, > > LIMIT > > > => 10G/sec, SCOPE => MACHINE > > > > > > Following is the error we faced. Pasting debug logs of regions server: > > > > > > 2016-09-17 22:35:40,674 DEBUG [B.defaultRpcServer.handler= > > 55,queue=1,port=58526] > > > quotas.RegionServerQuotaManager: Throttling exception for user=root > > > table=ns1:table1 numWrites=0 numReads=0 numScans=1: read size limit > > > exceeded - wait 0.00sec > > > > > > 2016-09-17 22:35:40,676 DEBUG [B.defaultRpcServer.handler= > > 55,queue=1,port=58526] > > > ipc.RpcServer: B.defaultRpcServer.handler=55,queue=1,port=58526: > callId: > > > 52 service: ClientService methodName: Scan size: 28 connection: > > > 10.65.141.170:42806 > > > > > > org.apache.hadoop.hbase.quotas.ThrottlingException: read size limit > > > exceeded - wait 0.00sec > > > > > > at org.apache.hadoop.hbase.quotas.ThrottlingException. > > > throwThrottlingException(ThrottlingException.java:107) > > > > > > at org.apache.hadoop.hbase.quotas.ThrottlingException. > > > throwReadSizeExceeded(ThrottlingException.java:101) > > > > > > at org.apache.hadoop.hbase.quotas.TimeBasedLimiter. > > > checkQuota(TimeBasedLimiter.java:139) > > > > > > at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota( > > > DefaultOperationQuota.java:59) > > > > > > at org.apache.hadoop.hbase.quotas.RegionServerQuotaManager.checkQuota( > > > RegionServerQuotaManager.java:180) > > > > > > at org.apache.hadoop.hbase.quotas.RegionServerQuotaManager.checkQuota( > > > RegionServerQuotaManager.java:125) > > > > > > at org.apache.hadoop.hbase.regionserver.RSRpcServices. > > > scan(RSRpcServices.java:2265) > > > > > > at org.apache.hadoop.hbase.protobuf.generated. > > > ClientProtos$ClientService$2.callBlockingMethod( > ClientProtos.java:32205) > > > > > > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2114) > > > > > > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:101) > > > > > > at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop( > > > RpcExecutor.java:130) > > > > > > at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107) > > > > > > at java.lang.Thread.run(Thread.java:745) > > > > > > > > > Thanks, > > > > > > Sumit > > > > > > > > > > > >
