this is the log I am getting while running 2017-01-05 11:41:49,629 DEBUG [B.defaultRpcServer.handler=15,queue=0,port=16020] ipc.RpcServer: B.defaultRpcServer.handler=15,queue=0,port=16020: callId: 3 service: ClientService methodName: Scan size: 23 connection: hostname:port org.apache.hadoop.hbase.regionserver.LeaseException: lease '706' does not exist at org.apache.hadoop.hbase.regionserver.Leases.removeLease(Leases.java:221) at org.apache.hadoop.hbase.regionserver.Leases.cancelLease(Leases.java:206) at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2491) at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32205) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2114) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:101) at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130) at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107) at java.lang.Thread.run(Thread.java:744) 2017-01-05 11:41:49,629 TRACE [B.defaultRpcServer.handler=18,queue=0,port=16020] ipc.RpcServer: callId: 2 service: ClientService methodName: Scan size: 29 connection: hostname:port param: scanner_id: 706 number_of_rows: 2147483647 close_scanner: false next_call_seq: 0 client_handles_partials: true client_handles_heartbeats: true connection: hostname:port, response scanner_id: 706 more_results: true stale: false more_results_in_region: false queueTime: 1 processingTime: 60136 totalTime: 60137
On Wed, Jan 11, 2017 at 4:45 PM, Rajeshkumar J <rajeshkumarit8...@gmail.com> wrote: > I have tried setting hbase.client.scanner.caching to 10, 100, 1000 and > 10000. But it still fails. > > On Wed, Jan 11, 2017 at 2:17 PM, Yu Li <car...@gmail.com> wrote: > >> Hi Rajeshkumar, >> >> Simply speaking, in your case I'd suggest to set >> hbase.client.scanner.caching >> to a small value like 10 but not zero, and leave the >> hbase.client.scanner.caching or hbase.rpc.timeout setting as default. >> Below >> are some detailed explanation: >> >> Setting hbase.client.scanner.caching to zero will cause the scan returns >> no >> data (please check whether you are getting a wrong result with this >> setting), so please don't make such settings unless you intend to renew >> the >> scan lease. See RsRpcServices#scan source code for more details if >> interested. >> >> For version 1.1.1 (actually after HBASE-16973), the default value of >> hbase.client.scanner.caching >> is Integer.MAX_VALUE, and it won't return (for a single scanner.next call) >> until the result size reached hbase.client.scanner.max.result.size or >> times >> out. If you are using a sparse filter, then it's strongly suggested to >> set hbase.client.scanner.caching >> to some small value but not zero. See HBASE-16973 for more details if >> interested. >> >> Regarding hbase.rpc.timeout and hbase.client.scanner.timeout.period: >> * For scan it's hbase.client.scanner.timeout.period taking effect >> * For all other kinds of non-admin operations like >> get/put/append/increment/multi hbase.rpc.timeout takes effect, unless you >> specified hbase.client.operation.timeout. >> Check ClientScanner#scannerTimeout, ScannerCallable#call, >> AbstractRpcClient#channelOperationTimeout and >> AbstractRpcClient#callBlockingMethod for more details if interested. >> >> Best Regards, >> Yu >> >> On 11 January 2017 at 15:29, Rajeshkumar J <rajeshkumarit8...@gmail.com> >> wrote: >> >> > Hi, As i got lease expire exception i have increased the values of >> these >> > properties hbase.client.scanner.timeout.period and hbase.rpc.timeout. >> Is >> > there any disadvantage increasing these two values? >> > >> > On Mon, Jan 2, 2017 at 11:01 AM, Rajeshkumar J < >> > rajeshkumarit8...@gmail.com> >> > wrote: >> > >> > > I am using hbase 1.1.1. >> > > >> > > I have following property value in hbase-site.xml and I didn't set >> > value >> > > hbase.client.scanner.caching property(so it must pick default value >> for >> > > this) >> > > >> > > <property> >> > > <name>hbase.client.scanner.timeout.period</name> >> > > <value>60000</value> >> > > </property> >> > > >> > > <property> >> > > <name>hbase.rpc.timeout</name> >> > > <value>60000</value> >> > > </property> >> > > When scanning 30 tables in hbase(each table holds 45 million records) >> I >> > > got the following error >> > > >> > > WARN client.ScannerCallable [main]: Ignore, probably already closed >> > > org.apache.hadoop.hbase.regionserver.LeaseException: >> > > org.apache.hadoop.hbase.regionserver.LeaseException: lease does not >> > exist >> > > >> > > But when I added hbase.client.scanner.caching property value as zero I >> > > didn't get any error. Why this happened? >> > > Even though I gave zero it supposed to pick the default value? How can >> > > this process succeeds? >> > > >> > > Thanks >> > > >> > > On Mon, Jan 2, 2017 at 10:44 AM, Josh Elser <els...@apache.org> >> wrote: >> > > >> > >> I am having an extremely difficult time parsing this one, >> Rajeshkumar. >> > >> >> > >> If you still have a question/something you don't understand, I'd >> > >> appreciate it if you could try to re-state it a little more clearly. >> > >> >> > >> >> > >> Rajeshkumar J wrote: >> > >> >> > >>> I am using hbase 1.1.1. Yes while setting zero it must pick the >> default >> > >>> value that is INTEGER.MAX_VALUE in hbase 1.1.1. Already we have set >> the >> > >>> value as INTEGER.MAX_VALUE only but earlier it had failed but now >> > >>> succeeded. So only I am asking this. Also I didn't change any >> property >> > >>> whatever I used in process which failed in the process which >> succeeded >> > >>> except assigning Zero value for this property >> > >>> >> > >>> On Thu, Dec 29, 2016 at 1:33 AM, Josh Elser<els...@apache.org> >> wrote: >> > >>> >> > >>> Most likely, since you gave a nonsensical value, HBase used a >> default >> > >>>> value instead of the one you provided. Since you have not shared >> the >> > >>>> version of HBase which you are using, I would recommend that you >> look >> > at >> > >>>> the code. It should be very obvious what value is being used >> instead >> > of >> > >>>> the >> > >>>> bogus value you provided. >> > >>>> >> > >>>> >> > >>>> Rajeshkumar J wrote: >> > >>>> >> > >>>> I have tried setting hbase.client.scanner.caching property to 0 >> with >> > >>>>> above >> > >>>>> two values as 60000. It runs fine. But >> hbase.client.scanner.caching >> > >>>>> property tells no of rows to be fetched for every scanner but >> here I >> > >>>>> have >> > >>>>> given zero. How this worked? >> > >>>>> >> > >>>>> On Tue, Dec 27, 2016 at 8:05 PM, Josh Elser<els...@apache.org> >> > >>>>> wrote: >> > >>>>> >> > >>>>> hbase.client.scanner.timeout.period is a timeout specifically for >> > RPCs >> > >>>>> >> > >>>>>> that come from the HBase Scanner classes (e.g. ClientScanner) >> while >> > >>>>>> hbase.rpc.timeout is the default timeout for any RPC. I believe >> that >> > >>>>>> the >> > >>>>>> hbase.client.scanner.timeout.period is also used by the >> > >>>>>> RegionServers to >> > >>>>>> define the lifetime of the Lease (the cause of the LeaseException >> > >>>>>> you're >> > >>>>>> seeing). >> > >>>>>> >> > >>>>>> Generally, when you see these kinds of exceptions while scanning >> > data >> > >>>>>> in >> > >>>>>> HBase, it is just a factor of your hardware and current >> performance >> > >>>>>> (in >> > >>>>>> other words, how long it takes to read your data). I can't really >> > >>>>>> give a >> > >>>>>> firm answer because it is dependent on your system's performance. >> > You >> > >>>>>> should be able to approximate the performance with some >> > >>>>>> back-of-the-envelope math. >> > >>>>>> >> > >>>>>> Some changes like https://issues.apache.org/jira >> /browse/HBASE-13090 >> > >>>>>> and >> > >>>>>> https://issues.apache.org/jira/browse/HBASE-13333 should help >> > reduce >> > >>>>>> the >> > >>>>>> need for you to tweak configuration properties in the future. >> > >>>>>> >> > >>>>>> >> > >>>>>> Rajeshkumar J wrote: >> > >>>>>> >> > >>>>>> Hi, >> > >>>>>> >> > >>>>>>> I have following property value as below >> > >>>>>>> >> > >>>>>>> <property> >> > >>>>>>> <name>hbase.client.scanner.timeout.period</name> >> > >>>>>>> <value>60000</value> >> > >>>>>>> </property> >> > >>>>>>> >> > >>>>>>> <property> >> > >>>>>>> <name>hbase.rpc.timeout</name> >> > >>>>>>> <value>60000</value> >> > >>>>>>> </property> >> > >>>>>>> When scanning 30 tables in hbase(each table holds 45 million >> > >>>>>>> records) I >> > >>>>>>> got >> > >>>>>>> >> > >>>>>>> WARN client.ScannerCallable [main]: Ignore, probably already >> closed >> > >>>>>>> org.apache.hadoop.hbase.regionserver.LeaseException: >> > >>>>>>> org.apache.hadoop.hbase.regionserver.LeaseException: lease does >> > not >> > >>>>>>> exist >> > >>>>>>> >> > >>>>>>> But when I change the value to >> > >>>>>>> >> > >>>>>>> <property> >> > >>>>>>> <name>hbase.client.scanner.timeout.period</name> >> > >>>>>>> <value>70000</value> >> > >>>>>>> </property> >> > >>>>>>> >> > >>>>>>> <property> >> > >>>>>>> <name>hbase.rpc.timeout</name> >> > >>>>>>> <value>70000</value> >> > >>>>>>> </property> >> > >>>>>>> It runs successfully. >> > >>>>>>> Can any one tell me the reason for this failure and also is >> there >> > any >> > >>>>>>> factor to calculate these property values for any data. >> > >>>>>>> >> > >>>>>>> >> > >>>>>>> >> > >>>>>>> >> > >>> >> > > >> > >> > >
