I have tried setting hbase.client.scanner.caching to 10, 100, 1000 and 10000. But it still fails.
On Wed, Jan 11, 2017 at 2:17 PM, Yu Li <car...@gmail.com> wrote: > Hi Rajeshkumar, > > Simply speaking, in your case I'd suggest to set > hbase.client.scanner.caching > to a small value like 10 but not zero, and leave the > hbase.client.scanner.caching or hbase.rpc.timeout setting as default. Below > are some detailed explanation: > > Setting hbase.client.scanner.caching to zero will cause the scan returns no > data (please check whether you are getting a wrong result with this > setting), so please don't make such settings unless you intend to renew the > scan lease. See RsRpcServices#scan source code for more details if > interested. > > For version 1.1.1 (actually after HBASE-16973), the default value of > hbase.client.scanner.caching > is Integer.MAX_VALUE, and it won't return (for a single scanner.next call) > until the result size reached hbase.client.scanner.max.result.size or > times > out. If you are using a sparse filter, then it's strongly suggested to > set hbase.client.scanner.caching > to some small value but not zero. See HBASE-16973 for more details if > interested. > > Regarding hbase.rpc.timeout and hbase.client.scanner.timeout.period: > * For scan it's hbase.client.scanner.timeout.period taking effect > * For all other kinds of non-admin operations like > get/put/append/increment/multi hbase.rpc.timeout takes effect, unless you > specified hbase.client.operation.timeout. > Check ClientScanner#scannerTimeout, ScannerCallable#call, > AbstractRpcClient#channelOperationTimeout and > AbstractRpcClient#callBlockingMethod for more details if interested. > > Best Regards, > Yu > > On 11 January 2017 at 15:29, Rajeshkumar J <rajeshkumarit8...@gmail.com> > wrote: > > > Hi, As i got lease expire exception i have increased the values of these > > properties hbase.client.scanner.timeout.period and hbase.rpc.timeout. Is > > there any disadvantage increasing these two values? > > > > On Mon, Jan 2, 2017 at 11:01 AM, Rajeshkumar J < > > rajeshkumarit8...@gmail.com> > > wrote: > > > > > I am using hbase 1.1.1. > > > > > > I have following property value in hbase-site.xml and I didn't set > > value > > > hbase.client.scanner.caching property(so it must pick default value for > > > this) > > > > > > <property> > > > <name>hbase.client.scanner.timeout.period</name> > > > <value>60000</value> > > > </property> > > > > > > <property> > > > <name>hbase.rpc.timeout</name> > > > <value>60000</value> > > > </property> > > > When scanning 30 tables in hbase(each table holds 45 million records) I > > > got the following error > > > > > > WARN client.ScannerCallable [main]: Ignore, probably already closed > > > org.apache.hadoop.hbase.regionserver.LeaseException: > > > org.apache.hadoop.hbase.regionserver.LeaseException: lease does not > > exist > > > > > > But when I added hbase.client.scanner.caching property value as zero I > > > didn't get any error. Why this happened? > > > Even though I gave zero it supposed to pick the default value? How can > > > this process succeeds? > > > > > > Thanks > > > > > > On Mon, Jan 2, 2017 at 10:44 AM, Josh Elser <els...@apache.org> wrote: > > > > > >> I am having an extremely difficult time parsing this one, Rajeshkumar. > > >> > > >> If you still have a question/something you don't understand, I'd > > >> appreciate it if you could try to re-state it a little more clearly. > > >> > > >> > > >> Rajeshkumar J wrote: > > >> > > >>> I am using hbase 1.1.1. Yes while setting zero it must pick the > default > > >>> value that is INTEGER.MAX_VALUE in hbase 1.1.1. Already we have set > the > > >>> value as INTEGER.MAX_VALUE only but earlier it had failed but now > > >>> succeeded. So only I am asking this. Also I didn't change any > property > > >>> whatever I used in process which failed in the process which > succeeded > > >>> except assigning Zero value for this property > > >>> > > >>> On Thu, Dec 29, 2016 at 1:33 AM, Josh Elser<els...@apache.org> > wrote: > > >>> > > >>> Most likely, since you gave a nonsensical value, HBase used a default > > >>>> value instead of the one you provided. Since you have not shared the > > >>>> version of HBase which you are using, I would recommend that you > look > > at > > >>>> the code. It should be very obvious what value is being used instead > > of > > >>>> the > > >>>> bogus value you provided. > > >>>> > > >>>> > > >>>> Rajeshkumar J wrote: > > >>>> > > >>>> I have tried setting hbase.client.scanner.caching property to 0 with > > >>>>> above > > >>>>> two values as 60000. It runs fine. But hbase.client.scanner.caching > > >>>>> property tells no of rows to be fetched for every scanner but here > I > > >>>>> have > > >>>>> given zero. How this worked? > > >>>>> > > >>>>> On Tue, Dec 27, 2016 at 8:05 PM, Josh Elser<els...@apache.org> > > >>>>> wrote: > > >>>>> > > >>>>> hbase.client.scanner.timeout.period is a timeout specifically for > > RPCs > > >>>>> > > >>>>>> that come from the HBase Scanner classes (e.g. ClientScanner) > while > > >>>>>> hbase.rpc.timeout is the default timeout for any RPC. I believe > that > > >>>>>> the > > >>>>>> hbase.client.scanner.timeout.period is also used by the > > >>>>>> RegionServers to > > >>>>>> define the lifetime of the Lease (the cause of the LeaseException > > >>>>>> you're > > >>>>>> seeing). > > >>>>>> > > >>>>>> Generally, when you see these kinds of exceptions while scanning > > data > > >>>>>> in > > >>>>>> HBase, it is just a factor of your hardware and current > performance > > >>>>>> (in > > >>>>>> other words, how long it takes to read your data). I can't really > > >>>>>> give a > > >>>>>> firm answer because it is dependent on your system's performance. > > You > > >>>>>> should be able to approximate the performance with some > > >>>>>> back-of-the-envelope math. > > >>>>>> > > >>>>>> Some changes like https://issues.apache.org/ > jira/browse/HBASE-13090 > > >>>>>> and > > >>>>>> https://issues.apache.org/jira/browse/HBASE-13333 should help > > reduce > > >>>>>> the > > >>>>>> need for you to tweak configuration properties in the future. > > >>>>>> > > >>>>>> > > >>>>>> Rajeshkumar J wrote: > > >>>>>> > > >>>>>> Hi, > > >>>>>> > > >>>>>>> I have following property value as below > > >>>>>>> > > >>>>>>> <property> > > >>>>>>> <name>hbase.client.scanner.timeout.period</name> > > >>>>>>> <value>60000</value> > > >>>>>>> </property> > > >>>>>>> > > >>>>>>> <property> > > >>>>>>> <name>hbase.rpc.timeout</name> > > >>>>>>> <value>60000</value> > > >>>>>>> </property> > > >>>>>>> When scanning 30 tables in hbase(each table holds 45 million > > >>>>>>> records) I > > >>>>>>> got > > >>>>>>> > > >>>>>>> WARN client.ScannerCallable [main]: Ignore, probably already > closed > > >>>>>>> org.apache.hadoop.hbase.regionserver.LeaseException: > > >>>>>>> org.apache.hadoop.hbase.regionserver.LeaseException: lease does > > not > > >>>>>>> exist > > >>>>>>> > > >>>>>>> But when I change the value to > > >>>>>>> > > >>>>>>> <property> > > >>>>>>> <name>hbase.client.scanner.timeout.period</name> > > >>>>>>> <value>70000</value> > > >>>>>>> </property> > > >>>>>>> > > >>>>>>> <property> > > >>>>>>> <name>hbase.rpc.timeout</name> > > >>>>>>> <value>70000</value> > > >>>>>>> </property> > > >>>>>>> It runs successfully. > > >>>>>>> Can any one tell me the reason for this failure and also is there > > any > > >>>>>>> factor to calculate these property values for any data. > > >>>>>>> > > >>>>>>> > > >>>>>>> > > >>>>>>> > > >>> > > > > > >
