Hm, weird. Looks like now it goes well. What I did is once again restarted several regionservers (noticed that some where holding regions which were stuck in "in transition" state). And started back the job.
Btw, forgot to mention that caching on Scanner was set to only 64 and I the estimate for records size is < 100KB. Anyhow, the weird thing was that the limits were set to 10min and it started to fail even before 10 min has passed (sometimes after 3-4 minutes). Have to go and can't watch the job to the end, will write whether it succeed or not. If face same problems again will try brutal variant: set caching = 10 (or even 1), set batch = 10 (or even 1). Alex On Sun, Feb 12, 2012 at 1:49 PM, Alex Baranau <[email protected]>wrote: > Hi, > > 0.90.4-cdh3u2 > > Alex > > > On Sun, Feb 12, 2012 at 1:44 PM, <[email protected]> wrote: > >> Which version of hbase are you using ? >> >> Thanks >> >> >> >> On Feb 12, 2012, at 10:41 AM, Alex Baranau <[email protected]> >> wrote: >> >> > Hello, >> > >> > I'm getting scanner lease exceptions during mapreduce job [1] after >> running >> > it for less than 7 minutes. Though I have set >> > hbase.regionserver.lease.period to 600000 (i.e. 10 min) in hbase >> > configuration on master and all regionservers and master (and restarted >> > all). Also set it in job's configuration. >> > >> > Is there anything else I could do to eliminate the exception? >> > >> > Thanks, >> > Alex >> > >> > [1] >> > >> > org.apache.hadoop.hbase.regionserver.LeaseException: >> > org.apache.hadoop.hbase.regionserver.LeaseException: lease >> > ʼ3854333525977315054ʼ does not exist >> > at >> > org.apache.hadoop.hbase.regionserver.Leases.removeLease(Leases.java:230) >> > >> > at >> > >> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1862) >> > >> > at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown >> > Source) >> > >> > at >> > >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >> > Last >> > at >> > java.lang.reflect.Method.invoke(Method.java:597) >> > 4KB >> > al FAILED at >> > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570) >> > Last >> > at >> > >> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039) >> > 8KB >> > >> > All >> > at >> > sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native >> > Method) >> > >> > at >> > >> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) >> > >> > at >> > >> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) >> > >> > at >> > java.lang.reflect.Constructor.newInstance(Constructor.java:513) >> > >> > at >> > org.ap >> > >
