Great! thank you this was really helpful I appreciate it. regards, CJ
On Wed, Dec 9, 2009 at 7:00 PM, Jean-Daniel Cryans <[email protected]>wrote: > In 0.19.3 (which is very old btw). the default > hbase.regionserver.lease.period is 60 seconds so you have to make sure > you report back to the region server by doing a next() on the scanner > before it expires else you get a ScannerTimeoutException. > > Also if you use scanner pre-fetching (by default you don't) then you > have to be even more careful. > > If your use case requires you to spend more than one minute on a row > after a next() invocation, then you should change the default value of > hbase.regionserver.lease.period to something much higher. > > J-D > > On Wed, Dec 9, 2009 at 9:39 AM, Xine Jar <[email protected]> wrote: > > *Hallo, can you please help me, I am getting the exception > > ScannerTimeoutException. > > > > **I-My Network:* > > I have 6 machines running hadoop 0.19.1 and hbase 0.19.3 > > > > *II-My Program:* > > I have an hbase table containing one column family and four > > columns:xcoordinate, ycoordinate, nodeid, temperature. > > The hbase table has 100 rows each row contains the x and y coordinates of > a > > sensor node, the node ID and a temperature value. > > > > *Map function:* > > 1-scans the htable and searches all pair of nodes that are d1 kilometers > > away from each others. > > 2-For each pair it calculates the difference of their temperatures. > > 3-It adds the result of each 4 differences together, and sends to the > > reducer the key and the value in a Text format: > > key="d1" > > value= "result counter" ; The counter indicates the number of > > differences that are summed to gether <=4 > > > > *Reduce function:* > > 1- Sums the results of the same key all together > > 2- Sums the counters of the same key all together > > 3-divides a division: results/counters > > > > *III-Problem* > > For a small number of nodes in the hbase and with a small number of > > distances, the program works fine. Everytime I increase the total number > of > > nodes in the hbase > > and the number of distances, i get a ScannerTimeOutException: > > > > *Error message*: > > > > *9/12/09 19:41:48 INFO mapred.JobClient: Task Id : > > attempt_200912091918_0004_m_000000_0, Status : FAILED* > > *org.apache.hadoop.hbase.client.ScannerTimeoutException* > > * at > > > org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:1669)* > > * at > > > org.apache.hadoop.hbase.mapred.TableInputFormatBase$TableRecordReader.next(TableInputFormatBase.java:212) > > * > > * at > > > org.apache.hadoop.hbase.mapred.TableInputFormatBase$TableRecordReader.next(TableInputFormatBase.java:87) > > * > > * at > > > org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.java:192) > > * > > * at > > > org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:176)* > > * at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)* > > * at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)* > > * at org.apache.hadoop.mapred.Child.main(Child.java:158)* > > *Caused by: org.apache.hadoop.hbase.UnknownScannerException: > > org.apache.hadoop.hbase.UnknownScannerException: Name: > 1917118727592717270* > > * at > > > org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1582) > > * > > * at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)* > > * at > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > * > > * at java.lang.reflect.Method.invoke(Method.java:597)* > > * at > > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:632)* > > * at > > > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:912)* > > > > * at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > > Method)* > > * at > > > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) > > * > > * at > > > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) > > * > > * at > java.lang.reflect.Constructor.newInstance(Constructor.java:513)* > > * at > > > org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:94) > > * > > * at > > > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:852) > > * > > * at > > > org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:1665)* > > > > *NameNode:* After statring up it generates the repetitive sentence > > > > *2009-12-09 20:15:49,009 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* > > NameSystem.processReport: block blk_-8866672104836793422_1226 on > > 134.130.223.151:50010 size 134119 does not belong to any file.* > > *2009-12-09 20:15:49,010 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* > > NameSystem.addToInvalidates: blk_-8866672104836793422 is added to > invalidSet > > of 134.130.223.151:50010* > > > > > > *Tasktracker:* After starting up it generates the repetitive sentence > > *org.apache.hadoop.util.DiskCheckerDiskErrorException: Could not find > > taskTracker/jobcachejob_....../attempt.../output/file.out in any of the > > configured directories* > > > > > > *IV-Heeeeeeeeeelp* > > *I have checked this problem on the web and I have seen that people tried > to > > set the "dfs.datanode.socket.timeout" to 0. I have done the same and > > restarted hadoop and hbase, but the problem was still the same. Could you > > please tell me where this error is coming from? and how is it possible to > > fix it?* > > > > *Many Thanks, > > CJ* > > >
