Please upgrade to 0.20.x hbase and hadoop if you can. All is much better there.
You have only 100 rows so you have only one region? Is that right? After opening your scanner, how much time passes between calls to next? If your calculation is taking long than the lease time, you'll see this timeout. St.Ack On Wed, Dec 9, 2009 at 9:39 AM, Xine Jar <[email protected]> wrote: > *Hallo, can you please help me, I am getting the exception > ScannerTimeoutException. > > **I-My Network:* > I have 6 machines running hadoop 0.19.1 and hbase 0.19.3 > > *II-My Program:* > I have an hbase table containing one column family and four > columns:xcoordinate, ycoordinate, nodeid, temperature. > The hbase table has 100 rows each row contains the x and y coordinates of a > sensor node, the node ID and a temperature value. > > *Map function:* > 1-scans the htable and searches all pair of nodes that are d1 kilometers > away from each others. > 2-For each pair it calculates the difference of their temperatures. > 3-It adds the result of each 4 differences together, and sends to the > reducer the key and the value in a Text format: > key="d1" > value= "result counter" ; The counter indicates the number of > differences that are summed to gether <=4 > > *Reduce function:* > 1- Sums the results of the same key all together > 2- Sums the counters of the same key all together > 3-divides a division: results/counters > > *III-Problem* > For a small number of nodes in the hbase and with a small number of > distances, the program works fine. Everytime I increase the total number of > nodes in the hbase > and the number of distances, i get a ScannerTimeOutException: > > *Error message*: > > *9/12/09 19:41:48 INFO mapred.JobClient: Task Id : > attempt_200912091918_0004_m_000000_0, Status : FAILED* > *org.apache.hadoop.hbase.client.ScannerTimeoutException* > * at > org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:1669)* > * at > > org.apache.hadoop.hbase.mapred.TableInputFormatBase$TableRecordReader.next(TableInputFormatBase.java:212) > * > * at > > org.apache.hadoop.hbase.mapred.TableInputFormatBase$TableRecordReader.next(TableInputFormatBase.java:87) > * > * at > > org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.java:192) > * > * at > > org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:176)* > * at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)* > * at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)* > * at org.apache.hadoop.mapred.Child.main(Child.java:158)* > *Caused by: org.apache.hadoop.hbase.UnknownScannerException: > org.apache.hadoop.hbase.UnknownScannerException: Name: 1917118727592717270* > * at > > org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1582) > * > * at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)* > * at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > * > * at java.lang.reflect.Method.invoke(Method.java:597)* > * at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:632)* > * at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:912)* > > * at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method)* > * at > > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) > * > * at > > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) > * > * at > java.lang.reflect.Constructor.newInstance(Constructor.java:513)* > * at > > org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:94) > * > * at > > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:852) > * > * at > org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:1665)* > > *NameNode:* After statring up it generates the repetitive sentence > > *2009-12-09 20:15:49,009 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* > NameSystem.processReport: block blk_-8866672104836793422_1226 on > 134.130.223.151:50010 size 134119 does not belong to any file.* > *2009-12-09 20:15:49,010 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* > NameSystem.addToInvalidates: blk_-8866672104836793422 is added to > invalidSet > of 134.130.223.151:50010* > > > *Tasktracker:* After starting up it generates the repetitive sentence > *org.apache.hadoop.util.DiskCheckerDiskErrorException: Could not find > taskTracker/jobcachejob_....../attempt.../output/file.out in any of the > configured directories* > > > *IV-Heeeeeeeeeelp* > *I have checked this problem on the web and I have seen that people tried > to > set the "dfs.datanode.socket.timeout" to 0. I have done the same and > restarted hadoop and hbase, but the problem was still the same. Could you > please tell me where this error is coming from? and how is it possible to > fix it?* > > *Many Thanks, > CJ* >
