What Andrew said and then, you might also look back in your logs to see if you can find where the region 743469791 first gets exceptions.
Grep 743469791 in your master to find the unencoded name and use this grepping the master to find which host it was last deployed to. Then go there and grep for the unencoded regionname. Might give a clue as to what your cluster is suffering Dru. St.Ack On Mon, Jan 12, 2009 at 12:28 PM, Dru Jensen <drujen...@gmail.com> wrote: > I have an MR process that populates a table but fails with this error: > > java.io.IOException: java.io.IOException: Cannot open filename > /hbase/webmaps/743469791/header/mapfiles/4735713346568547172/data > at > org.apache.hadoop.hdfs.DFSClient$DFSInputStream.openInfo(DFSClient.java:1395) > at > org.apache.hadoop.hdfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:1720) > at > org.apache.hadoop.hdfs.DFSClient$DFSInputStream.blockSeekTo(DFSClient.java:1536) > at > org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:1663) > at > org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:1593) > at java.io.DataInputStream.readInt(DataInputStream.java:370) > at > org.apache.hadoop.hbase.io.SequenceFile$Reader.readRecordLength(SequenceFile.java:1909) > at > org.apache.hadoop.hbase.io.SequenceFile$Reader.next(SequenceFile.java:1939) > at > org.apache.hadoop.hbase.io.SequenceFile$Reader.next(SequenceFile.java:1844) > at > org.apache.hadoop.hbase.io.MapFile$Reader.seekInternal(MapFile.java:472) > at > org.apache.hadoop.hbase.io.MapFile$Reader.getClosest(MapFile.java:567) > at > org.apache.hadoop.hbase.io.MapFile$Reader.getClosest(MapFile.java:550) > at > org.apache.hadoop.hbase.io.BloomFilterMapFile$Reader.getClosest(BloomFilterMapFile.java:112) > at org.apache.hadoop.hbase.regionserver.HStore.get(HStore.java:1485) > at > org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:975) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1504) > at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:632) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:894) > > The MR process needs to check to make sure the row is the latest before > inserting it so it calls htable.get(key) in the mr process. This is where > the process fails. > > I also tried performing a get 'table', 'key' in the shell with the same > error. I verified that the file is missing in hadoop. It looks like HBase > did not update its reference after a split/region assignment. Is anyone > else experiencing this? > > using 732094 from trunk. > > Thanks for your help, > Dru
