If have only one copy of the block and it is mostly corrupted.. Namenode
itself can not correct it. Of course, DFSClient should not print error
in a infinite loop.
I think there was an old bug where crc file got overwritten by 0 length
file.
One work around for you is to go to the datanode and remove the .crc
file for this block (find /datanodedir -name blk_5994030096182059653\*).
Be careful not to remove the block file itself.
longer term fix : upgrade to more recent version.
Raghu.
murali krishna wrote:
Hi,
When I try to read one of the file from dfs, I get the following error in an
infinite loop (using 0.15.3)
“08/10/28 23:43:15 INFO fs.DFSClient: Could not obtain block
blk_5994030096182059653 from any node: java.io.IOException: No live nodes
contain current block”
Fsck showed that the file is HEALTHY but under replicated (1 instead of
configured 2). I checked the datanode log where the only replica exists for
that block and I can see repeated errors while serving that bock.
2008-10-22 23:55:39,378 WARN org.apache.hadoop.dfs.DataNode: Failed to transfer
blk_59940300961820596
53 to 68.142.212.228:50010 got java.io.EOFException
at java.io.DataInputStream.readShort(DataInputStream.java:298)
at org.apache.hadoop.dfs.DataNode$BlockSender.<init>(DataNode.java:1061)
at org.apache.hadoop.dfs.DataNode$DataTransfer.run(DataNode.java:1446)
at java.lang.Thread.run(Thread.java:619)
Any idea what is going on and how can I fix this ?
Thanks,
Murali
