Hey YY,
At a more basic level -- have you run fsck on that file? What were
the results?
Brian
On Jan 27, 2009, at 10:54 AM, Bill Au wrote:
Did you start your namenode with the -upgrade after upgrading from
0.18.1 to
0.19.0?
Bill
On Mon, Jan 26, 2009 at 8:18 PM, Yuanyuan Tian <[email protected]>
wrote:
Hi,
I just upgraded hadoop from 0.18.1 to 0.19.0 following the
instructions on
http://wiki.apache.org/hadoop/Hadoop_Upgrade. After upgrade, I run
fsck,
everything seems fine. All the files can be listed in hdfs and the
sizes
are also correct. But when a mapreduce job tries to read the files as
input, the following error messages are returned for some of the
files:
java.io.IOException: Could not obtain block:
blk_-2827537120880440835_1131
file=/user/hmail/NSF/50k_nntp_clean2.nsf.fs.kvp
at org.apache.hadoop.hdfs.DFSClient
$DFSInputStream.chooseDataNode(DFSClient.java:1708)
at org.apache.hadoop.hdfs.DFSClient
$DFSInputStream.blockSeekTo
(DFSClient.java:1536)
at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read
(DFSClient.java:1663)
at java.io.DataInputStream.read(DataInputStream.java:150)
at java.io.ObjectInputStream$PeekInputStream.read
(ObjectInputStream.java:2283)
at java.io.ObjectInputStream$PeekInputStream.readFully
(ObjectInputStream.java:2296)
at java.io.ObjectInputStream
$BlockDataInputStream.readShort
(ObjectInputStream.java:2767)
at java.io.ObjectInputStream.readStreamHeader
(ObjectInputStream.java:798)
at java.io.ObjectInputStream.(ObjectInputStream.java:298)
at
emailanalytics.importer.parallelimport.EmailContentRecordReader.
(EmailContentRecordReader.java:32)
at
emailanalytics
.importer.parallelimport.EmailContentFormat.getRecordReader
(EmailContentFormat.java:20)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:321)
at org.apache.hadoop.mapred.Child.main(Child.java:155)
I also tried to browse these files through the HDFS web interface,
java.io.EOFException is returned.
Is there any way to recover the files?
Thanks very much,
YY
