Hi,
I see the following in our log:
org.apache.hadoop.fs.ChecksumException: Checksum error:
file:/domain_crawler/crawlworker/crawl/777_pv_10_4_20100128_122735_353.done.tsk/segments/20100131163704/parse_data/part-00000/data
at 42990592
at
org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.readChunk(ChecksumFileSystem.java:211)
at
org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:238)
at org.apache.hadoop.fs.FSInputChecker.fill(FSInputChecker.java:177)
at
org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:194)
at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:159)
at java.io.DataInputStream.readFully(DataInputStream.java:178)
at
org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:64)
at
org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:102)
at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1933)
at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1833)
at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1879)
at
com.rialto.webmap.common.SeqFileWriter.writeSeqFile(SeqFileWriter.java:189)
HFWriter uses Hadoop version 0.18.3
Please share your experience solving this error.
Thanks
