Hi, We currently use hadoop 0.20.1 I found the following trace in our log:
2010-05-17 16:19:55,389 INFO [FSInputChecker] Found checksum error: b[0, 512]=53455106246f72672e6170616368652e6861646f6f702e696f2e426f6f6c65616e5772697461626c652b636f6d2e6361727269657269712e6d326d2e636f6d6d6f6e732e4f746155706c6f61645772697461626c6501002a6f72672e6170616368652e6861646f6f702e696f2e636f6d70726573732e44656661756c74436f64656300000000aa555ae318451e8e76c0230f56f74d1f000014660000000101789cdddb09785355da00e0ef26699bae49ba2f94de2ed436857ab3364194b414294a355296e29aa449079025968ae82f78cb22051c088202e2125040d9acb88bc36410441d9dbf50dc1e06c57143749c8e0e0c7bff73ee92a6e9a7f23f33cef33f7f1f0839eff9ee3df77ef7dcedeb03130facc763b437197d76abd96237983c6e8399731b3d564395d9e2b6797c6e60b9268fdb5d55d56c31180c6693c56bb21bc9327683d5e036dadd162fb056afc76ae17cd62a8bd1e7717b0d3e6f95d5e2357b4c55be260fd76406d6e76b32992c9e2aafdb62b659bccd9cdb63b134f92c4d4d1e5b95d5e401d66430d8dc06b7d76e68f61a9acd469bdb6a37703ea3c9e33556594c1c19a5b9c964f671568bc96b34b83d55551eb2850693db6d767306b3d10a6c1367367bb9660f89b47a7d469fd96771db7d06aec9cbd9c9c69351ec5683917ce38c16bbcd62b17bcc4dcd96263347f6c4d2ecf536937d31588c16a3bdb9 org.apache.hadoop.fs.ChecksumException: Checksum error: file:/incoming_ub/ciq_dca_uploads/2010/03/26/100813/17_00.ub at 0 at org.apache.hadoop.fs.FSInputChecker.verifySum(FSInputChecker.java:277) at org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:241) at org.apache.hadoop.fs.FSInputChecker.fill(FSInputChecker.java:176) at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:193) at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:158) at java.io.DataInputStream.readFully(DataInputStream.java:178) at java.io.DataInputStream.readFully(DataInputStream.java:152) at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1450) at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1428) at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1417) at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1412) at org.apache.hadoop.mapred.SequenceFileRecordReader.<init>(SequenceFileRecordReader.java:43) at com.ciq.m2m.platform.util.hadoop.ReportingSequenceFileRecordReader.<init>(ReportingSequenceFileRecordReader.java:54) at sun.reflect.GeneratedConstructorAccessor39.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at com.ciq.m2m.platform.util.hadoop.MultiFileRecordReader.assureReader(MultiFileRecordReader.java:128) at com.ciq.m2m.platform.util.hadoop.MultiFileRecordReader.readNextFromAnyReader(MultiFileRecordReader.java:146) at com.ciq.m2m.platform.util.hadoop.MultiFileRecordReader.readNextFromAnyReader(MultiFileRecordReader.java:181) at com.ciq.m2m.platform.util.hadoop.MultiFileRecordReader.next(MultiFileRecordReader.java:122) at com.ciq.m2m.platform.util.hadoop.MultiFileRecordReader.next(MultiFileRecordReader.java:20) at com.ciq.m2m.platform.mmp2.input.BaseOtaUploadRecordReader.readNextUpload(BaseOtaUploadRecordReader.java:125) at com.ciq.m2m.platform.mmp2.input.OtaUploadToBinaryPackageRecordReader.next(OtaUploadToBinaryPackageRecordReader.java:176) at com.ciq.m2m.platform.mmp2.input.OtaUploadToBinaryPackageRecordReader.next(OtaUploadToBinaryPackageRecordReader.java:56) at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.java:192) at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:176) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:176) If you saw similar trace before, please share your comment. Thanks