Checksum Error --------------- Key: NUTCH-742 URL: https://issues.apache.org/jira/browse/NUTCH-742 Project: Nutch Issue Type: Bug Components: indexer Affects Versions: 1.0.0 Environment: linux ubuntu8.0.4 64bit 10datanode 4G of memory per node Reporter: mawanqiang
Approximately 1 million data used to create index when nutch1.0 error. The error is: java.lang.RuntimeException: problem advancing post rec#6758513 at org.apache.hadoop.mapred.Task$ValuesIterator.next(Task.java:883) at org.apache.hadoop.mapred.ReduceTask$ReduceValuesIterator.moveToNext(ReduceTask.java:237) at org.apache.hadoop.mapred.ReduceTask$ReduceValuesIterator.next(ReduceTask.java:233) at org.apache.nutch.indexer.IndexerMapReduce.reduce(IndexerMapReduce.java:79) at org.apache.nutch.indexer.IndexerMapReduce.reduce(IndexerMapReduce.java:50) at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:436) at org.apache.hadoop.mapred.Child.main(Child.java:158) Caused by: org.apache.hadoop.fs.ChecksumException: Checksum Error at org.apache.hadoop.mapred.IFileInputStream.doRead(IFileInputStream.java:153) at org.apache.hadoop.mapred.IFileInputStream.read(IFileInputStream.java:90) at org.apache.hadoop.mapred.IFile$Reader.readData(IFile.java:301) at org.apache.hadoop.mapred.IFile$Reader.rejigData(IFile.java:331) at org.apache.hadoop.mapred.IFile$Reader.readNextBlock(IFile.java:315) at org.apache.hadoop.mapred.IFile$Reader.next(IFile.java:377) at org.apache.hadoop.mapred.Merger$Segment.next(Merger.java:174) at org.apache.hadoop.mapred.Merger$MergeQueue.adjustPriorityQueue(Merger.java:277) at org.apache.hadoop.mapred.Merger$MergeQueue.next(Merger.java:297) at org.apache.hadoop.mapred.Task$ValuesIterator.readNextKey(Task.java:922) at org.apache.hadoop.mapred.Task$ValuesIterator.next(Task.java:881) ... 6 more -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.