[
https://issues.apache.org/jira/browse/MAHOUT-471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12899432#action_12899432
]
Han Hui Wen edited comment on MAHOUT-471 at 8/17/10 11:08 AM:
---------------------------------------------------------------
one possible reason as following:
We now use Varint to write int ,it actually writes a few bytes instead of one
int.
In writing phase ,it's no problem ,but in reading phase it may cause one int
stored in several blocks (not sure here ),
so it maybe the stream is ended,but we have not found end tag (the first bit of
the byte is "1").
was (Author: huiwenhan):
one possible reason as following:
We now use Varint to write int ,it actually writes a few bytes instead of one
int.
In writing phase ,it's no problem ,but in reading phase it may cause one int
stored in several blocks (not sure here ),
so it maybe the steam is ended,but we have not found end tag (the first bit of
the byte is "1").
> RowSimilarityJob-Mapper-EntriesToVectorsReducer failure
> --------------------------------------------------------
>
> Key: MAHOUT-471
> URL: https://issues.apache.org/jira/browse/MAHOUT-471
> Project: Mahout
> Issue Type: Bug
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Han Hui Wen
> Priority: Minor
> Fix For: 0.4
>
>
> I used Boolean Data and SIMILARITY_TANIMOTO_COEFFICIENT
> java.io.IOException: Task: attempt_201008101359_0084_r_000000_0 - The reduce
> copier failed
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:380)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> Caused by: java.io.IOException: Intermediate merge failed
> at
> org.apache.hadoop.mapred.ReduceTask$ReduceCopier$InMemFSMergeThread.doInMemMerge(ReduceTask.java:2576)
> at
> org.apache.hadoop.mapred.ReduceTask$ReduceCopier$InMemFSMergeThread.run(ReduceTask.java:2501)
> Caused by: java.lang.RuntimeException: java.io.EOFException
> at
> org.apache.hadoop.io.WritableComparator.compare(WritableComparator.java:103)
> at org.apache.hadoop.mapred.Merger$MergeQueue.lessThan(Merger.java:373)
> at org.apache.hadoop.util.PriorityQueue.upHeap(PriorityQueue.java:123)
> at org.apache.hadoop.util.PriorityQueue.put(PriorityQueue.java:50)
> at org.apache.hadoop.mapred.Merger$MergeQueue.merge(Merger.java:447)
> at org.apache.hadoop.mapred.Merger$MergeQueue.merge(Merger.java:381)
> at org.apache.hadoop.mapred.Merger.merge(Merger.java:107)
> at org.apache.hadoop.mapred.Merger.merge(Merger.java:93)
> at
> org.apache.hadoop.mapred.ReduceTask$ReduceCopier$InMemFSMergeThread.doInMemMerge(ReduceTask.java:2551)
> ... 1 more
> Caused by: java.io.EOFException
> at java.io.DataInputStream.readByte(DataInputStream.java:250)
> at org.apache.mahout.math.Varint.readUnsignedVarInt(Varint.java:159)
> at org.apache.mahout.math.Varint.readSignedVarInt(Varint.java:140)
> at
> org.apache.mahout.math.hadoop.similarity.SimilarityMatrixEntryKey.readFields(SimilarityMatrixEntryKey.java:65)
> at
> org.apache.hadoop.io.WritableComparator.compare(WritableComparator.java:97)
> ... 9 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
