[
https://issues.apache.org/jira/browse/HADOOP-4714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12650126#action_12650126
]
Jothi Padmanabhan commented on HADOOP-4714:
-------------------------------------------
It could also be possible that we have lesser records in each spill file
because each record is of a bigger size. Since the limits for spill files are
by size (io.sort.mb) and not record count, we probably are not touching the
10,000 mark at all?
Christian, could you tell us what would be the approximate/average record size
of the intermediate map output and the io.sort.mb in this case?
> map tasks timing out during merge phase
> ---------------------------------------
>
> Key: HADOOP-4714
> URL: https://issues.apache.org/jira/browse/HADOOP-4714
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.18.1
> Reporter: Christian Kunz
>
> With compression of transient data turned on some parts of the merge phase
> seem to not report progress enough.
> We see a lot of task failures during the merge phase, most of them timing out
> (even with a 20 min timeout)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
