[
https://issues.apache.org/jira/browse/HADOOP-4714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12651777#action_12651777
]
Christian Kunz commented on HADOOP-4714:
----------------------------------------
Although this would solve the issue for our particular case, I can imagine a
situation (e.g. single reducer with highly aggregated huge records) where this
would not help, i.e. the time component needs to be factored into the progress
reporting. Progress should always be reported at smaller intervals than the
timeout which is configurable and could be a small number.
> map tasks timing out during merge phase
> ---------------------------------------
>
> Key: HADOOP-4714
> URL: https://issues.apache.org/jira/browse/HADOOP-4714
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.18.1
> Reporter: Christian Kunz
> Assignee: Jothi Padmanabhan
> Attachments: hadoop-4714.patch
>
>
> With compression of transient data turned on some parts of the merge phase
> seem to not report progress enough.
> We see a lot of task failures during the merge phase, most of them timing out
> (even with a 20 min timeout)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
