[
https://issues.apache.org/jira/browse/HADOOP-4714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12650115#action_12650115
]
Christian Kunz commented on HADOOP-4714:
----------------------------------------
These tasks report a time of failure when there are still messages logged to
the syslog file, indicating that this might be a lack of progress reporting.
E.g.
attempt_200811221852_0001_m_093617_0 FAILED 0.00% 24-Nov-2008
06:26:23 (1hrs, 6mins, 20sec)
Task attempt_200811221852_0001_m_093617_0 failed to report status for 1218
seconds. Killing!
syslog:
...
2008-11-24 06:26:21,607 INFO org.apache.hadoop.mapred.MapTask: Index:
(1358423777, 193912, 94564)
2008-11-24 06:26:21,636 INFO org.apache.hadoop.mapred.Merger: Merging 13 sorted
segments
2008-11-24 06:26:21,654 INFO org.apache.hadoop.mapred.Merger: Down to the last
merge-pass, with 13 segments left of total size: 101764 bytes
2008-11-24 06:26:21,680 INFO org.apache.hadoop.mapred.MapTask: Index:
(1358518341, 173266, 89638)
2008-11-24 06:26:21,709 INFO org.apache.hadoop.mapred.Merger: Merging 13 sorted
segments
2008-11-24 06:26:21,749 INFO org.apache.hadoop.mapred.Merger: Down to the last
merge-pass, with 13 segments left of total size: 121477 bytes
2008-11-24 06:26:21,780 INFO org.apache.hadoop.mapred.MapTask: Index:
(1358607979, 219875, 102296)
2008-11-24 06:26:21,809 INFO org.apache.hadoop.mapred.Merger: Merging 13 sorted
segments
2008-11-24 06:26:21,827 INFO org.apache.hadoop.mapred.Merger: Down to the last
merge-pass, with 13 segments left of total size: 110955 bytes
2008-11-24 06:26:21,855 INFO org.apache.hadoop.mapred.MapTask: Index:
(1358710275, 191382, 97152)
2008-11-24 06:26:22,023 INFO org.apache.hadoop.mapred.Merger: Merging 13 sorted
segments
2008-11-24 06:26:22,178 INFO org.apache.hadoop.mapred.Merger: Down to the last
merge-pass, with 13 segments left of total size: 105632 bytes
> map tasks timing out during merge phase
> ---------------------------------------
>
> Key: HADOOP-4714
> URL: https://issues.apache.org/jira/browse/HADOOP-4714
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.18.1
> Reporter: Christian Kunz
>
> With compression of transient data turned on some parts of the merge phase
> seem to not report progress enough.
> We see a lot of task failures during the merge phase, most of them timing out
> (even with a 20 min timeout)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
