[ http://issues.apache.org/jira/browse/HADOOP-813?page=all ]
Devaraj Das updated HADOOP-813:
-------------------------------
Attachment: 813.patch
Thanks Owen for the review. I have removed the sortComplete variable and made
the thread exit only on interrupt instead.
The comment on "lots of changes" is actually due to the fact that I introduced
a try{ } finally{ } block and wrapped/indented existing code within the 'try'
block. This led to lots of diffs.
> map tasks lost during sort
> --------------------------
>
> Key: HADOOP-813
> URL: http://issues.apache.org/jira/browse/HADOOP-813
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.10.0
> Reporter: Owen O'Malley
> Assigned To: Devaraj Das
> Fix For: 0.10.0
>
> Attachments: 813.patch, 813.patch
>
>
> On a 500 node cluster, I had a bunch of map tasks get "lost" because they
> failed to report progress for 10 minutes. They appear to be in the sort stage
> at the end of the map. I hypothesize that the patch for HADOOP-331 does not
> update the map's progress during the sort/merge. If the sort/merge takes more
> than 10 minutes, the task is lost.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
