[
https://issues.apache.org/jira/browse/MAPREDUCE-4330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13613520#comment-13613520
]
Siddharth Seth commented on MAPREDUCE-4330:
-------------------------------------------
bq. Omkar, the code change to JobImpl itself looks good.
Looking at this a little more, JobImpl needs some changes as well. What's
happening rightnow is that multiple SUCCEEDED events can make it into the
CompleteEventList. Only the latest will be remembered as the successful event
for a task. While Shuffle can handle this, if one of the forgotten attempts
fails - it's status will not be updated in the list, and subsequent shuffle
attempts could see failures.
We can either choose to remember multiple successful attempts per task id and
leave all successful events in the main list, or maintain a backup list for
additional successful events - which can be used on failure of the previous
successful attempt.
> TaskAttemptCompletedEventTransition invalidates previously successful attempt
> without checking if the newly completed attempt is successful
> -------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4330
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4330
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.23.1
> Reporter: Bikas Saha
> Assignee: omkar vinit joshi
> Attachments: MAPREDUCE-4330-21032013.1.patch,
> MAPREDUCE-4330-21032013.patch
>
>
> The previously completed attempt is removed from
> successAttemptCompletionEventNoMap and marked OBSOLETE.
> After that, if the newly completed attempt is successful then it is added to
> the successAttemptCompletionEventNoMap.
> This seems wrong because the newly completed attempt could be failed and thus
> there is no need to invalidate the successful attempt.
> One error case would be when a speculative attempt completes with
> killed/failed after the successful version has completed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
