[
https://issues.apache.org/jira/browse/MAPREDUCE-5891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14107011#comment-14107011
]
Jason Lowe commented on MAPREDUCE-5891:
---------------------------------------
bq. The reason I choose to do two things separately here is because the retry
connection has some IOExceptions (like things get failed in verifyConnection()
is not because of NM restart) that we don't need to retry anymore. Wrap this
kind of exceptions within retry may sounds unnecessary. Thoughts?
Ah, yes good point. We shouldn't retry exceptions that we know aren't related
to the NM bouncing, like secret hash verification failures.
I was under the impression that the copyMapOutput retry could cause a reconnect
which itself would have retries. If that's not the case then there's no issue
with nested retries.
> Improved shuffle error handling across NM restarts
> --------------------------------------------------
>
> Key: MAPREDUCE-5891
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5891
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Affects Versions: 2.5.0
> Reporter: Jason Lowe
> Assignee: Junping Du
> Attachments: MAPREDUCE-5891-demo.patch
>
>
> To minimize the number of map fetch failures reported by reducers across an
> NM restart it would be nice if reducers only reported a fetch failure after
> trying for at specified period of time to retrieve the data.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
