[
https://issues.apache.org/jira/browse/MAPREDUCE-5891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14106341#comment-14106341
]
Junping Du commented on MAPREDUCE-5891:
---------------------------------------
Thanks [~jlowe] for quickly feedback!
bq. If we get an error and need to reconnect then we should rebuild the URL
based on the remaining maps rather than keep using the original URL and
throwing away data for maps we already processed.
That's very good point, will fix it in patch latter.
bq. It seems like we should only need one-level of retry rather than two. I
think we only need to retry at the top-level where we try to copy map outputs.
If copying map outputs fails either because of a severed connection, a
connection reset/timeout, or whatever and we have not spent too much time
trying to recover from errors for the same map output then we retry.
One-level of retry sounds good. Actually, the current implementation is mostly
a one-level retry, but just retry connection and copyMapOutput separately. The
reason I choose to do two things separately here is because the retry
connection has some IOExceptions (like things get failed in verifyConnection()
is not because of NM restart) that we don't need to retry anymore. Wrap this
kind of exceptions within retry may sounds unnecessary. Thoughts?
bq. We probably should use Time.monotonicNow or something similar that won't
break the desired delay behavior if the clock changes. "miliseconds" should be
"milliseconds"
Also good points, will fix it in latter patch.
> Improved shuffle error handling across NM restarts
> --------------------------------------------------
>
> Key: MAPREDUCE-5891
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5891
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Affects Versions: 2.5.0
> Reporter: Jason Lowe
> Assignee: Junping Du
> Attachments: MAPREDUCE-5891-demo.patch
>
>
> To minimize the number of map fetch failures reported by reducers across an
> NM restart it would be nice if reducers only reported a fetch failure after
> trying for at specified period of time to retrieve the data.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
