[
https://issues.apache.org/jira/browse/MAPREDUCE-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158026#comment-13158026
]
Mahadev konar commented on MAPREDUCE-3121:
------------------------------------------
@Eli,
The patch attached allows the NM to tolerate disk failures as the TT did in
MR-2413. The current MR code (both in 0.23 and 0.20.205) does not distinguish
why a task failed, could be a disk failure on a node or IO error. So the client
will see it as a failure. We could later sometime do optimizations to be smart
about disk failures (saying a task failure on a node with disk failure doesnt
count against max num failures) but I think it might have diminishing returns
in production environments. Want to open a jira to track it?
> NodeManager should handle disk-failures
> ---------------------------------------
>
> Key: MAPREDUCE-3121
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3121
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Ravi Gummadi
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: 3121.patch, 3121.v1.1.patch, 3121.v1.patch,
> 3121.v2.patch, 3121.v3.patch
>
>
> This is akin to MAPREDUCE-2413 but for YARN's NodeManager. We want to
> minimize the impact of transient/permanent disk failures on containers. With
> larger number of disks per node, the ability to continue to run containers on
> other disks is crucial.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
