[
https://issues.apache.org/jira/browse/MAPREDUCE-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13117330#comment-13117330
]
Vinod Kumar Vavilapalli commented on MAPREDUCE-3121:
----------------------------------------------------
In spirit, this is same as MAPREDUCE-2413, but we'll probably get away with a
far cleaner implementation with NodeManager.
- Overall, we have a list of local dirs and a list of log-dirs which we want
to periodically check for sanity and have NM automatically offline one/more of
them when they go bad.
- We won't touch(for e.g., kill) any running containers when a disk goes bad.
We'll let the running containers fail if at all.
- All the future containers will avoid the bad disks and get their paths to
work on the remaining good disks.
- NM should perhaps piggyback on the health report, and update to the RM if
any disks are going bad, helps a lot for operability.
- Also, if all or a major percentage of disks turn bad at startup,
NodeManager should either fail to start or get started, and report the RM that
it is bad and needs operator intervention.
- If all or a major percentage of disks turn bad during runtime, NodeManager
can update the RM and ask for a halt in the scheduling.
> NodeManager should handle disk-failures
> ---------------------------------------
>
> Key: MAPREDUCE-3121
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3121
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Fix For: 0.23.0
>
>
> This is akin to MAPREDUCE-2413 but for YARN's NodeManager. We want to
> minimize the impact of transient/permanent disk failures on containers. With
> larger number of disks per node, the ability to continue to run containers on
> other disks is crucial.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
