[
https://issues.apache.org/jira/browse/YARN-914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14312677#comment-14312677
]
Vinod Kumar Vavilapalli commented on YARN-914:
----------------------------------------------
Is the decommission_timeout a server side config or specifiable for each
decommission request? The current refreshNodes approach will not enable a per
request config. IAC, I think we should also have a CLI command to decommission
the node which optionally waits till the decommission succeeds.
Regarding storage of the decommission state, YARN-2567 also plans to make sure
that the state of all nodes is maintained up to date on the state-store. That
helps with many other cases too. We should combine these efforts. /cc [~jianhe]
Regarding long running services, I think it makes sense to let the admin
initiating the decommission know - not in terms of policy but as a diagnostic.
Other than waiting for a timeout, the admin may not have noticed that a service
is running on this node before the decommission is triggered.
bq. Alternatively we can remove graceful decommission timeout for YARN layer
and let external decommission script handle that. If the script considers the
graceful decommission takes too long, it can ask YARN to do the immediate
decommission.
This is the umbrella concern I have. There are two ways to do this: Let YARN
manage the decommission process or manage it on top of YARN. If the later is
the approach, I don't see a lot to be done here besides YARN-291. No?
> Support graceful decommission of nodemanager
> --------------------------------------------
>
> Key: YARN-914
> URL: https://issues.apache.org/jira/browse/YARN-914
> Project: Hadoop YARN
> Issue Type: Improvement
> Affects Versions: 2.0.4-alpha
> Reporter: Luke Lu
> Assignee: Junping Du
> Attachments: Gracefully Decommission of NodeManager (v1).pdf
>
>
> When NMs are decommissioned for non-fault reasons (capacity change etc.),
> it's desirable to minimize the impact to running applications.
> Currently if a NM is decommissioned, all running containers on the NM need to
> be rescheduled on other NMs. Further more, for finished map tasks, if their
> map output are not fetched by the reducers of the job, these map tasks will
> need to be rerun as well.
> We propose to introduce a mechanism to optionally gracefully decommission a
> node manager.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
