[
https://issues.apache.org/jira/browse/YARN-5829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15905999#comment-15905999
]
ASF GitHub Bot commented on YARN-5829:
--------------------------------------
GitHub user szegedim opened a pull request:
https://github.com/apache/hadoop/pull/201
YARN-5829 Reserve containers for preemptors
YARN-5829 Reserve containers for preemptors.
The change does the following.
1. When a container is selected for preemption, it stores the preemptor
next to the container.
2. When a container is finally completed it creates a reservation for the
preemptor application.
3. On the next node update we check first reservations, then preemption
reservations. They are satisfied before any other request, even if those have
higher priorities. This is used tho avoid stealing preempted containers.
4. It also disables standard reservations because I noticed that the
preempted applications reserve nodes for themselves stealing back the preempted
resources.
5. If a preemptor is deleted before the preemption or after the preemption,
the assignment won't happen at node update.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/szegedim/hadoop YARN-5829
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/hadoop/pull/201.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #201
----
commit 07acc98178ea88bdd1621dc39e2aa28354be7f63
Author: Miklos Szegedi <miklos.szeg...@cloudera.com>
Date: 2017-03-10T04:50:08Z
YARN-5829 Initial suggestion. Track preempted containers and create
reservation, when they free up
commit 7009aac04f0eb9301868b50c6e0c1720db3b92e2
Author: Miklos Szegedi <miklos.szeg...@cloudera.com>
Date: 2017-03-11T01:00:28Z
YARN-5829 Added unit test, fixed concurrent exception bug with multiple
preempted applications.
commit bbaa3a5980d1e25d2788c5f80539dfcdff500184
Author: Miklos Szegedi <miklos.szeg...@cloudera.com>
Date: 2017-03-11T01:42:15Z
YARN-5829 Make sure deleted applications do not satisfy reservations.
----
> FS preemption should reserve a node before considering containers on it for
> preemption
> --------------------------------------------------------------------------------------
>
> Key: YARN-5829
> URL: https://issues.apache.org/jira/browse/YARN-5829
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: fairscheduler
> Reporter: Karthik Kambatla
> Assignee: Miklos Szegedi
>
> FS preemption evaluates nodes for preemption, and subsequently preempts
> identified containers. If this node is not reserved for a specific
> application, any other application could be allocated resources on this node.
> Reserving the node for the starved application before preempting containers
> would help avoid this.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
