[
https://issues.apache.org/jira/browse/YARN-8095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16433883#comment-16433883
]
kyungwan nam commented on YARN-8095:
------------------------------------
Hi. [~cheersyang]
sorry for not clear explain.
let’s consider below,
nodes assigned to the default partition: n1~n4
nodes assigned to the label1 non-exclusive (sharable) partition: n5, n6
{quote}
in situation where 'batch' Queue is fully used.
{quote}
n1~n4 resources was fully allocated to root.batch.
{quote}
app1, which is submitted to 'longlived' Queue is allocated to label1 partition
by non-exclusive allocation
{quote}
at that time, there was no resources in the default partition (n1~n4).
as the preemption is trigger, the preemption candidates are selected from
root.batch.
but, at the same time, app1 containers can be allocated on the label1 partition
(n5, n6) due to *non-exclusive allocation*, which introduced in YARN-3214.
below is from YARN-3214
(1) Labeled apps get the preference on Labeled nodes
(2) If there is no ask for labeled resources we can assign those nodes to non
labeled apps
(3) If there is any future ask for those resources , we will preempt the non
labeled apps and give them back to labeled apps.
*non-exclusive allocation* that i said is (2)
if I can disable *non-exclusive allocation* for root.longlived,
as time goes on, the preemption candidates will be killed, app1 containers will
be allocated on default partition.
{quote}
after then, if an app is submitted to 'label1' Queue, app1 containers which are
placed in 'label1' partition are preempted first.
{quote}
(3) is the reason.
Thanks
> Allow disable non-exclusive allocation
> --------------------------------------
>
> Key: YARN-8095
> URL: https://issues.apache.org/jira/browse/YARN-8095
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacity scheduler
> Affects Versions: 2.8.3
> Reporter: kyungwan nam
> Priority: Major
>
> We have 'longlived' Queue, which is used for long-lived apps.
> In situation where default Partition resources are not enough, containers for
> long-lived app can be allocated to sharable Partition.
> Since then, containers for long-lived app can be easily preempted.
> We don’t want long-lived apps to be killed abruptly.
> Currently, non-exclusive allocation can happen regardless of whether the
> queue is accessible to the sharable Partition.
> It would be good if non-exclusive allocation can be disabled at queue level.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
