[
https://issues.apache.org/jira/browse/YARN-7424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16237768#comment-16237768
]
Eric Payne commented on YARN-7424:
----------------------------------
Thanks for the review, [~sunilg].
bq.
"yarn.resourcemanager.monitor.capacity.preemption.intra-queue-preemption.minimum-threshold"
could be configured to start intra queue preemption on a queue. But yes, this
is generic in all queues.
IIUC {{minimum-threshold}} will prevent intra-queue preemption from acting
within a queue until the queue's used resources are above {{minimum-thresshold
* capacity}}, which is not really helpful here.
bq. max-allowable-limit helps only to control preemption in a given round of
preemption calculation. This could be configured to a very low value so that
only few resource will be preempted in such cases.
Yes, and we could try to reduce this value even more, which could potentially
be helpful. I am just surprised that reducing this from 20% to 3% did not have
nearly as much effect as I expected.
bq. Now the solution which you mentioned will help to control preemption.
Actually, after thinking about it more, the proposed solution is not very
useful. Here's why:
- Queue1 is configured with 1% MULP
- User1 submits app1 to queue1 and consumes 100% of the resources
- User2 submits app2 to queue1 and requests resources
- The preemption monitor preempts resources from aap1 and the capacity
scheduler gives them to app2 until app2 is at 1%
- User 3 submits app3 to queue1 and requests resources.
- The preemption monitor preempts resources from app1, but the capacity
scheduler doesn't give them to app3. It gives them to app2 because the user
limit resource value is 33%, and app2 came before app3, and user2 is below 33%.
- So, with the proposed solution, user3 keeps asking for resources and the
preemption monitor keeps taking them from app1 and the capacity scheduler keeps
giving them to app2 until user2 is above 33%.
If you multiply this out to 60 users all asking for resources in a queue with
1% MULP, it is doing pretty much the exact same amount of preempting and
balancing as before. In order to create the "desired" behavior, we would have
to fundamentally change the way the capacity scheduler works, which we don't
want to do.
> Capacity Scheduler Intra-queue preemption: add property to only preempt up to
> configured MULP
> ---------------------------------------------------------------------------------------------
>
> Key: YARN-7424
> URL: https://issues.apache.org/jira/browse/YARN-7424
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacity scheduler, scheduler preemption
> Affects Versions: 3.0.0-beta1, 2.8.2
> Reporter: Eric Payne
> Assignee: Eric Payne
> Priority: Major
>
> If the queue's configured minimum user limit percent (MULP) is something
> small like 1%, all users will max out well over their MULP until 100 users
> have apps in the queue. Since the intra-queue preemption monitor tries to
> balance the resource among the users, most of the time in this use case it
> will be preempting containers on behalf of users that are already over their
> MULP guarantee.
> This JIRA proposes that a property should be provided so that a queue can be
> configured to only preempt on behalf of a user until that user has reached
> its MULP.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
