[
https://issues.apache.org/jira/browse/YARN-8379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16525328#comment-16525328
]
Wangda Tan commented on YARN-8379:
----------------------------------
bq. we could definitely make a method inside PreemptionCandidatesSelector, and
call it explicitly to reset curCandidates per round, but this way it makes the
code even harder to read. Any better suggestions here?
Can we simply new the curCandidates map inside {{selectCandidates}} for each
selector?
bq. This test case was intend to demonstrate selected candidates will be
actually killed after custom timeout was reached. This part of code is the
intention.
What I can see from the UT is, queue1 gets all containers (39G) and queue2 asks
a 4G container. After wait the 4G containers will be preempted from queue1. I
think our purpose is: both queue1 / queue2 are overutilized, we need to balance
resources from queue1 to queue2 and only after X secs, containers from queue1
will be preempted. correct? It will be similar to follow the example
{{testPreemptionToBalanceUsedPlusPendingLessThanGuaranteed}}.
> Add an option to allow Capacity Scheduler preemption to balance satisfied
> queues
> --------------------------------------------------------------------------------
>
> Key: YARN-8379
> URL: https://issues.apache.org/jira/browse/YARN-8379
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Wangda Tan
> Assignee: Zian Chen
> Priority: Major
> Attachments: YARN-8379.001.patch, YARN-8379.002.patch,
> YARN-8379.003.patch, YARN-8379.004.patch, YARN-8379.005.patch,
> ericpayne.confs.tgz
>
>
> Existing capacity scheduler only supports preemption for an underutilized
> queue to reach its guaranteed resource. In addition to that, there’s an
> requirement to get better balance between queues when all of them reach
> guaranteed resource but with different fairness resource.
> An example is, 3 queues with capacity, queue_a = 30%, queue_b = 30%, queue_c
> = 40%. At time T. queue_a is using 30%, queue_b is using 70%. Existing
> scheduler preemption won't happen. But this is unfair to queue_a since
> queue_a has the same guaranteed resources.
> Before YARN-5864, capacity scheduler do additional preemption to balance
> queues. We changed the logic since it could preempt too many containers
> between queues when all queues are satisfied.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
