[
https://issues.apache.org/jira/browse/YARN-8379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wangda Tan updated YARN-8379:
-----------------------------
Description:
Existing capacity scheduler only supports preemption for an underutilized queue
to reach its guaranteed resource. In addition to that, there’s an requirement
to get better balance between queues when all of them reach guaranteed resource
but with different fairness resource.
An example is, 3 queues with capacity, queue_a = 30%, queue_b = 30%, queue_c =
40%. At time T. queue_a is using 30%, queue_b is using 70%. Existing scheduler
preemption won't happen. But this is unfair to queue_a since queue_a has the
same guaranteed resources.
Before YARN-5864, capacity scheduler do additional preemption to balance
queues. We changed the logic since it could preempt too many containers between
queues when all queues are satisfied.
was:
Existing capacity scheduler only supports preemption for an underutilized queue
to reach its guaranteed resource. In addition to that, there’s an requirement
to get better balance between queues when all of them reach guaranteed resource
but with different fairness resource.
An example is, 3 queues with capacity, queue_a = 30%, queue_b = 30%, queue_c =
40%. At time T. queue_a is using 30%, queue_b is using 70%. Existing scheduler
preemption won't happen. But this is unfair to queue_b since queue_a has the
same guaranteed resources.
Before YARN-5864, capacity scheduler do additional preemption to balance
queues. We changed the logic since it could preempt too many containers between
queues when all queues are satisfied.
> Add an option to allow Capacity Scheduler preemption to balance satisfied
> queues
> --------------------------------------------------------------------------------
>
> Key: YARN-8379
> URL: https://issues.apache.org/jira/browse/YARN-8379
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Wangda Tan
> Assignee: Wangda Tan
> Priority: Major
>
> Existing capacity scheduler only supports preemption for an underutilized
> queue to reach its guaranteed resource. In addition to that, there’s an
> requirement to get better balance between queues when all of them reach
> guaranteed resource but with different fairness resource.
> An example is, 3 queues with capacity, queue_a = 30%, queue_b = 30%, queue_c
> = 40%. At time T. queue_a is using 30%, queue_b is using 70%. Existing
> scheduler preemption won't happen. But this is unfair to queue_a since
> queue_a has the same guaranteed resources.
> Before YARN-5864, capacity scheduler do additional preemption to balance
> queues. We changed the logic since it could preempt too many containers
> between queues when all queues are satisfied.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
