[
https://issues.apache.org/jira/browse/YARN-1707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14075076#comment-14075076
]
Carlo Curino commented on YARN-1707:
------------------------------------
The attached patch is part of the YARN-1051 effort, as for the other patches in
this series does not work on itself but it has been cut for ease of reviewing.
Given previous discussions, we introduced subclasses for ParentQueue and
LeafQueue that are dynamically addable/removable/resizeable,
as well as changes in the CapacityScheduler to support the "move" of
applications across queues. These are core features, we tested on a cluster
running lots of gridmix and manual jobs, and seems to work fine, but I am sure
there are corner cases and possibly metrics that are not updated
correctly under all cases. We should also create a new set of tests for the
dynamic behavior of the CapacityScheduler.
> Making the CapacityScheduler more dynamic
> -----------------------------------------
>
> Key: YARN-1707
> URL: https://issues.apache.org/jira/browse/YARN-1707
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler
> Reporter: Carlo Curino
> Assignee: Carlo Curino
> Attachments: YARN-1707.patch
>
>
> The CapacityScheduler is a rather static at the moment, and refreshqueue
> provides a rather heavy-handed way to reconfigure it. Moving towards
> long-running services (tracked in YARN-896) and to enable more advanced
> admission control and resource parcelling we need to make the
> CapacityScheduler more dynamic. This is instrumental to the umbrella jira
> YARN-1051.
> Concretely this require the following changes:
> * create queues dynamically
> * destroy queues dynamically
> * dynamically change queue parameters (e.g., capacity)
> * modify refreshqueue validation to enforce sum(child.getCapacity())<= 100%
> instead of ==100%
> We limit this to LeafQueues.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
