[
https://issues.apache.org/jira/browse/YARN-9116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733952#comment-16733952
]
Weiwei Yang commented on YARN-9116:
-----------------------------------
Hi [~aihuaxu]
{quote}Actually that would introduce many queue level configuration if we don't
introduce new property even with such inheritance. Even after we implement
inheritance mechanism, we have to set the global to be 120G/256vCores (the
maximum value allowed in the cluster) and then override all the top queues to
be 16G/16vCores and set the larger container top queue to 120G/256vCores.
{quote}
I was thinking if we can setup something like following (use memory as example):
{code:java}
yarn.scheduler.capacity.root.maximum-allocation-mb=16G
yarn.scheduler.capacity.root.large.maximum-allocation-mb=120G
{code}
so if the queue structure is like
{code:java}
-- root (16G)
---- a (16G)
---- b (16G)
---- c (16G)
---- large (120G)
{code}
This implicits the queue level max-allocation is inherited from its parent and
it can be overwritten at the same time.
Would this work?
Thanks
> Capacity Scheduler: add the default maximum-allocation-mb and
> maximum-allocation-vcores for the queues
> ------------------------------------------------------------------------------------------------------
>
> Key: YARN-9116
> URL: https://issues.apache.org/jira/browse/YARN-9116
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
> Affects Versions: 2.7.0
> Reporter: Aihua Xu
> Assignee: Aihua Xu
> Priority: Major
> Attachments: YARN-9116.1.patch
>
>
> YARN-1582 adds the support of maximum-allocation-mb configuration per queue
> which is targeting to support larger container features on dedicated queues
> (larger maximum-allocation-mb/maximum-allocation-vcores for such queue) .
> While to achieve larger container configuration, we need to increase the
> global maximum-allocation-mb/maximum-allocation-vcores (e.g. 120G/256) and
> then override those configurations with desired values on the queues since
> queue configuration can't be larger than cluster configuration. There are
> many queues in the system and if we forget to configure such values when
> adding a new queue, then such queue gets default 120G/256 which typically is
> not what we want.
> We can come up with a queue-default configuration (set to normal queue
> configuration like 16G/8), so the leaf queues gets such values by default.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
