[
https://issues.apache.org/jira/browse/YARN-9116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16737855#comment-16737855
]
Wangda Tan commented on YARN-9116:
----------------------------------
[~cheersyang], [~aihuaxu],
The proposal from Weiwei sounds good to me. We have several other fields in CS
have similar inheritable behaviors. To me it is not an incompatible change
since we don't expect user set maximum-allocation of parent queues.
And also, the old config only support setting maximum allocation for mem and
vcores, can we add a field to support customized resources like GPU/FPGA? I
think we can refer to Absolute Resource Specification in Capacity Scheduler.
See:
[https://hadoop.apache.org/docs/r3.1.1/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html],
search for "Resource Allocation using Absolute Resources configuration". I
think we should either deprecated the old maximum-allocation-mb/vcores or at
least make maximum-allocation can override maximum-allocation-mb/vcores.
There's an on-going work of YARN-9161 to add support of Absolute Resource for
customized resource types to Capacity Scheduler. We should make sure consistent
format is being used.
Thoughts?
> Capacity Scheduler: add the default maximum-allocation-mb and
> maximum-allocation-vcores for the queues
> ------------------------------------------------------------------------------------------------------
>
> Key: YARN-9116
> URL: https://issues.apache.org/jira/browse/YARN-9116
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
> Affects Versions: 2.7.0
> Reporter: Aihua Xu
> Assignee: Aihua Xu
> Priority: Major
> Attachments: YARN-9116.1.patch
>
>
> YARN-1582 adds the support of maximum-allocation-mb configuration per queue
> which is targeting to support larger container features on dedicated queues
> (larger maximum-allocation-mb/maximum-allocation-vcores for such queue) .
> While to achieve larger container configuration, we need to increase the
> global maximum-allocation-mb/maximum-allocation-vcores (e.g. 120G/256) and
> then override those configurations with desired values on the queues since
> queue configuration can't be larger than cluster configuration. There are
> many queues in the system and if we forget to configure such values when
> adding a new queue, then such queue gets default 120G/256 which typically is
> not what we want.
> We can come up with a queue-default configuration (set to normal queue
> configuration like 16G/8), so the leaf queues gets such values by default.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
