[
https://issues.apache.org/jira/browse/YARN-8720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weiwei Yang updated YARN-8720:
------------------------------
Fix Version/s: 2.9.2
> CapacityScheduler does not enforce max resource allocation check at queue
> level
> -------------------------------------------------------------------------------
>
> Key: YARN-8720
> URL: https://issues.apache.org/jira/browse/YARN-8720
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler, capacityscheduler, resourcemanager
> Affects Versions: 2.7.0
> Reporter: Tarun Parimi
> Assignee: Tarun Parimi
> Priority: Major
> Fix For: 3.2.0, 2.9.2, 3.0.4, 3.1.2
>
> Attachments: YARN-8720-branch-2.8.001.patch, YARN-8720.001.patch,
> YARN-8720.002.patch
>
>
> The value of
> yarn.scheduler.capacity.<queue-path>.maximum-allocation-mb/vcores is not
> strictly enforced when applications request containers. An
> InvalidResourceRequestException is thrown only when the ResourceRequest is
> greater than the global value of yarn.scheduler.maximum-allocation-mb/vcores
> . So for an example configuration such as below,
>
> {code:java}
> yarn.scheduler.maximum-allocation-mb=4096
> yarn.scheduler.capacity.root.test.maximum-allocation-mb=2048
> {code}
>
> The below DSShell command runs successfully and asks an AM container of size
> 4096MB which is greater than max 2048MB configured in test queue.
> {code:java}
> yarn jar $YARN_HOME/hadoop-yarn-applications-distributedshell.jar
> -num_containers 1 -jar
> $YARN_HOME/hadoop-yarn-applications-distributedshell.jar -shell_command
> "sleep 60" -container_memory=4096 -master_memory=4096 -queue=test{code}
> Instead it should not launch the application and fail with
> InvalidResourceRequestException . The child container however will be
> requested with size 2048MB as DSShell AppMaster does the below check before
> ResourceRequest ask with RM.
> {code:java}
> // A resource ask cannot exceed the max.
> if (containerMemory > maxMem) {
> LOG.info("Container memory specified above max threshold of cluster."
> + " Using max value." + ", specified=" + containerMemory + ", max="
> + maxMem);
> containerMemory = maxMem;
> }{code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
