Tarun Parimi created YARN-8720:
----------------------------------
Summary: CapacityScheduler does not enforce
yarn.scheduler.capacity.<queue-path>.maximum-allocation-mb/vcores when
configured
Key: YARN-8720
URL: https://issues.apache.org/jira/browse/YARN-8720
Project: Hadoop YARN
Issue Type: Bug
Components: capacity scheduler, capacityscheduler, resourcemanager
Affects Versions: 2.7.0
Reporter: Tarun Parimi
The value of yarn.scheduler.capacity.<queue-path>.maximum-allocation-mb/vcores
is not strictly enforced when applications request containers. An
InvalidResourceRequestException is thrown only when the ResourceRequest is
greater than the global value of yarn.scheduler.maximum-allocation-mb/vcores .
So for an example configuration such as below,
{code:java}
yarn.scheduler.maximum-allocation-mb=4096
yarn.scheduler.capacity.root.test.maximum-allocation-mb=2048
{code}
The below DSShell command runs successfully and asks an AM container of size
4096MB which is greater than max 2048MB configured in test queue.
{code:java}
yarn jar $YARN_HOME/hadoop-yarn-applications-distributedshell.jar
-num_containers 1 -jar $YARN_HOME/hadoop-yarn-applications-distributedshell.jar
-shell_command "sleep 60" -container_memory=4096 -master_memory=4096
-queue=test{code}
Instead it should not launch the application and fail with
InvalidResourceRequestException . The child container however will be requested
with size 2048MB as DSShell AppMaster does the below check before
ResourceRequest ask with RM.
{code:java}
// A resource ask cannot exceed the max.
if (containerMemory > maxMem) {
LOG.info("Container memory specified above max threshold of cluster."
+ " Using max value." + ", specified=" + containerMemory + ", max="
+ maxMem);
containerMemory = maxMem;
}{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
