[ https://issues.apache.org/jira/browse/YARN-7575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269175#comment-16269175 ]
Eric Payne commented on YARN-7575: ---------------------------------- [~sunilg], the fix for the UI NPE looks good, but the other problem I'm having is that when I increase a node size, the queue doesn't grow. My configs are as follows: - 4 node managers, 5120GB and 10 Vcores each for a total of [20480GB, 40 VCores] - {{yarn.scheduler.capacity.root.default.capacity}}: [memory=10240,vcores=20] - {{yarn.scheduler.capacity.root.eng.capacity}}: [memory=10240,vcores=20] - Note that I do not set root.capacity, nor do I set any maximum-capacity. My use case is as follows: - I start a job requesting 22.5GB and 45 vcores (container size=0.5GB) - the job consumes 20GB and 40 vcores - I add 2.5GB and 5 vcores to one of the nodes: {{yarn rmadmin -updateNodeResource host:port 7680 15}} - One more container is assigned to the job, but that only brings the job to 20.5GB and 41 vcores. > When using absolute capacity configuration with no max capacity, scheduler UI > NPEs and can't grow queue > ------------------------------------------------------------------------------------------------------- > > Key: YARN-7575 > URL: https://issues.apache.org/jira/browse/YARN-7575 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler > Reporter: Eric Payne > Attachments: YARN-7575-YARN-5881.001.patch > > > I encountered the following while reviewing and testing branch YARN-5881. > The design document from YARN-5881 says that for max-capacity: > {quote} > 3) For each queue, we require: > a) if max-resource not set, it automatically set to parent.max-resource > {quote} > When I try leaving blank {{yarn.scheduler.capacity.< > queue-path>.maximum-capacity}}, the RMUI scheduler page refuses to render. It > looks like it's in {{CapacitySchedulerPage$ LeafQueueInfoBlock}}: > {noformat} > 2017-11-28 11:29:16,974 [qtp43473566-220] ERROR webapp.Dispatcher: error > handling URI: /cluster/scheduler > java.lang.reflect.InvocationTargetException > ... > at > org.apache.hadoop.yarn.server.resourcemanager.webapp.CapacitySchedulerPage$LeafQueueInfoBlock.renderQueueCapacityInfo(CapacitySchedulerPage.java:164) > at > org.apache.hadoop.yarn.server.resourcemanager.webapp.CapacitySchedulerPage$LeafQueueInfoBlock.renderLeafQueueInfoWithoutParition(CapacitySchedulerPage.java:129) > {noformat} > Also... A job will run in the leaf queue with no max capacity set and it will > grow to the max capacity of the cluster, but if I add resources to the node, > the job won't grow any more even though it has pending resources. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org