Prabhu Joseph created YARN-10108:
------------------------------------
Summary: FS-CS converter: nestedUserQueue with default rule maps
to invalid queue mapping
Key: YARN-10108
URL: https://issues.apache.org/jira/browse/YARN-10108
Project: Hadoop YARN
Issue Type: Sub-task
Affects Versions: 3.3.0
Reporter: Prabhu Joseph
FS Queue Placement Policy
{code:java}
<queuePlacementPolicy>
<rule name="nestedUserQueue">
<rule name="default" queue="users"/>
</rule>
</queuePlacementPolicy>
{code}
gets mapped to an invalid CS queue mapping "u:%user:root.users.%user"
RM fails to start with above queue mapping in CS
{code:java}
2020-01-28 00:19:12,889 FATAL
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error starting
ResourceManager
org.apache.hadoop.service.ServiceStateException: java.io.IOException: mapping
contains invalid or non-leaf queue [%user] and invalid parent queue [root.users]
at
org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:173)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:829)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:1247)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:324)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1534)
Caused by: java.io.IOException: mapping contains invalid or non-leaf queue
[%user] and invalid parent queue [root.users]
at
org.apache.hadoop.yarn.server.resourcemanager.placement.QueuePlacementRuleUtils.validateQueueMappingUnderParentQueue(QueuePlacementRuleUtils.java:48)
at
org.apache.hadoop.yarn.server.resourcemanager.placement.UserGroupMappingPlacementRule.validateAndGetAutoCreatedQueueMapping(UserGroupMappingPlacementRule.java:363)
at
org.apache.hadoop.yarn.server.resourcemanager.placement.UserGroupMappingPlacementRule.initialize(UserGroupMappingPlacementRule.java:300)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.getUserGroupMappingPlacementRule(CapacityScheduler.java:671)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.updatePlacementRules(CapacityScheduler.java:712)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.initializeQueues(CapacityScheduler.java:753)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.initScheduler(CapacityScheduler.java:361)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.serviceInit(CapacityScheduler.java:426)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
... 7 more
{code}
QueuePlacementConverter#handleNestedRule has to be fixed.
{code:java}
else if (pr instanceof DefaultPlacementRule) {
DefaultPlacementRule defaultRule = (DefaultPlacementRule) pr;
mapping.append("u:" + USER + ":")
.append(defaultRule.defaultQueueName)
.append("." + USER);
}
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
