[
https://issues.apache.org/jira/browse/YARN-10099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Bacsko updated YARN-10099:
--------------------------------
Summary: FS-CS converter: handle allow-undeclared-pools and
user-as-default-queue properly and fix misc issues (was: FS-CS converter:
handle allow-undeclared-pools and user-as-default-queue properly and fix
mapping rule separator)
> FS-CS converter: handle allow-undeclared-pools and user-as-default-queue
> properly and fix misc issues
> -----------------------------------------------------------------------------------------------------
>
> Key: YARN-10099
> URL: https://issues.apache.org/jira/browse/YARN-10099
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Peter Bacsko
> Assignee: Peter Bacsko
> Priority: Major
> Labels: fs2cs
> Attachments: YARN-10099-001.patch, YARN-10099-002.patch,
> YARN-10099-003.patch, YARN-10099-004.patch, YARN-10099-005.patch
>
>
> This ticket is intended to fix two issues:
> 1. Based on the latest documentation, there are two important properties that
> are ignored if we have placement rules:
> ||Property||Explanation||
> |yarn.scheduler.fair.allow-undeclared-pools|If this is true, new queues can
> be created at application submission time, whether because they are specified
> as the application’s queue by the submitter or because they are placed there
> by the user-as-default-queue property. If this is false, any time an app
> would be placed in a queue that is not specified in the allocations file, it
> is placed in the “default” queue instead. Defaults to true. *If a queue
> placement policy is given in the allocations file, this property is ignored.*|
> |yarn.scheduler.fair.user-as-default-queue|Whether to use the username
> associated with the allocation as the default queue name, in the event that a
> queue name is not specified. If this is set to “false” or unset, all jobs
> have a shared default queue, named “default”. Defaults to true. *If a queue
> placement policy is given in the allocations file, this property is ignored.*|
> Right now these settings affects the conversion regardless of the placement
> rules.
> 2. A converted configuration throws this error:
> {noformat}
> 2020-01-27 03:35:35,007 INFO
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioned
> to standby state
> 2020-01-27 03:35:35,008 FATAL
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error starting
> ResourceManager
> java.lang.IllegalArgumentException: Illegal queue mapping
> u:%user:%user;u:%user:root.users.%user;u:%user:root.default
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.getQueueMappings(CapacitySchedulerConfiguration.java:1113)
> at
> org.apache.hadoop.yarn.server.resourcemanager.placement.UserGroupMappingPlacementRule.initialize(UserGroupMappingPlacementRule.java:244)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.getUserGroupMappingPlacementRule(CapacityScheduler.java:671)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.updatePlacementRules(CapacityScheduler.java:712)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.initializeQueues(CapacityScheduler.java:753)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.initScheduler(CapacityScheduler.java:361)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.serviceInit(CapacityScheduler.java:426)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
> {noformat}
> Mapping rules should be separated by a "," character, not by a semicolon.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
