[
https://issues.apache.org/jira/browse/YARN-10287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prabhu Joseph updated YARN-10287:
---------------------------------
Attachment: YARN-10287-001.patch
> Update scheduler-conf corrupts the CS configuration when removing queue which
> is referred in queue mapping
> ----------------------------------------------------------------------------------------------------------
>
> Key: YARN-10287
> URL: https://issues.apache.org/jira/browse/YARN-10287
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
> Affects Versions: 3.3.0
> Reporter: Akhil PB
> Assignee: Prabhu Joseph
> Priority: Major
> Attachments: YARN-10287-001.patch
>
>
> Update scheduler-conf corrupts the CS configuration when removing queue which
> is referred in queue mapping. The deletion is failed with below error
> message but the queue got removed from CS configuration and job submission
> failed and not removed from the backend ZKConfigurationStore. On subsequent
> modify using scheduler-conf, the queue appears again from ZKConfigurationStore
> {code}
> 2020-05-22 12:38:38,252 ERROR
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices: Exception
> thrown when modifying configuration.
> java.io.IOException: Failed to re-init queues : mapping contains invalid or
> non-leaf queue Prod
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:478)
> at
> org.apache.hadoop.yarn.server.resourcemanager.AdminService.refreshQueues(AdminService.java:430)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2389)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2377)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices.updateSchedulerConfiguration(RMWebServices.java:2377)
> {code}
> *Repro:*
> {code}
> 1. Setup Queue Mapping
> yarn.scheduler.capacity.root.queues=default,dummy
> yarn.scheduler.capacity.queue-mappings=g:hadoop:dummy
> 2. Stop the root.dummy queue
> <update-queue>
> <queue-name>root.dummy</queue-name>
> <params>
> <entry>
> <key>state</key>
> <value>STOPPED</value>
> </entry>
> </params>
> </update-queue>
>
>
> 3. Delete the root.dummy queue
> curl --negotiate -u : -X PUT -d @abc.xml -H "Content-type: application/xml"
> 'http://<RM_IP>:8088/ws/v1/cluster/scheduler-conf?user.name=yarn'
> <sched-conf>
> <update-queue>
> <queue-name>root.default</queue-name>
> <params>
> <entry>
> <key>capacity</key>
> <value>100</value>
> </entry>
> </params>
> </update-queue>
> <remove-queue>root.dummy</remove-queue>
> </sched-conf>
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
