[jira] [Updated] (YARN-10287) Update scheduler-conf corrupts the CS configuration when removing queue which is referred in queue mapping
[
https://issues.apache.org/jira/browse/YARN-10287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated YARN-10287:
--
Labels: pull-request-available (was: )
> Update scheduler-conf corrupts the CS configuration when removing queue which
> is referred in queue mapping
> --
>
> Key: YARN-10287
> URL: https://issues.apache.org/jira/browse/YARN-10287
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
>Affects Versions: 3.3.0
>Reporter: Akhil PB
>Assignee: Ashutosh Gupta
>Priority: Major
> Labels: pull-request-available
> Attachments: YARN-10287-001.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Update scheduler-conf corrupts the CS configuration when removing queue which
> is referred in queue mapping. The deletion is failed with below error
> message but the queue got removed from CS configuration and job submission
> failed and not removed from the backend ZKConfigurationStore. On subsequent
> modify using scheduler-conf, the queue appears again from ZKConfigurationStore
> {code}
> 2020-05-22 12:38:38,252 ERROR
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices: Exception
> thrown when modifying configuration.
> java.io.IOException: Failed to re-init queues : mapping contains invalid or
> non-leaf queue Prod
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:478)
> at
> org.apache.hadoop.yarn.server.resourcemanager.AdminService.refreshQueues(AdminService.java:430)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2389)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2377)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices.updateSchedulerConfiguration(RMWebServices.java:2377)
> {code}
> *Repro:*
> {code}
> 1. Setup Queue Mapping
> yarn.scheduler.capacity.root.queues=default,dummy
> yarn.scheduler.capacity.queue-mappings=g:hadoop:dummy
> 2. Stop the root.dummy queue
>
>root.dummy
>
>
>state
>STOPPED
>
>
>
>
>
> 3. Delete the root.dummy queue
> curl --negotiate -u : -X PUT -d @abc.xml -H "Content-type: application/xml"
> 'http://:8088/ws/v1/cluster/scheduler-conf?user.name=yarn'
>
>
> root.default
>
>
> capacity
> 100
>
>
>
> root.dummy
>
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10287) Update scheduler-conf corrupts the CS configuration when removing queue which is referred in queue mapping
[
https://issues.apache.org/jira/browse/YARN-10287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prabhu Joseph updated YARN-10287:
-
Description:
Update scheduler-conf corrupts the CS configuration when removing queue which
is referred in queue mapping. The deletion is failed with below error message
but the queue got removed and job submission failed and not removed from the
ZKConfigurationStore. On subsequent modify using scheduler-conf, the queue
appears again from ZKConfigurationStore
{code}
2020-05-22 12:38:38,252 ERROR
org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices: Exception
thrown when modifying configuration.
java.io.IOException: Failed to re-init queues : mapping contains invalid or
non-leaf queue Prod
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:478)
at
org.apache.hadoop.yarn.server.resourcemanager.AdminService.refreshQueues(AdminService.java:430)
at
org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2389)
at
org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2377)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
at
org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices.updateSchedulerConfiguration(RMWebServices.java:2377)
{code}
*Repro:*
{code}
1. Setup Queue Mapping
yarn.scheduler.capacity.root.queues=default,dummy
yarn.scheduler.capacity.queue-mappings=g:hadoop:dummy
2. Stop the root.dummy queue
root.dummy
state
STOPPED
3. Delete the root.dummy queue
curl --negotiate -u : -X PUT -d @abc.xml -H "Content-type: application/xml"
'http://:8088/ws/v1/cluster/scheduler-conf?user.name=yarn'
root.default
capacity
100
root.dummy
{code}
was:
Update scheduler-conf corrupts the CS configuration when removing queue which
is referred in queue mapping. The deletion is failed with below error message
but the queue got removed and job submission failed but not removed from the
ZKConfigurationStore. On subsequent modify using scheduler-conf, the queue
appears again from ZKConfigurationStore
{code}
2020-05-22 12:38:38,252 ERROR
org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices: Exception
thrown when modifying configuration.
java.io.IOException: Failed to re-init queues : mapping contains invalid or
non-leaf queue Prod
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:478)
at
org.apache.hadoop.yarn.server.resourcemanager.AdminService.refreshQueues(AdminService.java:430)
at
org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2389)
at
org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2377)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
at
org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices.updateSchedulerConfiguration(RMWebServices.java:2377)
{code}
*Repro:*
{code}
1. Setup Queue Mapping
yarn.scheduler.capacity.root.queues=default,dummy
yarn.scheduler.capacity.queue-mappings=g:hadoop:dummy
2. Stop the root.dummy queue
root.dummy
state
STOPPED
3. Delete the root.dummy queue
curl --negotiate -u : -X PUT -d @abc.xml -H "Content-type: application/xml"
'http://:8088/ws/v1/cluster/scheduler-conf?user.name=yarn'
root.default
capacity
100
root.dummy
{code}
> Update scheduler-conf corrupts the CS configuration when removing queue which
> is referred in queue mapping
> --
>
> Key: YARN-10287
> URL: https://issues.apache.org/jira/browse/YARN-10287
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
>
> Update scheduler-conf corrupts the CS configuration when rem
[jira] [Updated] (YARN-10287) Update scheduler-conf corrupts the CS configuration when removing queue which is referred in queue mapping
[
https://issues.apache.org/jira/browse/YARN-10287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prabhu Joseph updated YARN-10287:
-
Description:
Update scheduler-conf corrupts the CS configuration when removing queue which
is referred in queue mapping. The deletion is failed with below error message
but the queue got removed from CS configuration and job submission failed and
not removed from the backend ZKConfigurationStore. On subsequent modify using
scheduler-conf, the queue appears again from ZKConfigurationStore
{code}
2020-05-22 12:38:38,252 ERROR
org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices: Exception
thrown when modifying configuration.
java.io.IOException: Failed to re-init queues : mapping contains invalid or
non-leaf queue Prod
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:478)
at
org.apache.hadoop.yarn.server.resourcemanager.AdminService.refreshQueues(AdminService.java:430)
at
org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2389)
at
org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2377)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
at
org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices.updateSchedulerConfiguration(RMWebServices.java:2377)
{code}
*Repro:*
{code}
1. Setup Queue Mapping
yarn.scheduler.capacity.root.queues=default,dummy
yarn.scheduler.capacity.queue-mappings=g:hadoop:dummy
2. Stop the root.dummy queue
root.dummy
state
STOPPED
3. Delete the root.dummy queue
curl --negotiate -u : -X PUT -d @abc.xml -H "Content-type: application/xml"
'http://:8088/ws/v1/cluster/scheduler-conf?user.name=yarn'
root.default
capacity
100
root.dummy
{code}
was:
Update scheduler-conf corrupts the CS configuration when removing queue which
is referred in queue mapping. The deletion is failed with below error message
but the queue got removed and job submission failed and not removed from the
ZKConfigurationStore. On subsequent modify using scheduler-conf, the queue
appears again from ZKConfigurationStore
{code}
2020-05-22 12:38:38,252 ERROR
org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices: Exception
thrown when modifying configuration.
java.io.IOException: Failed to re-init queues : mapping contains invalid or
non-leaf queue Prod
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:478)
at
org.apache.hadoop.yarn.server.resourcemanager.AdminService.refreshQueues(AdminService.java:430)
at
org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2389)
at
org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2377)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
at
org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices.updateSchedulerConfiguration(RMWebServices.java:2377)
{code}
*Repro:*
{code}
1. Setup Queue Mapping
yarn.scheduler.capacity.root.queues=default,dummy
yarn.scheduler.capacity.queue-mappings=g:hadoop:dummy
2. Stop the root.dummy queue
root.dummy
state
STOPPED
3. Delete the root.dummy queue
curl --negotiate -u : -X PUT -d @abc.xml -H "Content-type: application/xml"
'http://:8088/ws/v1/cluster/scheduler-conf?user.name=yarn'
root.default
capacity
100
root.dummy
{code}
> Update scheduler-conf corrupts the CS configuration when removing queue which
> is referred in queue mapping
> --
>
> Key: YARN-10287
> URL: https://issues.apache.org/jira/browse/YARN-10287
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
>
> Update scheduler-conf corrupts
[jira] [Updated] (YARN-10287) Update scheduler-conf corrupts the CS configuration when removing queue which is referred in queue mapping
[
https://issues.apache.org/jira/browse/YARN-10287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prabhu Joseph updated YARN-10287:
-
Reporter: Akhil PB (was: Prabhu Joseph)
> Update scheduler-conf corrupts the CS configuration when removing queue which
> is referred in queue mapping
> --
>
> Key: YARN-10287
> URL: https://issues.apache.org/jira/browse/YARN-10287
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
>Affects Versions: 3.3.0
>Reporter: Akhil PB
>Assignee: Prabhu Joseph
>Priority: Major
>
> Update scheduler-conf corrupts the CS configuration when removing queue which
> is referred in queue mapping. The deletion is failed with below error
> message but the queue got removed from CS configuration and job submission
> failed and not removed from the backend ZKConfigurationStore. On subsequent
> modify using scheduler-conf, the queue appears again from ZKConfigurationStore
> {code}
> 2020-05-22 12:38:38,252 ERROR
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices: Exception
> thrown when modifying configuration.
> java.io.IOException: Failed to re-init queues : mapping contains invalid or
> non-leaf queue Prod
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:478)
> at
> org.apache.hadoop.yarn.server.resourcemanager.AdminService.refreshQueues(AdminService.java:430)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2389)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2377)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices.updateSchedulerConfiguration(RMWebServices.java:2377)
> {code}
> *Repro:*
> {code}
> 1. Setup Queue Mapping
> yarn.scheduler.capacity.root.queues=default,dummy
> yarn.scheduler.capacity.queue-mappings=g:hadoop:dummy
> 2. Stop the root.dummy queue
>
>root.dummy
>
>
>state
>STOPPED
>
>
>
>
>
> 3. Delete the root.dummy queue
> curl --negotiate -u : -X PUT -d @abc.xml -H "Content-type: application/xml"
> 'http://:8088/ws/v1/cluster/scheduler-conf?user.name=yarn'
>
>
> root.default
>
>
> capacity
> 100
>
>
>
> root.dummy
>
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10287) Update scheduler-conf corrupts the CS configuration when removing queue which is referred in queue mapping
[
https://issues.apache.org/jira/browse/YARN-10287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prabhu Joseph updated YARN-10287:
-
Attachment: YARN-10287-001.patch
> Update scheduler-conf corrupts the CS configuration when removing queue which
> is referred in queue mapping
> --
>
> Key: YARN-10287
> URL: https://issues.apache.org/jira/browse/YARN-10287
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
>Affects Versions: 3.3.0
>Reporter: Akhil PB
>Assignee: Prabhu Joseph
>Priority: Major
> Attachments: YARN-10287-001.patch
>
>
> Update scheduler-conf corrupts the CS configuration when removing queue which
> is referred in queue mapping. The deletion is failed with below error
> message but the queue got removed from CS configuration and job submission
> failed and not removed from the backend ZKConfigurationStore. On subsequent
> modify using scheduler-conf, the queue appears again from ZKConfigurationStore
> {code}
> 2020-05-22 12:38:38,252 ERROR
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices: Exception
> thrown when modifying configuration.
> java.io.IOException: Failed to re-init queues : mapping contains invalid or
> non-leaf queue Prod
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:478)
> at
> org.apache.hadoop.yarn.server.resourcemanager.AdminService.refreshQueues(AdminService.java:430)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2389)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2377)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices.updateSchedulerConfiguration(RMWebServices.java:2377)
> {code}
> *Repro:*
> {code}
> 1. Setup Queue Mapping
> yarn.scheduler.capacity.root.queues=default,dummy
> yarn.scheduler.capacity.queue-mappings=g:hadoop:dummy
> 2. Stop the root.dummy queue
>
>root.dummy
>
>
>state
>STOPPED
>
>
>
>
>
> 3. Delete the root.dummy queue
> curl --negotiate -u : -X PUT -d @abc.xml -H "Content-type: application/xml"
> 'http://:8088/ws/v1/cluster/scheduler-conf?user.name=yarn'
>
>
> root.default
>
>
> capacity
> 100
>
>
>
> root.dummy
>
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
