[jira] [Updated] (YARN-3487) CapacityScheduler scheduler lock obtained unnecessarily
[ https://issues.apache.org/jira/browse/YARN-3487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-3487: - Summary: CapacityScheduler scheduler lock obtained unnecessarily (was: CapacityScheduler.getQueueInfo grabs scheduler lock unnecessarily) > CapacityScheduler scheduler lock obtained unnecessarily > --- > > Key: YARN-3487 > URL: https://issues.apache.org/jira/browse/YARN-3487 > Project: Hadoop YARN > Issue Type: Bug > Components: capacityscheduler >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Critical > Attachments: YARN-3487.001.patch > > > Recently saw a significant slowdown of applications on a large cluster, and > we noticed there were a large number of blocked threads on the RM. Most of > the blocked threads were waiting for the CapacityScheduler lock while calling > getQueueInfo. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3487) CapacityScheduler scheduler lock obtained unnecessarily
[ https://issues.apache.org/jira/browse/YARN-3487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-3487: - Attachment: YARN-3487.002.patch We're also grabbing the scheduler lock in getQueue which just looks up the queue from the concurrent hash map. In addition checkAccess is locking to lookup the queue then invoke a method on the queue. Updated the patch to remove the lock on those two methods as well. > CapacityScheduler scheduler lock obtained unnecessarily > --- > > Key: YARN-3487 > URL: https://issues.apache.org/jira/browse/YARN-3487 > Project: Hadoop YARN > Issue Type: Bug > Components: capacityscheduler >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Critical > Attachments: YARN-3487.001.patch, YARN-3487.002.patch > > > Recently saw a significant slowdown of applications on a large cluster, and > we noticed there were a large number of blocked threads on the RM. Most of > the blocked threads were waiting for the CapacityScheduler lock while calling > getQueueInfo. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3487) CapacityScheduler scheduler lock obtained unnecessarily
[ https://issues.apache.org/jira/browse/YARN-3487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-3487: - Attachment: YARN-3487.003.patch Thanks for the feedback, Wangda and Sunil. In the interest of keeping this JIRA simple to expedite the getQueueInfo and getQueue fix this version of the patch restores the lock on checkAccess. IIRC there's already another JIRA proposing to add read/write locks to the CapacityScheduler to handle rare events like queue config refresh. > CapacityScheduler scheduler lock obtained unnecessarily > --- > > Key: YARN-3487 > URL: https://issues.apache.org/jira/browse/YARN-3487 > Project: Hadoop YARN > Issue Type: Bug > Components: capacityscheduler >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Critical > Attachments: YARN-3487.001.patch, YARN-3487.002.patch, > YARN-3487.003.patch > > > Recently saw a significant slowdown of applications on a large cluster, and > we noticed there were a large number of blocked threads on the RM. Most of > the blocked threads were waiting for the CapacityScheduler lock while calling > getQueueInfo. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3487) CapacityScheduler scheduler lock obtained unnecessarily
[ https://issues.apache.org/jira/browse/YARN-3487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-3487: - Issue Type: Sub-task (was: Bug) Parent: YARN-3091 > CapacityScheduler scheduler lock obtained unnecessarily > --- > > Key: YARN-3487 > URL: https://issues.apache.org/jira/browse/YARN-3487 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Critical > Attachments: YARN-3487.001.patch, YARN-3487.002.patch, > YARN-3487.003.patch > > > Recently saw a significant slowdown of applications on a large cluster, and > we noticed there were a large number of blocked threads on the RM. Most of > the blocked threads were waiting for the CapacityScheduler lock while calling > getQueueInfo. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3487) CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue
[ https://issues.apache.org/jira/browse/YARN-3487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-3487: - Labels: 2.6.1-candidate (was: ) > CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue > - > > Key: YARN-3487 > URL: https://issues.apache.org/jira/browse/YARN-3487 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Critical > Labels: 2.6.1-candidate > Fix For: 2.7.1 > > Attachments: YARN-3487.001.patch, YARN-3487.002.patch, > YARN-3487.003.patch > > > Recently saw a significant slowdown of applications on a large cluster, and > we noticed there were a large number of blocked threads on the RM. Most of > the blocked threads were waiting for the CapacityScheduler lock while calling > getQueueInfo. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3487) CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue
[ https://issues.apache.org/jira/browse/YARN-3487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-3487: - Labels: (was: 2.6.1-candidate) > CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue > - > > Key: YARN-3487 > URL: https://issues.apache.org/jira/browse/YARN-3487 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Critical > Fix For: 2.7.1 > > Attachments: YARN-3487.001.patch, YARN-3487.002.patch, > YARN-3487.003.patch > > > Recently saw a significant slowdown of applications on a large cluster, and > we noticed there were a large number of blocked threads on the RM. Most of > the blocked threads were waiting for the CapacityScheduler lock while calling > getQueueInfo. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3487) CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue
[ https://issues.apache.org/jira/browse/YARN-3487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated YARN-3487: -- Labels: 2.6.1-candidate (was: ) > CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue > - > > Key: YARN-3487 > URL: https://issues.apache.org/jira/browse/YARN-3487 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Critical > Labels: 2.6.1-candidate > Fix For: 2.7.1 > > Attachments: YARN-3487.001.patch, YARN-3487.002.patch, > YARN-3487.003.patch > > > Recently saw a significant slowdown of applications on a large cluster, and > we noticed there were a large number of blocked threads on the RM. Most of > the blocked threads were waiting for the CapacityScheduler lock while calling > getQueueInfo. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3487) CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue
[ https://issues.apache.org/jira/browse/YARN-3487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated YARN-3487: -- Fix Version/s: 2.6.1 Pulled this into 2.6.1. Ran compilation before the push. Patch applied cleanly. > CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue > - > > Key: YARN-3487 > URL: https://issues.apache.org/jira/browse/YARN-3487 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Critical > Labels: 2.6.1-candidate > Fix For: 2.6.1, 2.7.1 > > Attachments: YARN-3487.001.patch, YARN-3487.002.patch, > YARN-3487.003.patch > > > Recently saw a significant slowdown of applications on a large cluster, and > we noticed there were a large number of blocked threads on the RM. Most of > the blocked threads were waiting for the CapacityScheduler lock while calling > getQueueInfo. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3487) CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue
[ https://issues.apache.org/jira/browse/YARN-3487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3487: - Fix Version/s: 2.8.0 > CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue > - > > Key: YARN-3487 > URL: https://issues.apache.org/jira/browse/YARN-3487 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Critical > Labels: 2.6.1-candidate > Fix For: 2.6.1, 2.8.0, 2.7.1, 3.0.0-alpha1 > > Attachments: YARN-3487.001.patch, YARN-3487.002.patch, > YARN-3487.003.patch > > > Recently saw a significant slowdown of applications on a large cluster, and > we noticed there were a large number of blocked threads on the RM. Most of > the blocked threads were waiting for the CapacityScheduler lock while calling > getQueueInfo. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-3487) CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue
[ https://issues.apache.org/jira/browse/YARN-3487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-3487: - Summary: CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue (was: CapacityScheduler scheduler lock obtained unnecessarily) > CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue > - > > Key: YARN-3487 > URL: https://issues.apache.org/jira/browse/YARN-3487 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Critical > Attachments: YARN-3487.001.patch, YARN-3487.002.patch, > YARN-3487.003.patch > > > Recently saw a significant slowdown of applications on a large cluster, and > we noticed there were a large number of blocked threads on the RM. Most of > the blocked threads were waiting for the CapacityScheduler lock while calling > getQueueInfo. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
