[jira] [Comment Edited] (YARN-8858) CapacityScheduler should respect maximum node resource when per-queue maximum-allocation is being used.
[ https://issues.apache.org/jira/browse/YARN-8858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16675306#comment-16675306 ] Weiwei Yang edited comment on YARN-8858 at 11/5/18 3:45 PM: Thanks [~ajisakaa] for the reminder, this one got slipped from my list. I just uploaded v2 patch for branch-2.8 to fix the UT. BTW, when I fix this, I think testContainerRejectionWhenAskBeyondDynamicMax might be flaky, I will create another ticket to fix on rest of branches. was (Author: cheersyang): Thanks [~ajisakaa] for the reminder, this one got slipped from my list. I just uploaded v2 patch for branch-2.8 to fix the UT. > CapacityScheduler should respect maximum node resource when per-queue > maximum-allocation is being used. > --- > > Key: YARN-8858 > URL: https://issues.apache.org/jira/browse/YARN-8858 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Sumana Sathish >Assignee: Wangda Tan >Priority: Major > Fix For: 2.10.0, 3.2.0, 2.9.2, 3.0.4, 3.1.2, 3.3.0 > > Attachments: YARN-8858-branch-2.8.001.patch, > YARN-8858-branch-2.8.002.patch, YARN-8858.001.patch, YARN-8858.002.patch > > > This issue happens after YARN-8720. > Before that, AMS uses scheduler.getMaximumAllocation to do the normalization. > After that, AMS uses LeafQueue.getMaximumAllocation. The scheduler one uses > nodeTracker.getMaximumAllocation, but the LeafQueue.getMaximum doesn't. > We should use the scheduler.getMaximumAllocation to cap the per-queue's > maximum-allocation every time. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-8858) CapacityScheduler should respect maximum node resource when per-queue maximum-allocation is being used.
[ https://issues.apache.org/jira/browse/YARN-8858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16644322#comment-16644322 ] Weiwei Yang edited comment on YARN-8858 at 10/10/18 1:31 AM: - +1. The UT failure should not be related. I applied the patch and tested locally, they work fine. I will commit this shortly. BTW, the checksytle issues were caused by some unused imports in the test class, I will remove them during the commit. was (Author: cheersyang): +1. The UT failure should not be related. I applied the patch and tested locally, they work fine. I will commit this shortly. > CapacityScheduler should respect maximum node resource when per-queue > maximum-allocation is being used. > --- > > Key: YARN-8858 > URL: https://issues.apache.org/jira/browse/YARN-8858 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Sumana Sathish >Assignee: Wangda Tan >Priority: Critical > Attachments: YARN-8858.001.patch, YARN-8858.002.patch > > > This issue happens after YARN-8720. > Before that, AMS uses scheduler.getMaximumAllocation to do the normalization. > After that, AMS uses LeafQueue.getMaximumAllocation. The scheduler one uses > nodeTracker.getMaximumAllocation, but the LeafQueue.getMaximum doesn't. > We should use the scheduler.getMaximumAllocation to cap the per-queue's > maximum-allocation every time. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-8858) CapacityScheduler should respect maximum node resource when per-queue maximum-allocation is being used.
[ https://issues.apache.org/jira/browse/YARN-8858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16642861#comment-16642861 ] Weiwei Yang edited comment on YARN-8858 at 10/9/18 6:25 AM: Patch looks good to me, pending on jenkins. was (Author: cheersyang): Patch looks good to me, pending on jenkins. > CapacityScheduler should respect maximum node resource when per-queue > maximum-allocation is being used. > --- > > Key: YARN-8858 > URL: https://issues.apache.org/jira/browse/YARN-8858 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Sumana Sathish >Assignee: Wangda Tan >Priority: Critical > Attachments: YARN-8858.001.patch > > > This issue happens after YARN-8720. > Before that, AMS uses scheduler.getMaximumAllocation to do the normalization. > After that, AMS uses LeafQueue.getMaximumAllocation. The scheduler one uses > nodeTracker.getMaximumAllocation, but the LeafQueue.getMaximum doesn't. > We should use the scheduler.getMaximumAllocation to cap the per-queue's > maximum-allocation every time. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-8858) CapacityScheduler should respect maximum node resource when per-queue maximum-allocation is being used.
[ https://issues.apache.org/jira/browse/YARN-8858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16642828#comment-16642828 ] Wangda Tan edited comment on YARN-8858 at 10/9/18 5:39 AM: --- [~cheersyang] / [~sunil.gov...@gmail.com], can we get this patch committed to 3.2.0? -YARN-8720- introduced minor behavior change, we should fix it if possible. was (Author: leftnoteasy): [~cheersyang] / [~sunil.gov...@gmail.com], can we get this patch committed to 3.2.0? It is a (minor) behavior change for YARN-8720. > CapacityScheduler should respect maximum node resource when per-queue > maximum-allocation is being used. > --- > > Key: YARN-8858 > URL: https://issues.apache.org/jira/browse/YARN-8858 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Sumana Sathish >Assignee: Wangda Tan >Priority: Critical > Attachments: YARN-8858.001.patch > > > This issue happens after YARN-8720. > Before that, AMS uses scheduler.getMaximumAllocation to do the normalization. > After that, AMS uses LeafQueue.getMaximumAllocation. The scheduler one uses > nodeTracker.getMaximumAllocation, but the LeafQueue.getMaximum doesn't. > We should use the scheduler.getMaximumAllocation to cap the per-queue's > maximum-allocation every time. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
