[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15297535#comment-15297535 ] Naganarasimha G R commented on YARN-3362: - will close this jira once YARN-4751 is committed and closed... > Add node label usage in RM CapacityScheduler web UI > --- > > Key: YARN-3362 > URL: https://issues.apache.org/jira/browse/YARN-3362 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager, webapp >Reporter: Wangda Tan >Assignee: Naganarasimha G R > Fix For: 2.7.4 > > Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue > Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, > 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png, > AppInLabelXnoStatsInSchedPage.png, CSWithLabelsView.png, > No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29 > at 11.42.17 AM.png, YARN-3362-branch-2.7.002.patch, > YARN-3362-branch-2.7.003.patch, YARN-3362-branch-2.7.004.patch, > YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, > YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, > YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, > YARN-3362.20150512-1.patch, capacity-scheduler.xml > > > We don't have node label usage in RM CapacityScheduler web UI now, without > this, user will be hard to understand what happened to nodes have labels > assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285058#comment-15285058 ] Naganarasimha G R commented on YARN-3362: - Have committed the patch to 2.7 branch, Thanks [~eepayne], for working on 2.7 patch. Thanks [~sunilg] and [~wangda] for the review. Also we need to push YARN-4751 asap, as current patch is not complete without it > Add node label usage in RM CapacityScheduler web UI > --- > > Key: YARN-3362 > URL: https://issues.apache.org/jira/browse/YARN-3362 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager, webapp >Reporter: Wangda Tan >Assignee: Naganarasimha G R > Fix For: 2.8.0 > > Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue > Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, > 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png, > AppInLabelXnoStatsInSchedPage.png, CSWithLabelsView.png, > No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29 > at 11.42.17 AM.png, YARN-3362-branch-2.7.002.patch, > YARN-3362-branch-2.7.003.patch, YARN-3362-branch-2.7.004.patch, > YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, > YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, > YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, > YARN-3362.20150512-1.patch, capacity-scheduler.xml > > > We don't have node label usage in RM CapacityScheduler web UI now, without > this, user will be hard to understand what happened to nodes have labels > assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284075#comment-15284075 ] Naganarasimha G R commented on YARN-3362: - Hi [~eepayne], As in earlier patches findbugs and test case are not related to the patch, latest patch LGTM. committing it shortly ! > Add node label usage in RM CapacityScheduler web UI > --- > > Key: YARN-3362 > URL: https://issues.apache.org/jira/browse/YARN-3362 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager, webapp >Reporter: Wangda Tan >Assignee: Naganarasimha G R > Fix For: 2.8.0 > > Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue > Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, > 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png, > AppInLabelXnoStatsInSchedPage.png, CSWithLabelsView.png, > No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29 > at 11.42.17 AM.png, YARN-3362-branch-2.7.002.patch, > YARN-3362-branch-2.7.003.patch, YARN-3362-branch-2.7.004.patch, > YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, > YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, > YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, > YARN-3362.20150512-1.patch, capacity-scheduler.xml > > > We don't have node label usage in RM CapacityScheduler web UI now, without > this, user will be hard to understand what happened to nodes have labels > assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15282689#comment-15282689 ] Eric Payne commented on YARN-3362: -- Thanks, [~naganarasimha...@apache.org]. Have you been able to look at this latest patch? > Add node label usage in RM CapacityScheduler web UI > --- > > Key: YARN-3362 > URL: https://issues.apache.org/jira/browse/YARN-3362 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager, webapp >Reporter: Wangda Tan >Assignee: Naganarasimha G R > Fix For: 2.8.0 > > Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue > Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, > 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png, > AppInLabelXnoStatsInSchedPage.png, CSWithLabelsView.png, > No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29 > at 11.42.17 AM.png, YARN-3362-branch-2.7.002.patch, > YARN-3362-branch-2.7.003.patch, YARN-3362-branch-2.7.004.patch, > YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, > YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, > YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, > YARN-3362.20150512-1.patch, capacity-scheduler.xml > > > We don't have node label usage in RM CapacityScheduler web UI now, without > this, user will be hard to understand what happened to nodes have labels > assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[
https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15276736#comment-15276736
]
Hadoop QA commented on YARN-3362:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 14m 30s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
54s {color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 23s
{color} | {color:green} branch-2.7 passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 26s
{color} | {color:green} branch-2.7 passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s {color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 32s
{color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} branch-2.7 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 2s
{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
in branch-2.7 has 1 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s
{color} | {color:green} branch-2.7 passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 23s
{color} | {color:green} branch-2.7 passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 21s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 21s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 25s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 25s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 23s
{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
patch generated 50 new + 687 unchanged - 43 fixed = 737 total (was 730)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 31s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 3610 line(s) that end in whitespace. Use
git apply --whitespace=fix. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 1m 22s
{color} | {color:red} The patch has 497 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 21s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 49m 42s {color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK
v1.8.0_91. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 50m 14s {color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK
v1.7.0_101. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 131m 19s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_91 Failed junit tests |
hadoop.yarn.server.resourcemanager.TestAMAuthorization |
| |
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15276449#comment-15276449 ] Naganarasimha Garla commented on YARN-3362: --- Sure will review as it's also close can commit it too... > Add node label usage in RM CapacityScheduler web UI > --- > > Key: YARN-3362 > URL: https://issues.apache.org/jira/browse/YARN-3362 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager, webapp >Reporter: Wangda Tan >Assignee: Naganarasimha G R > Fix For: 2.8.0 > > Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue > Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, > 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png, > AppInLabelXnoStatsInSchedPage.png, CSWithLabelsView.png, > No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29 > at 11.42.17 AM.png, YARN-3362-branch-2.7.002.patch, > YARN-3362-branch-2.7.003.patch, YARN-3362-branch-2.7.004.patch, > YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, > YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, > YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, > YARN-3362.20150512-1.patch, capacity-scheduler.xml > > > We don't have node label usage in RM CapacityScheduler web UI now, without > this, user will be hard to understand what happened to nodes have labels > assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[
https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15273289#comment-15273289
]
Hadoop QA commented on YARN-3362:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 15m 15s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
55s {color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 24s
{color} | {color:green} branch-2.7 passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 26s
{color} | {color:green} branch-2.7 passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s {color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 32s
{color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} branch-2.7 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 1s
{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
in branch-2.7 has 1 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s
{color} | {color:green} branch-2.7 passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 23s
{color} | {color:green} branch-2.7 passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 22s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 22s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 24s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 24s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 23s
{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
patch generated 51 new + 688 unchanged - 43 fixed = 739 total (was 731)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 31s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 3261 line(s) that end in whitespace. Use
git apply --whitespace=fix. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 1m 14s
{color} | {color:red} The patch has 497 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 21s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 49m 28s {color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK
v1.8.0_91. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 50m 33s {color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK
v1.7.0_101. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 2m 20s
{color} | {color:red} Patch generated 61 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 134m 3s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_91 Failed junit tests |
hadoop.yarn.server.resourcemanager.TestClientRMTokens |
| | hadoop.yarn.server.resourcemanager.TestAMAuthorization |
|
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15271837#comment-15271837 ] Naganarasimha G R commented on YARN-3362: - Hi [~eepayne], Can you have a look at checkstyle, may be only few are rightly related to the patch ? > Add node label usage in RM CapacityScheduler web UI > --- > > Key: YARN-3362 > URL: https://issues.apache.org/jira/browse/YARN-3362 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager, webapp >Reporter: Wangda Tan >Assignee: Naganarasimha G R > Fix For: 2.8.0 > > Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue > Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, > 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png, > AppInLabelXnoStatsInSchedPage.png, CSWithLabelsView.png, > No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29 > at 11.42.17 AM.png, YARN-3362-branch-2.7.002.patch, > YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, > YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, > YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, > YARN-3362.20150512-1.patch, capacity-scheduler.xml > > > We don't have node label usage in RM CapacityScheduler web UI now, without > this, user will be hard to understand what happened to nodes have labels > assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[
https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15271792#comment-15271792
]
Hadoop QA commented on YARN-3362:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 16m 7s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
5s {color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 24s
{color} | {color:green} branch-2.7 passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 26s
{color} | {color:green} branch-2.7 passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s {color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 34s
{color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} branch-2.7 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 0s
{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
in branch-2.7 has 1 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s
{color} | {color:green} branch-2.7 passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 24s
{color} | {color:green} branch-2.7 passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 22s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 22s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 24s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 24s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 25s
{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
patch generated 70 new + 687 unchanged - 43 fixed = 757 total (was 730)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 30s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 2913 line(s) that end in whitespace. Use
git apply --whitespace=fix. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 1m 12s
{color} | {color:red} The patch has 497 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 22s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 52m 35s {color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK
v1.8.0_91. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 53m 13s {color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK
v1.7.0_95. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 2m 11s
{color} | {color:red} Patch generated 61 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 140m 55s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_91 Failed junit tests |
hadoop.yarn.server.resourcemanager.TestClientRMTokens |
| | hadoop.yarn.server.resourcemanager.TestAMAuthorization |
| JDK
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[
https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15267205#comment-15267205
]
Eric Payne commented on YARN-3362:
--
{noformat}
Processing: YARN-3362
YARN-3362 patch is being downloaded at Mon May 2 05:27:05 UTC 2016 from
https://issues.apache.org/jira/secure/attachment/12799131/AppInLabelXnoStatsInSchedPage.png
-> Downloaded
ERROR: Unsure how to process YARN-3362.
{noformat}
> Add node label usage in RM CapacityScheduler web UI
> ---
>
> Key: YARN-3362
> URL: https://issues.apache.org/jira/browse/YARN-3362
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler, resourcemanager, webapp
>Reporter: Wangda Tan
>Assignee: Naganarasimha G R
> Fix For: 2.8.0
>
> Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue
> Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png,
> 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png,
> AppInLabelXnoStatsInSchedPage.png, CSWithLabelsView.png,
> No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29
> at 11.42.17 AM.png, YARN-3362-branch-2.7.002.patch,
> YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch,
> YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch,
> YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch,
> YARN-3362.20150512-1.patch, capacity-scheduler.xml
>
>
> We don't have node label usage in RM CapacityScheduler web UI now, without
> this, user will be hard to understand what happened to nodes have labels
> assign to it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15267203#comment-15267203 ] Eric Payne commented on YARN-3362: -- Thanks a lot, [~Naganarasimha]. I don't think the YARN pre-commit build worked. I the following error message in the console output from both of the following builds: https://builds.apache.org/job/PreCommit-YARN-Build/11301/console https://builds.apache.org/job/PreCommit-YARN-Build/11304/console > Add node label usage in RM CapacityScheduler web UI > --- > > Key: YARN-3362 > URL: https://issues.apache.org/jira/browse/YARN-3362 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager, webapp >Reporter: Wangda Tan >Assignee: Naganarasimha G R > Fix For: 2.8.0 > > Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue > Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, > 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png, > AppInLabelXnoStatsInSchedPage.png, CSWithLabelsView.png, > No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29 > at 11.42.17 AM.png, YARN-3362-branch-2.7.002.patch, > YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, > YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, > YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, > YARN-3362.20150512-1.patch, capacity-scheduler.xml > > > We don't have node label usage in RM CapacityScheduler web UI now, without > this, user will be hard to understand what happened to nodes have labels > assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15266135#comment-15266135 ] Naganarasimha G R commented on YARN-3362: - manually started jenkins build as it had not yet started. > Add node label usage in RM CapacityScheduler web UI > --- > > Key: YARN-3362 > URL: https://issues.apache.org/jira/browse/YARN-3362 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager, webapp >Reporter: Wangda Tan >Assignee: Naganarasimha G R > Fix For: 2.8.0 > > Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue > Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, > 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png, > AppInLabelXnoStatsInSchedPage.png, CSWithLabelsView.png, > No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29 > at 11.42.17 AM.png, YARN-3362-branch-2.7.002.patch, > YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, > YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, > YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, > YARN-3362.20150512-1.patch, capacity-scheduler.xml > > > We don't have node label usage in RM CapacityScheduler web UI now, without > this, user will be hard to understand what happened to nodes have labels > assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15265873#comment-15265873 ] Naganarasimha G R commented on YARN-3362: - fine with me.. reopening the ticket and starting the jenkins! > Add node label usage in RM CapacityScheduler web UI > --- > > Key: YARN-3362 > URL: https://issues.apache.org/jira/browse/YARN-3362 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager, webapp >Reporter: Wangda Tan >Assignee: Naganarasimha G R > Fix For: 2.8.0 > > Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue > Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, > 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png, > AppInLabelXnoStatsInSchedPage.png, CSWithLabelsView.png, > No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29 > at 11.42.17 AM.png, YARN-3362-branch-2.7.002.patch, > YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, > YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, > YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, > YARN-3362.20150512-1.patch, capacity-scheduler.xml > > > We don't have node label usage in RM CapacityScheduler web UI now, without > this, user will be hard to understand what happened to nodes have labels > assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15265865#comment-15265865 ] Wangda Tan commented on YARN-3362: -- [~Naganarasimha], I would prefer the previous one: reopen this ticket, kick Jenkins and change status of this JIRA. YARN-4751 can be committed separately. Typically we should backport JIRA to older version with same ID, changing ID sometimes causes trouble when doing back port. Thoughts? > Add node label usage in RM CapacityScheduler web UI > --- > > Key: YARN-3362 > URL: https://issues.apache.org/jira/browse/YARN-3362 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager, webapp >Reporter: Wangda Tan >Assignee: Naganarasimha G R > Fix For: 2.8.0 > > Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue > Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, > 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png, > AppInLabelXnoStatsInSchedPage.png, CSWithLabelsView.png, > No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29 > at 11.42.17 AM.png, YARN-3362-branch-2.7.002.patch, > YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, > YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, > YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, > YARN-3362.20150512-1.patch, capacity-scheduler.xml > > > We don't have node label usage in RM CapacityScheduler web UI now, without > this, user will be hard to understand what happened to nodes have labels > assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15265773#comment-15265773 ] Naganarasimha G R commented on YARN-3362: - Sorry for the long delay, I verified the patch in the order you have mentioned, i built and crossverified the patch on 2.7 and i am fine with the modifications too. Just one query is how to go about with the merge? would it be good to reopen this jira and change the state to patch available to so that the jenkins is triggered or merge these changes with YARN-4751. I would prefer the later , thoughts ? cc / [~wangda] > Add node label usage in RM CapacityScheduler web UI > --- > > Key: YARN-3362 > URL: https://issues.apache.org/jira/browse/YARN-3362 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager, webapp >Reporter: Wangda Tan >Assignee: Naganarasimha G R > Fix For: 2.8.0 > > Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue > Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, > 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png, > AppInLabelXnoStatsInSchedPage.png, CSWithLabelsView.png, > No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29 > at 11.42.17 AM.png, YARN-3362-branch-2.7.002.patch, > YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, > YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, > YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, > YARN-3362.20150512-1.patch, capacity-scheduler.xml > > > We don't have node label usage in RM CapacityScheduler web UI now, without > this, user will be hard to understand what happened to nodes have labels > assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15262659#comment-15262659 ] Eric Payne commented on YARN-3362: -- [~naganarasimha...@apache.org], is there any update? > Add node label usage in RM CapacityScheduler web UI > --- > > Key: YARN-3362 > URL: https://issues.apache.org/jira/browse/YARN-3362 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager, webapp >Reporter: Wangda Tan >Assignee: Naganarasimha G R > Fix For: 2.8.0 > > Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue > Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, > 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png, > AppInLabelXnoStatsInSchedPage.png, CSWithLabelsView.png, > No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29 > at 11.42.17 AM.png, YARN-3362-branch-2.7.002.patch, > YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, > YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, > YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, > YARN-3362.20150512-1.patch, capacity-scheduler.xml > > > We don't have node label usage in RM CapacityScheduler web UI now, without > this, user will be hard to understand what happened to nodes have labels > assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251882#comment-15251882 ] Naganarasimha Garla commented on YARN-3362: --- Will have chk once, please wait till EOD... > Add node label usage in RM CapacityScheduler web UI > --- > > Key: YARN-3362 > URL: https://issues.apache.org/jira/browse/YARN-3362 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager, webapp >Reporter: Wangda Tan >Assignee: Naganarasimha G R > Fix For: 2.8.0 > > Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue > Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, > 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png, > AppInLabelXnoStatsInSchedPage.png, CSWithLabelsView.png, > No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29 > at 11.42.17 AM.png, YARN-3362-branch-2.7.002.patch, > YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, > YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, > YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, > YARN-3362.20150512-1.patch, capacity-scheduler.xml > > > We don't have node label usage in RM CapacityScheduler web UI now, without > this, user will be hard to understand what happened to nodes have labels > assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[
https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251820#comment-15251820
]
Eric Payne commented on YARN-3362:
--
{quote}
bq. If you think it would be cleaner, I can combine the patches and submit it
as one patch here in YARN-3362.
I think its looks fine for me.
{quote}
Thanks, [~sunilg], for your comments. [~Naganarasimha], given that we will keep
the patches separate, do you want to comment further on the 2.7 patch for
YARN-3362?
> Add node label usage in RM CapacityScheduler web UI
> ---
>
> Key: YARN-3362
> URL: https://issues.apache.org/jira/browse/YARN-3362
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler, resourcemanager, webapp
>Reporter: Wangda Tan
>Assignee: Naganarasimha G R
> Fix For: 2.8.0
>
> Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue
> Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png,
> 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png,
> AppInLabelXnoStatsInSchedPage.png, CSWithLabelsView.png,
> No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29
> at 11.42.17 AM.png, YARN-3362-branch-2.7.002.patch,
> YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch,
> YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch,
> YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch,
> YARN-3362.20150512-1.patch, capacity-scheduler.xml
>
>
> We don't have node label usage in RM CapacityScheduler web UI now, without
> this, user will be hard to understand what happened to nodes have labels
> assign to it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[
https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15246141#comment-15246141
]
Sunil G commented on YARN-3362:
---
I tested with both patches, and metrics was coming fine (tested w/o labels and
with labels). But when I uploaded only YARN-3362 patch, then same pblm came as
[~Naganarasimha Garla] has mentioned.
In YARN-4751, {{AbstractCSQueue}} changes helps to display queue Used Capacity
correctly. I think that was the differentiated change for this.
bq.If you think it would be cleaner, I can combine the patches and submit it as
one patch here in YARN-3362.
I think its looks fine for me.
> Add node label usage in RM CapacityScheduler web UI
> ---
>
> Key: YARN-3362
> URL: https://issues.apache.org/jira/browse/YARN-3362
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler, resourcemanager, webapp
>Reporter: Wangda Tan
>Assignee: Naganarasimha G R
> Fix For: 2.8.0
>
> Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue
> Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png,
> 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png,
> AppInLabelXnoStatsInSchedPage.png, CSWithLabelsView.png,
> No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29
> at 11.42.17 AM.png, YARN-3362-branch-2.7.002.patch,
> YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch,
> YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch,
> YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch,
> YARN-3362.20150512-1.patch, capacity-scheduler.xml
>
>
> We don't have node label usage in RM CapacityScheduler web UI now, without
> this, user will be hard to understand what happened to nodes have labels
> assign to it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15245787#comment-15245787 ] Naganarasimha G R commented on YARN-3362: - Oops let me apply both and test i had applied only the current patch(YARN-3362-branch-2.7.002.patch ). Thanks for the update! > Add node label usage in RM CapacityScheduler web UI > --- > > Key: YARN-3362 > URL: https://issues.apache.org/jira/browse/YARN-3362 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager, webapp >Reporter: Wangda Tan >Assignee: Naganarasimha G R > Fix For: 2.8.0 > > Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue > Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, > 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png, > AppInLabelXnoStatsInSchedPage.png, CSWithLabelsView.png, > No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29 > at 11.42.17 AM.png, YARN-3362-branch-2.7.002.patch, > YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, > YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, > YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, > YARN-3362.20150512-1.patch, capacity-scheduler.xml > > > We don't have node label usage in RM CapacityScheduler web UI now, without > this, user will be hard to understand what happened to nodes have labels > assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[
https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15245728#comment-15245728
]
Eric Payne commented on YARN-3362:
--
{quote}
After applying the patch then partition usage information needs to be displayed
for the queue under the partition. But after applying the patch i was not able
to see the stats in the webui even though the application is submitted to
partition and it is running
{quote}
[~Naganarasimha], in order to see the stats and bar graph metrics, both
[YARN-3362-branch-2.7.002.patch|https://issues.apache.org/jira/secure/attachment/12798831/YARN-3362-branch-2.7.002.patch]
and
[YARN-4751-branch-2.7.004.patch|https://issues.apache.org/jira/secure/attachment/12798877/YARN-4751-branch-2.7.004.patch]
need to be applied. First YARN-3362-branch-2.7.002.patch should be applied and
then YARN-4751-branch-2.7.004.patch. Did you do that and are still seeing
problems?
If you think it would be cleaner, I can combine the patches and submit it as
one patch here in YARN-3362.
> Add node label usage in RM CapacityScheduler web UI
> ---
>
> Key: YARN-3362
> URL: https://issues.apache.org/jira/browse/YARN-3362
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler, resourcemanager, webapp
>Reporter: Wangda Tan
>Assignee: Naganarasimha G R
> Fix For: 2.8.0
>
> Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue
> Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png,
> 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png,
> AppInLabelXnoStatsInSchedPage.png, CSWithLabelsView.png,
> No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29
> at 11.42.17 AM.png, YARN-3362-branch-2.7.002.patch,
> YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch,
> YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch,
> YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch,
> YARN-3362.20150512-1.patch, capacity-scheduler.xml
>
>
> We don't have node label usage in RM CapacityScheduler web UI now, without
> this, user will be hard to understand what happened to nodes have labels
> assign to it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15245669#comment-15245669 ] Naganarasimha G R commented on YARN-3362: - Hi [~eepayne], Thanks for providing the back ground, i went through the comments which you mentioned but still not able to understand whether the patch what you have shared would work independently ? After applying the patch then partition usage information needs to be displayed for the queue under the partition. But after applying the patch i was not able to see the stats in the webui even though the application is submitted to partition and it is running refer *AppInLabelXnoStatsInSchedPage.png*. And if i submit to default parition then webui stats are proper. If another patch is required to get the webui working, may be you can share the patch then i can review. Also dint have time to cross check why it was not working if required can do the analysis. And for other jira's its good to have and not compulsary > Add node label usage in RM CapacityScheduler web UI > --- > > Key: YARN-3362 > URL: https://issues.apache.org/jira/browse/YARN-3362 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager, webapp >Reporter: Wangda Tan >Assignee: Naganarasimha G R > Fix For: 2.8.0 > > Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue > Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, > 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png, > AppInLabelXnoStatsInSchedPage.png, CSWithLabelsView.png, > No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29 > at 11.42.17 AM.png, YARN-3362-branch-2.7.002.patch, > YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, > YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, > YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, > YARN-3362.20150512-1.patch, capacity-scheduler.xml > > > We don't have node label usage in RM CapacityScheduler web UI now, without > this, user will be hard to understand what happened to nodes have labels > assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[
https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15245610#comment-15245610
]
Eric Payne commented on YARN-3362:
--
{quote}
- When i submit a app to a queue with default node label expression set to a
partition then in the scheduler page stats was not getting displayed.
Other issues which are good to be backported :
# i think there is one other jira which was capturing the queue's default node
label expression that would also be helpful to be back ported
# App's AM's label is also good to be captured in the web ui and displayed.
# would it be good to back port MAPREDUCE-6304 ?
{quote}
[~Naganarasimha], thank you very much for your review and comments.
I'm sorry I did not make it clear in this JIRA. Please refer to conversations
in YARN-4751 between me, [~leftnoteasy], and [~sunilg].
The problem is that in order to provide all of the needed functionality in 2.7,
we either have to backport several JIRAs which would destabilize 2.7 (please
see [this
comment|https://issues.apache.org/jira/browse/YARN-4751?focusedCommentId=15218335=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15218335]
and [this
comment|https://issues.apache.org/jira/browse/YARN-4751?focusedCommentId=15219358=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15219358])
or we need to pick and choose pieces to backport without doing full
cherry-picks.
After the discussion in YARN-4751, we thought it would be best to submit a 2.7
patch here that only backported YARN-3362 and then add the fixes for the
metrics in YARN-4751. Please let me know what you think.
> Add node label usage in RM CapacityScheduler web UI
> ---
>
> Key: YARN-3362
> URL: https://issues.apache.org/jira/browse/YARN-3362
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler, resourcemanager, webapp
>Reporter: Wangda Tan
>Assignee: Naganarasimha G R
> Fix For: 2.8.0
>
> Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue
> Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png,
> 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png,
> AppInLabelXnoStatsInSchedPage.png, CSWithLabelsView.png,
> No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29
> at 11.42.17 AM.png, YARN-3362-branch-2.7.002.patch,
> YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch,
> YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch,
> YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch,
> YARN-3362.20150512-1.patch, capacity-scheduler.xml
>
>
> We don't have node label usage in RM CapacityScheduler web UI now, without
> this, user will be hard to understand what happened to nodes have labels
> assign to it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15244652#comment-15244652 ] Naganarasimha G R commented on YARN-3362: - attached AppInLabelXnoStatsInSchedPage.png for the same tried submitting to default queue and other hierarchical queue both had the same behavior > Add node label usage in RM CapacityScheduler web UI > --- > > Key: YARN-3362 > URL: https://issues.apache.org/jira/browse/YARN-3362 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager, webapp >Reporter: Wangda Tan >Assignee: Naganarasimha G R > Fix For: 2.8.0 > > Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue > Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, > 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png, > AppInLabelXnoStatsInSchedPage.png, CSWithLabelsView.png, > No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29 > at 11.42.17 AM.png, YARN-3362-branch-2.7.002.patch, > YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, > YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, > YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, > YARN-3362.20150512-1.patch, capacity-scheduler.xml > > > We don't have node label usage in RM CapacityScheduler web UI now, without > this, user will be hard to understand what happened to nodes have labels > assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15244649#comment-15244649 ] Naganarasimha G R commented on YARN-3362: - Hi [~eepayne], i was trying to compile, install and test, one major blocker was : * When i submit a app to a queue with default node label expression set to a partition then in the scheduler page stats was not getting displayed. Other issues which are good to be backported : # i think there is one other jira which was capturing the queue's default node label expression that would also be helpful to be back ported # App's AM's label is also good to be captured in the web ui and displayed. # would it be good to back port MAPREDUCE-6304 ? > Add node label usage in RM CapacityScheduler web UI > --- > > Key: YARN-3362 > URL: https://issues.apache.org/jira/browse/YARN-3362 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager, webapp >Reporter: Wangda Tan >Assignee: Naganarasimha G R > Fix For: 2.8.0 > > Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue > Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, > 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png, > CSWithLabelsView.png, No-space-between-Active_user_info-and-next-queues.png, > Screen Shot 2015-04-29 at 11.42.17 AM.png, YARN-3362-branch-2.7.002.patch, > YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, > YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, > YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, > YARN-3362.20150512-1.patch, capacity-scheduler.xml > > > We don't have node label usage in RM CapacityScheduler web UI now, without > this, user will be hard to understand what happened to nodes have labels > assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15243363#comment-15243363 ] Naganarasimha G R commented on YARN-3362: - Thanks [~eepayne], [~wangda], & [~sunilg] for waiting, taking a look at it shortly ... > Add node label usage in RM CapacityScheduler web UI > --- > > Key: YARN-3362 > URL: https://issues.apache.org/jira/browse/YARN-3362 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager, webapp >Reporter: Wangda Tan >Assignee: Naganarasimha G R > Fix For: 2.8.0 > > Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue > Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, > 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png, > CSWithLabelsView.png, No-space-between-Active_user_info-and-next-queues.png, > Screen Shot 2015-04-29 at 11.42.17 AM.png, YARN-3362-branch-2.7.002.patch, > YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, > YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, > YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, > YARN-3362.20150512-1.patch, capacity-scheduler.xml > > > We don't have node label usage in RM CapacityScheduler web UI now, without > this, user will be hard to understand what happened to nodes have labels > assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15243214#comment-15243214 ] Sunil G commented on YARN-3362: --- Thanks [~eepayne] for sharing patch here. Very much appreciate the same. Overall patch looks fine for me. Will wait for [~Naganarasimha Garla] also. > Add node label usage in RM CapacityScheduler web UI > --- > > Key: YARN-3362 > URL: https://issues.apache.org/jira/browse/YARN-3362 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager, webapp >Reporter: Wangda Tan >Assignee: Naganarasimha G R > Fix For: 2.8.0 > > Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue > Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, > 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png, > CSWithLabelsView.png, No-space-between-Active_user_info-and-next-queues.png, > Screen Shot 2015-04-29 at 11.42.17 AM.png, YARN-3362-branch-2.7.002.patch, > YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, > YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, > YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, > YARN-3362.20150512-1.patch, capacity-scheduler.xml > > > We don't have node label usage in RM CapacityScheduler web UI now, without > this, user will be hard to understand what happened to nodes have labels > assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242069#comment-15242069 ] Wangda Tan commented on YARN-3362: -- + [~Naganarasimha]. > Add node label usage in RM CapacityScheduler web UI > --- > > Key: YARN-3362 > URL: https://issues.apache.org/jira/browse/YARN-3362 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager, webapp >Reporter: Wangda Tan >Assignee: Naganarasimha G R > Fix For: 2.8.0 > > Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue > Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, > 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png, > CSWithLabelsView.png, No-space-between-Active_user_info-and-next-queues.png, > Screen Shot 2015-04-29 at 11.42.17 AM.png, YARN-3362-branch-2.7.002.patch, > YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, > YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, > YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, > YARN-3362.20150512-1.patch, capacity-scheduler.xml > > > We don't have node label usage in RM CapacityScheduler web UI now, without > this, user will be hard to understand what happened to nodes have labels > assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543514#comment-14543514 ] Hudson commented on YARN-3362: -- FAILURE: Integrated in Hadoop-Yarn-trunk #927 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/927/]) YARN-3362. Add node label usage in RM CapacityScheduler web UI. (Naganarasimha G R via wangda) (wangda: rev 0e85044e26da698c45185585310ae0e99448cd80) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/CapacitySchedulerPage.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerQueueInfo.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerInfo.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestCapacityScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerLeafQueueInfo.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java Add node label usage in RM CapacityScheduler web UI --- Key: YARN-3362 URL: https://issues.apache.org/jira/browse/YARN-3362 Project: Hadoop YARN Issue Type: Sub-task Components: capacityscheduler, resourcemanager, webapp Reporter: Wangda Tan Assignee: Naganarasimha G R Fix For: 2.8.0 Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png, CSWithLabelsView.png, No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29 at 11.42.17 AM.png, YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, YARN-3362.20150512-1.patch, capacity-scheduler.xml We don't have node label usage in RM CapacityScheduler web UI now, without this, user will be hard to understand what happened to nodes have labels assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543570#comment-14543570 ] Hudson commented on YARN-3362: -- SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #196 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/196/]) YARN-3362. Add node label usage in RM CapacityScheduler web UI. (Naganarasimha G R via wangda) (wangda: rev 0e85044e26da698c45185585310ae0e99448cd80) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/CapacitySchedulerPage.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerQueueInfo.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestCapacityScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerInfo.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerLeafQueueInfo.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java Add node label usage in RM CapacityScheduler web UI --- Key: YARN-3362 URL: https://issues.apache.org/jira/browse/YARN-3362 Project: Hadoop YARN Issue Type: Sub-task Components: capacityscheduler, resourcemanager, webapp Reporter: Wangda Tan Assignee: Naganarasimha G R Fix For: 2.8.0 Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png, CSWithLabelsView.png, No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29 at 11.42.17 AM.png, YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, YARN-3362.20150512-1.patch, capacity-scheduler.xml We don't have node label usage in RM CapacityScheduler web UI now, without this, user will be hard to understand what happened to nodes have labels assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543790#comment-14543790 ] Hudson commented on YARN-3362: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #195 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/195/]) YARN-3362. Add node label usage in RM CapacityScheduler web UI. (Naganarasimha G R via wangda) (wangda: rev 0e85044e26da698c45185585310ae0e99448cd80) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerLeafQueueInfo.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerInfo.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerQueueInfo.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestCapacityScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/CapacitySchedulerPage.java Add node label usage in RM CapacityScheduler web UI --- Key: YARN-3362 URL: https://issues.apache.org/jira/browse/YARN-3362 Project: Hadoop YARN Issue Type: Sub-task Components: capacityscheduler, resourcemanager, webapp Reporter: Wangda Tan Assignee: Naganarasimha G R Fix For: 2.8.0 Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png, CSWithLabelsView.png, No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29 at 11.42.17 AM.png, YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, YARN-3362.20150512-1.patch, capacity-scheduler.xml We don't have node label usage in RM CapacityScheduler web UI now, without this, user will be hard to understand what happened to nodes have labels assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543845#comment-14543845 ] Hudson commented on YARN-3362: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #185 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/185/]) YARN-3362. Add node label usage in RM CapacityScheduler web UI. (Naganarasimha G R via wangda) (wangda: rev 0e85044e26da698c45185585310ae0e99448cd80) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerInfo.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestCapacityScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerLeafQueueInfo.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerQueueInfo.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/CapacitySchedulerPage.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java Add node label usage in RM CapacityScheduler web UI --- Key: YARN-3362 URL: https://issues.apache.org/jira/browse/YARN-3362 Project: Hadoop YARN Issue Type: Sub-task Components: capacityscheduler, resourcemanager, webapp Reporter: Wangda Tan Assignee: Naganarasimha G R Fix For: 2.8.0 Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png, CSWithLabelsView.png, No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29 at 11.42.17 AM.png, YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, YARN-3362.20150512-1.patch, capacity-scheduler.xml We don't have node label usage in RM CapacityScheduler web UI now, without this, user will be hard to understand what happened to nodes have labels assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543807#comment-14543807 ] Hudson commented on YARN-3362: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2125 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2125/]) YARN-3362. Add node label usage in RM CapacityScheduler web UI. (Naganarasimha G R via wangda) (wangda: rev 0e85044e26da698c45185585310ae0e99448cd80) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerLeafQueueInfo.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/CapacitySchedulerPage.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestCapacityScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerInfo.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerQueueInfo.java Add node label usage in RM CapacityScheduler web UI --- Key: YARN-3362 URL: https://issues.apache.org/jira/browse/YARN-3362 Project: Hadoop YARN Issue Type: Sub-task Components: capacityscheduler, resourcemanager, webapp Reporter: Wangda Tan Assignee: Naganarasimha G R Fix For: 2.8.0 Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png, CSWithLabelsView.png, No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29 at 11.42.17 AM.png, YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, YARN-3362.20150512-1.patch, capacity-scheduler.xml We don't have node label usage in RM CapacityScheduler web UI now, without this, user will be hard to understand what happened to nodes have labels assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543895#comment-14543895 ] Hudson commented on YARN-3362: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2143 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2143/]) YARN-3362. Add node label usage in RM CapacityScheduler web UI. (Naganarasimha G R via wangda) (wangda: rev 0e85044e26da698c45185585310ae0e99448cd80) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestCapacityScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/CapacitySchedulerPage.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerQueueInfo.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerInfo.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerLeafQueueInfo.java Add node label usage in RM CapacityScheduler web UI --- Key: YARN-3362 URL: https://issues.apache.org/jira/browse/YARN-3362 Project: Hadoop YARN Issue Type: Sub-task Components: capacityscheduler, resourcemanager, webapp Reporter: Wangda Tan Assignee: Naganarasimha G R Fix For: 2.8.0 Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png, CSWithLabelsView.png, No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29 at 11.42.17 AM.png, YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, YARN-3362.20150512-1.patch, capacity-scheduler.xml We don't have node label usage in RM CapacityScheduler web UI now, without this, user will be hard to understand what happened to nodes have labels assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542570#comment-14542570 ] Wangda Tan commented on YARN-3362: -- [~Naganarasimha], Thanks for pointing YARN-3638, I think what you want can be addressed or at least more related YARN-3638, right? I suggest to move discussion to YARN-3638 to make this JIRA more concentrated. Add node label usage in RM CapacityScheduler web UI --- Key: YARN-3362 URL: https://issues.apache.org/jira/browse/YARN-3362 Project: Hadoop YARN Issue Type: Sub-task Components: capacityscheduler, resourcemanager, webapp Reporter: Wangda Tan Assignee: Naganarasimha G R Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png, CSWithLabelsView.png, No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29 at 11.42.17 AM.png, YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, YARN-3362.20150512-1.patch, capacity-scheduler.xml We don't have node label usage in RM CapacityScheduler web UI now, without this, user will be hard to understand what happened to nodes have labels assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542574#comment-14542574 ] Wangda Tan commented on YARN-3362: -- Last call for comments, as I'm planning to commit today. Add node label usage in RM CapacityScheduler web UI --- Key: YARN-3362 URL: https://issues.apache.org/jira/browse/YARN-3362 Project: Hadoop YARN Issue Type: Sub-task Components: capacityscheduler, resourcemanager, webapp Reporter: Wangda Tan Assignee: Naganarasimha G R Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png, CSWithLabelsView.png, No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29 at 11.42.17 AM.png, YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, YARN-3362.20150512-1.patch, capacity-scheduler.xml We don't have node label usage in RM CapacityScheduler web UI now, without this, user will be hard to understand what happened to nodes have labels assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542962#comment-14542962 ] Hudson commented on YARN-3362: -- FAILURE: Integrated in Hadoop-trunk-Commit #7825 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/7825/]) YARN-3362. Add node label usage in RM CapacityScheduler web UI. (Naganarasimha G R via wangda) (wangda: rev 0e85044e26da698c45185585310ae0e99448cd80) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/CapacitySchedulerPage.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerQueueInfo.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestCapacityScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerLeafQueueInfo.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerInfo.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java Add node label usage in RM CapacityScheduler web UI --- Key: YARN-3362 URL: https://issues.apache.org/jira/browse/YARN-3362 Project: Hadoop YARN Issue Type: Sub-task Components: capacityscheduler, resourcemanager, webapp Reporter: Wangda Tan Assignee: Naganarasimha G R Fix For: 2.8.0 Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png, CSWithLabelsView.png, No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29 at 11.42.17 AM.png, YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, YARN-3362.20150512-1.patch, capacity-scheduler.xml We don't have node label usage in RM CapacityScheduler web UI now, without this, user will be hard to understand what happened to nodes have labels assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542151#comment-14542151 ] Naganarasimha G R commented on YARN-3362: - Thanks for the comment [~Wangda], This idea is also good but i can think of few issues here * In most cases Queue hierarchy will be collapsed so the space between non leaf queue name and the bar might not look good * Bigger the name of the leaf queues more space between the queue name and the bar (at the upper level queues) * Space based calculation might not work out so implementing it with Hammock might be little complex. * Here were you thinking of publishing absolute usage or usage @ queue level, I feel absolute usage would be better (YARN-3638 also suggest it), but existing shows just usage @ queue level. Also i feel these modifications can take time, if this jira is important then we can push this in and we can further discuss about UI enhancements in another jira. thoughts? Add node label usage in RM CapacityScheduler web UI --- Key: YARN-3362 URL: https://issues.apache.org/jira/browse/YARN-3362 Project: Hadoop YARN Issue Type: Sub-task Components: capacityscheduler, resourcemanager, webapp Reporter: Wangda Tan Assignee: Naganarasimha G R Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png, CSWithLabelsView.png, No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29 at 11.42.17 AM.png, YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, YARN-3362.20150512-1.patch, capacity-scheduler.xml We don't have node label usage in RM CapacityScheduler web UI now, without this, user will be hard to understand what happened to nodes have labels assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543009#comment-14543009 ] Naganarasimha G R commented on YARN-3362: - Thanks for reviewing and committing the jira, [~Wangda], And Yes it would be better to discuss these CS queue hierarchy modifications in YARN-3638, will update our discussions there. Add node label usage in RM CapacityScheduler web UI --- Key: YARN-3362 URL: https://issues.apache.org/jira/browse/YARN-3362 Project: Hadoop YARN Issue Type: Sub-task Components: capacityscheduler, resourcemanager, webapp Reporter: Wangda Tan Assignee: Naganarasimha G R Fix For: 2.8.0 Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, 2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png, CSWithLabelsView.png, No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29 at 11.42.17 AM.png, YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, YARN-3362.20150512-1.patch, capacity-scheduler.xml We don't have node label usage in RM CapacityScheduler web UI now, without this, user will be hard to understand what happened to nodes have labels assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[
https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14539483#comment-14539483
]
Hadoop QA commented on YARN-3362:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 47s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 50s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 6s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 30s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 2s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 35s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 16s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:red}-1{color} | yarn tests | 51m 59s | Tests failed in
hadoop-yarn-server-resourcemanager. |
| | | 89m 6s | |
\\
\\
|| Reason || Tests ||
| Timed out tests |
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerQueueACLs
|
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12732150/YARN-3362.20150512-1.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 987abc9 |
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7878/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/7878/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7878/console |
This message was automatically generated.
Add node label usage in RM CapacityScheduler web UI
---
Key: YARN-3362
URL: https://issues.apache.org/jira/browse/YARN-3362
Project: Hadoop YARN
Issue Type: Sub-task
Components: capacityscheduler, resourcemanager, webapp
Reporter: Wangda Tan
Assignee: Naganarasimha G R
Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue
Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png,
2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png,
CSWithLabelsView.png, No-space-between-Active_user_info-and-next-queues.png,
Screen Shot 2015-04-29 at 11.42.17 AM.png,
YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch,
YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch,
YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch,
YARN-3362.20150512-1.patch, capacity-scheduler.xml
We don't have node label usage in RM CapacityScheduler web UI now, without
this, user will be hard to understand what happened to nodes have labels
assign to it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[
https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14540756#comment-14540756
]
Wangda Tan commented on YARN-3362:
--
[~Naganarasimha], thanks for updating, the latest result looks great!
About the queue-hierarchy discussion, I think maybe one alternative is, make a
hierarchy queue names, but align usage bars, like following:
{code}
root[-- 100% used]
- a [- 60% used]
- a1[- 40% used]
- a2[---]
- b [-]
- b1[-]
- b11 [--]
{code}
Which can also help comparing queue's resource but doesn't need a extra button
to hide/show queue hierarchy?
Add node label usage in RM CapacityScheduler web UI
---
Key: YARN-3362
URL: https://issues.apache.org/jira/browse/YARN-3362
Project: Hadoop YARN
Issue Type: Sub-task
Components: capacityscheduler, resourcemanager, webapp
Reporter: Wangda Tan
Assignee: Naganarasimha G R
Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue
Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png,
2015.05.10_3362_Queue_Hierarchy.png, 2015.05.12_3362_Queue_Hierarchy.png,
CSWithLabelsView.png, No-space-between-Active_user_info-and-next-queues.png,
Screen Shot 2015-04-29 at 11.42.17 AM.png,
YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch,
YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch,
YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch,
YARN-3362.20150512-1.patch, capacity-scheduler.xml
We don't have node label usage in RM CapacityScheduler web UI now, without
this, user will be hard to understand what happened to nodes have labels
assign to it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538878#comment-14538878 ] Wangda Tan commented on YARN-3362: -- The latest patch LGTM. Add node label usage in RM CapacityScheduler web UI --- Key: YARN-3362 URL: https://issues.apache.org/jira/browse/YARN-3362 Project: Hadoop YARN Issue Type: Sub-task Components: capacityscheduler, resourcemanager, webapp Reporter: Wangda Tan Assignee: Naganarasimha G R Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, 2015.05.10_3362_Queue_Hierarchy.png, CSWithLabelsView.png, No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29 at 11.42.17 AM.png, YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, capacity-scheduler.xml We don't have node label usage in RM CapacityScheduler web UI now, without this, user will be hard to understand what happened to nodes have labels assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538836#comment-14538836 ] Wangda Tan commented on YARN-3362: -- Hi Naga, Thanks for updating, 1) To your questions: https://issues.apache.org/jira/browse/YARN-3362?focusedCommentId=14537181page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14537181, You can refer to YARN-2824 for more information about why default cap of labeled resource set to zero. The default of max-cap is 100 because queue can use such resource without configure it. Let me know if you have more questions. 2) About showing resources of partitions, I think it's very helpful. I think you can include used-resource of each partition as well, You can file a separate ticket if it is hard to be added with this ticket. 3) About Hide Hierarchy, I think it's good for queue capacity comparison, but admin may get confused after checked Hide Hierarchy, it's better to be added to some other places instead of modify queue UI itself. Add node label usage in RM CapacityScheduler web UI --- Key: YARN-3362 URL: https://issues.apache.org/jira/browse/YARN-3362 Project: Hadoop YARN Issue Type: Sub-task Components: capacityscheduler, resourcemanager, webapp Reporter: Wangda Tan Assignee: Naganarasimha G R Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, 2015.05.10_3362_Queue_Hierarchy.png, CSWithLabelsView.png, No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29 at 11.42.17 AM.png, YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, YARN-3362.20150510-1.patch, YARN-3362.20150511-1.patch, capacity-scheduler.xml We don't have node label usage in RM CapacityScheduler web UI now, without this, user will be hard to understand what happened to nodes have labels assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[
https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14539144#comment-14539144
]
Naganarasimha G R commented on YARN-3362:
-
Thanks for the comment [~wangda], Here are my comments inline
bq. The default of max-cap is 100 because queue can use such resource without
configure it. Let me know if you have more questions.
IIUC, the current code only shows in the UI as 100% that too at the location
which i mentioned ( CapacitySchedulerQueueInfo ln 73) but actual absolute
capacity is zero which makes like no bar is shown. I think its a bug in the UI
and as well it should be in the way we are discussing(max-cap is 100 because
queue can use such resource without configure it), let me test and raise it.
bq. About showing resources of partitions, I think it's very helpful. I think
you can include used-resource of each partition as well, You can file a
separate ticket if it is hard to be added with this ticket.
I think its a simple change, give me some time will get the modifications done.
bq. About Hide Hierarchy, I think it's good for queue capacity comparison,
but admin may get confused after checked Hide Hierarchy, it's better to be
added to some other places instead of modify queue UI itself.
I meant, label on the button should also be toggling, i.e. when hierarchies are
shown then label can be {{Hide Hierarchy / Show only Leaf Queues}} and when
only Leaf Queues we can have label as {{Show Queue Hierarchy}}. Well i think if
the idea is ok can raise and handle in another jira as it involves lot more
discussions but i felt its particularly useful in our labels view as there are
too many things displayed in the UI and more the labels or more queue hierarchy
then it will be come more chaotic.
Add node label usage in RM CapacityScheduler web UI
---
Key: YARN-3362
URL: https://issues.apache.org/jira/browse/YARN-3362
Project: Hadoop YARN
Issue Type: Sub-task
Components: capacityscheduler, resourcemanager, webapp
Reporter: Wangda Tan
Assignee: Naganarasimha G R
Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue
Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png,
2015.05.10_3362_Queue_Hierarchy.png, CSWithLabelsView.png,
No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29
at 11.42.17 AM.png, YARN-3362.20150428-3-modified.patch,
YARN-3362.20150428-3.patch, YARN-3362.20150506-1.patch,
YARN-3362.20150507-1.patch, YARN-3362.20150510-1.patch,
YARN-3362.20150511-1.patch, capacity-scheduler.xml
We don't have node label usage in RM CapacityScheduler web UI now, without
this, user will be hard to understand what happened to nodes have labels
assign to it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[
https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537487#comment-14537487
]
Hadoop QA commented on YARN-3362:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 39s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 34s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 35s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 1m 46s | The applied patch generated
12 new checkstyle issues (total was 145, now 144). |
| {color:green}+1{color} | whitespace | 0m 2s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 42s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 16s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | yarn tests | 52m 10s | Tests passed in
hadoop-yarn-server-resourcemanager. |
| | | 89m 44s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12731834/YARN-3362.20150511-1.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 4536399 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/7857/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7857/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/7857/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7857/console |
This message was automatically generated.
Add node label usage in RM CapacityScheduler web UI
---
Key: YARN-3362
URL: https://issues.apache.org/jira/browse/YARN-3362
Project: Hadoop YARN
Issue Type: Sub-task
Components: capacityscheduler, resourcemanager, webapp
Reporter: Wangda Tan
Assignee: Naganarasimha G R
Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue
Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png,
2015.05.10_3362_Queue_Hierarchy.png, CSWithLabelsView.png,
No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29
at 11.42.17 AM.png, YARN-3362.20150428-3-modified.patch,
YARN-3362.20150428-3.patch, YARN-3362.20150506-1.patch,
YARN-3362.20150507-1.patch, YARN-3362.20150510-1.patch,
YARN-3362.20150511-1.patch, capacity-scheduler.xml
We don't have node label usage in RM CapacityScheduler web UI now, without
this, user will be hard to understand what happened to nodes have labels
assign to it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[
https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537231#comment-14537231
]
Hadoop QA commented on YARN-3362:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 43s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 35s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 36s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 0m 59s | The applied patch generated
15 new checkstyle issues (total was 145, now 147). |
| {color:green}+1{color} | whitespace | 0m 2s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 39s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 16s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | yarn tests | 52m 19s | Tests passed in
hadoop-yarn-server-resourcemanager. |
| | | 89m 12s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12731796/YARN-3362.20150510-1.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 4536399 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/7853/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7853/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/7853/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7853/console |
This message was automatically generated.
Add node label usage in RM CapacityScheduler web UI
---
Key: YARN-3362
URL: https://issues.apache.org/jira/browse/YARN-3362
Project: Hadoop YARN
Issue Type: Sub-task
Components: capacityscheduler, resourcemanager, webapp
Reporter: Wangda Tan
Assignee: Naganarasimha G R
Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue
Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png,
2015.05.10_3362_Queue_Hierarchy.png, CSWithLabelsView.png,
No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29
at 11.42.17 AM.png, YARN-3362.20150428-3-modified.patch,
YARN-3362.20150428-3.patch, YARN-3362.20150506-1.patch,
YARN-3362.20150507-1.patch, YARN-3362.20150510-1.patch, capacity-scheduler.xml
We don't have node label usage in RM CapacityScheduler web UI now, without
this, user will be hard to understand what happened to nodes have labels
assign to it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[
https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537183#comment-14537183
]
Naganarasimha G R commented on YARN-3362:
-
Also few suggestions in UI :
* Earlier over all available resource info was available in cluster metrics and
hence i think it was easier for viewer to relate the percentages but here
available resource info for a given label is present only in nodelabels page.
so i would suggest to have label resource info like :
{code}
+ Partition: xxx [memory:8192, vCores:8]
+ Queue: root
+ Queue: a
+ Queue: b
+ Partition: yyy [memory:4096, vCores:8]
+ Queue: root
+ Queue: a
+ Queue: b
{code}
* I think queue hierarchy is not always helpful when i want to relatively
compare capacities among leaf queue (through bars), so i would suggest
additional view without queue hierarchy when user clicks on a button [ kind of
toggle button].
{code}
---
| Hide Hierarchy |
---
+ Partition: xxx [memory:8192, vCores:8]
+ Queue Path: root.Q1.a
+ Queue Path: default
+ Partition: yyy [memory:4096, vCores:8]
+ Queue Path: root.Q1.a
- Queue Path: default
|- Partition Specific Metrics ---|
| |
|-|
|- Queue General Metrics |
| |
|-|
Active user info:
Table ..
{code}
Add node label usage in RM CapacityScheduler web UI
---
Key: YARN-3362
URL: https://issues.apache.org/jira/browse/YARN-3362
Project: Hadoop YARN
Issue Type: Sub-task
Components: capacityscheduler, resourcemanager, webapp
Reporter: Wangda Tan
Assignee: Naganarasimha G R
Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue
Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png,
2015.05.10_3362_Queue_Hierarchy.png, CSWithLabelsView.png,
No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29
at 11.42.17 AM.png, YARN-3362.20150428-3-modified.patch,
YARN-3362.20150428-3.patch, YARN-3362.20150506-1.patch,
YARN-3362.20150507-1.patch, YARN-3362.20150510-1.patch, capacity-scheduler.xml
We don't have node label usage in RM CapacityScheduler web UI now, without
this, user will be hard to understand what happened to nodes have labels
assign to it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[
https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14533120#comment-14533120
]
Wangda Tan commented on YARN-3362:
--
Hi [~Naganarasimha],
Thanks a lot for updating, looks much better now! I still have few minor
comments:
*For UI:*
1) Configured Capacity, Configured Max Capacity should be a part of Queue
Status for Partition...?
2) Not caused by your patch, Absolute Capacity should be Absolute Configured
Capacity, Absolute Max Capacity should be Absolute Configured Max Capacity,
could you update them in your patch?
3) Also not caused by your patch, there's no space between Active Users Info
and following queue, I'm not sure if there's any easy fix can do, please feel
free to file a separate ticket if it will be hard to be solved together.
*For implementation:*
1) One minor comment for style is, you can merge all capacities-related
rendering in CapacitySchedulerPage to a method similar to
renderCommonLeafQueueInfo, which you can merge some implementation for
{{render}} and {{renderLeafQueueInfoWithoutParition}}. And add a method
renderLeafQueueInfoWithParition to make {{render}} looks cleaner.
*For your question*
bq. May be i did not get this completely. label is exclusive-label i have done
in CapacitySchedulerPage.QueuesBlock.render(l num - 357)
I think for both CapacitySchedulerPage and CapacitySchedulerInfo, it should be:
{code}
if (label.getIsExclusive()
!((AbstractCSQueue)
root).accessibleToPartition(label.getLabelName())) {
{code}
When label is exclusive (nobody can use the label except queue is accessible to
the label) and (queue isn't accessible to the label), we don't need to continue.
Let me know your thoughts.
CC: [~vinodkv]/[~jianhe].
Add node label usage in RM CapacityScheduler web UI
---
Key: YARN-3362
URL: https://issues.apache.org/jira/browse/YARN-3362
Project: Hadoop YARN
Issue Type: Sub-task
Components: capacityscheduler, resourcemanager, webapp
Reporter: Wangda Tan
Assignee: Naganarasimha G R
Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue
Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, CSWithLabelsView.png,
Screen Shot 2015-04-29 at 11.42.17 AM.png,
YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch,
YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, capacity-scheduler.xml
We don't have node label usage in RM CapacityScheduler web UI now, without
this, user will be hard to understand what happened to nodes have labels
assign to it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14533174#comment-14533174 ] Naganarasimha G R commented on YARN-3362: - Thanks for the review [~wangda], bq. Also not caused by your patch, there's no space between Active Users Info and following queue, I'm not sure if there's any easy fix can do, please feel free to file a separate ticket if it will be hard to be solved together. Put some considerable amount of effort, dint workout. Hamlet should have add some kind of doc or a book, its seems like an enigma code to me. Will file a separate jira as its required in many places of CS page : Active Users Info and following queue, in between Dump scheduler log button and Application Queue , In CS Health Block For other comments will rework ASAP. Add node label usage in RM CapacityScheduler web UI --- Key: YARN-3362 URL: https://issues.apache.org/jira/browse/YARN-3362 Project: Hadoop YARN Issue Type: Sub-task Components: capacityscheduler, resourcemanager, webapp Reporter: Wangda Tan Assignee: Naganarasimha G R Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, CSWithLabelsView.png, No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29 at 11.42.17 AM.png, YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, capacity-scheduler.xml We don't have node label usage in RM CapacityScheduler web UI now, without this, user will be hard to understand what happened to nodes have labels assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14533122#comment-14533122 ] Wangda Tan commented on YARN-3362: -- Attached https://issues.apache.org/jira/secure/attachment/12731236/No-space-between-Active_user_info-and-next-queues.png; to showing there's no space between Active User Info and next queue. Add node label usage in RM CapacityScheduler web UI --- Key: YARN-3362 URL: https://issues.apache.org/jira/browse/YARN-3362 Project: Hadoop YARN Issue Type: Sub-task Components: capacityscheduler, resourcemanager, webapp Reporter: Wangda Tan Assignee: Naganarasimha G R Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, CSWithLabelsView.png, No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29 at 11.42.17 AM.png, YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, capacity-scheduler.xml We don't have node label usage in RM CapacityScheduler web UI now, without this, user will be hard to understand what happened to nodes have labels assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14533187#comment-14533187 ] Wangda Tan commented on YARN-3362: -- bq. Put some considerable amount of effort, dint workout. Hamlet should have add some kind of doc or a book, its seems like an enigma code to me. Will file a separate jira as its required in many places of CS page : Active Users Info and following queue, in between Dump scheduler log button and Application Queue , In CS Health Block Thanks for looking at this, that sounds good, we can address of them together in a separated JIRA. Add node label usage in RM CapacityScheduler web UI --- Key: YARN-3362 URL: https://issues.apache.org/jira/browse/YARN-3362 Project: Hadoop YARN Issue Type: Sub-task Components: capacityscheduler, resourcemanager, webapp Reporter: Wangda Tan Assignee: Naganarasimha G R Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, CSWithLabelsView.png, No-space-between-Active_user_info-and-next-queues.png, Screen Shot 2015-04-29 at 11.42.17 AM.png, YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, capacity-scheduler.xml We don't have node label usage in RM CapacityScheduler web UI now, without this, user will be hard to understand what happened to nodes have labels assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[
https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14533015#comment-14533015
]
Hadoop QA commented on YARN-3362:
-
\\
\\
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 15m 10s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 52s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 53s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 26s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 2s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 37s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 16s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | yarn tests | 58m 49s | Tests passed in
hadoop-yarn-server-resourcemanager. |
| | | 96m 4s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12731201/YARN-3362.20150507-1.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 8e991f4 |
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7762/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/7762/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7762/console |
This message was automatically generated.
Add node label usage in RM CapacityScheduler web UI
---
Key: YARN-3362
URL: https://issues.apache.org/jira/browse/YARN-3362
Project: Hadoop YARN
Issue Type: Sub-task
Components: capacityscheduler, resourcemanager, webapp
Reporter: Wangda Tan
Assignee: Naganarasimha G R
Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue
Expanded.png, 2015.05.07_3362_Queue_Hierarchy.png, CSWithLabelsView.png,
Screen Shot 2015-04-29 at 11.42.17 AM.png,
YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch,
YARN-3362.20150506-1.patch, YARN-3362.20150507-1.patch, capacity-scheduler.xml
We don't have node label usage in RM CapacityScheduler web UI now, without
this, user will be hard to understand what happened to nodes have labels
assign to it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14531614#comment-14531614 ] Vinod Kumar Vavilapalli commented on YARN-3362: --- Can you please post the latest screenshot? Add node label usage in RM CapacityScheduler web UI --- Key: YARN-3362 URL: https://issues.apache.org/jira/browse/YARN-3362 Project: Hadoop YARN Issue Type: Sub-task Components: capacityscheduler, resourcemanager, webapp Reporter: Wangda Tan Assignee: Naganarasimha G R Attachments: CSWithLabelsView.png, Screen Shot 2015-04-29 at 11.42.17 AM.png, YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, YARN-3362.20150506-1.patch, capacity-scheduler.xml We don't have node label usage in RM CapacityScheduler web UI now, without this, user will be hard to understand what happened to nodes have labels assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[
https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14531669#comment-14531669
]
Wangda Tan commented on YARN-3362:
--
Hi Naga,
Thanks for updating, I took a look at closer look at latest patch, and tried in
my local cluster, some comments in code:
1) There's some duplicated code for CapacitySchedulerInfo, you only need keep
one constructor, which takes label always
2) Also, {{protected CapacitySchedulerQueueInfoList getQueues}} doesn't need to
do specifial logic when {{null == nodeLabel}}, queue can be ignored when:
{{queue cannot access to label AND label is exclusive-label}}.
accessibleToPartition in AbstractCSQueue already considered when label==
3) CapacitySchedulerQueueInfo doesn't need two constructor as well
4) When you want to get NO_LABEL, you should pass
RMNodeLabelsManager.NO_LABEL instead of null.
Some comments in UI:
1) I've reconsidered the hierarchy, I think it's better to rename
Node_Label=... to Partition: For the Node Labels page, we should
add a column which is Node Label Type, and indicate they're partition.
2) NO_LABEL should be DEFAULT_PARTITION, NO_LABEL itself may not clear
enough, also in Node Labels page, we should change NO_LABEL to the same name
here. Changes in Node Labels page could add in separated JIRA.
3) It's better to indicate queue in hierarchy also, now we have:
{code}
+ Partition: xxx
+ root
+ a
+ b
{code}
I think it's better to add a Queue: before queue name:
{code}
+ Partition: xxx
+ Queue: root
+ Queue: a
+ Queue: b
{code}
Otherwise people may get confused which one is partition and which one is queue.
4) In queue's metrics overview table, it mixing partition-specific and
queue-generic metrics is not clear enough, I'm not sure if you can show it like
this (If existing Hamlet framework supports this or not):
{code}
+ Partition: xxx
+ Queue: root
+ Queue: a
|- Partition Specific Metrics ---|
| Used Capacity |0.0% |
| Absolute Capacity | 50.0% |
|-|
|- Queue General Metrics |
| Other Metrics |
---
Active user info:
Table ..
+ Queue: b
{code}
Add node label usage in RM CapacityScheduler web UI
---
Key: YARN-3362
URL: https://issues.apache.org/jira/browse/YARN-3362
Project: Hadoop YARN
Issue Type: Sub-task
Components: capacityscheduler, resourcemanager, webapp
Reporter: Wangda Tan
Assignee: Naganarasimha G R
Attachments: CSWithLabelsView.png, Screen Shot 2015-04-29 at 11.42.17
AM.png, YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch,
YARN-3362.20150506-1.patch, capacity-scheduler.xml
We don't have node label usage in RM CapacityScheduler web UI now, without
this, user will be hard to understand what happened to nodes have labels
assign to it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14531689#comment-14531689 ] Wangda Tan commented on YARN-3362: -- Attached folded/expanded queues screenshots. Add node label usage in RM CapacityScheduler web UI --- Key: YARN-3362 URL: https://issues.apache.org/jira/browse/YARN-3362 Project: Hadoop YARN Issue Type: Sub-task Components: capacityscheduler, resourcemanager, webapp Reporter: Wangda Tan Assignee: Naganarasimha G R Attachments: 2015.05.06 Folded Queues.png, 2015.05.06 Queue Expanded.png, CSWithLabelsView.png, Screen Shot 2015-04-29 at 11.42.17 AM.png, YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch, YARN-3362.20150506-1.patch, capacity-scheduler.xml We don't have node label usage in RM CapacityScheduler web UI now, without this, user will be hard to understand what happened to nodes have labels assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[
https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14531418#comment-14531418
]
Hadoop QA commented on YARN-3362:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 40s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 33s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 33s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 0m 46s | The applied patch generated
11 new checkstyle issues (total was 100, now 109). |
| {color:red}-1{color} | whitespace | 0m 2s | The patch has 7 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 33s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 1m 17s | The patch appears to introduce 1
new Findbugs (version 2.0.3) warnings. |
| {color:red}-1{color} | yarn tests | 62m 56s | Tests failed in
hadoop-yarn-server-resourcemanager. |
| | | 99m 19s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-yarn-server-resourcemanager |
| | Load of known null value in
org.apache.hadoop.yarn.server.resourcemanager.webapp.dao.CapacitySchedulerInfo.getQueues(CSQueue,
String) At CapacitySchedulerInfo.java:in
org.apache.hadoop.yarn.server.resourcemanager.webapp.dao.CapacitySchedulerInfo.getQueues(CSQueue,
String) At CapacitySchedulerInfo.java:[line 110] |
| Timed out tests |
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
|
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12730873/YARN-3362.20150506-1.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 185e63a |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/7740/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/7740/artifact/patchprocess/whitespace.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-YARN-Build/7740/artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7740/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/7740/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7740/console |
This message was automatically generated.
Add node label usage in RM CapacityScheduler web UI
---
Key: YARN-3362
URL: https://issues.apache.org/jira/browse/YARN-3362
Project: Hadoop YARN
Issue Type: Sub-task
Components: capacityscheduler, resourcemanager, webapp
Reporter: Wangda Tan
Assignee: Naganarasimha G R
Attachments: CSWithLabelsView.png, Screen Shot 2015-04-29 at 11.42.17
AM.png, YARN-3362.20150428-3-modified.patch, YARN-3362.20150428-3.patch,
YARN-3362.20150506-1.patch, capacity-scheduler.xml
We don't have node label usage in RM CapacityScheduler web UI now, without
this, user will be hard to understand what happened to nodes have labels
assign to it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14520074#comment-14520074 ] Wangda Tan commented on YARN-3362: -- bq. Well i understand that in the later patches we are targetting it more as partition than labels, but in that case shall i modify the same in other locations of WEB like node labels page, in CS page shall i mark it as Accessible Partitions ? Good point, I think we may need keep it to be label, and do the renaming in a separated patch. bq. in CS page shall i mark it as Accessible Partitions We can keep calling it label avoid confusion. bq. you mean if no node is mapped to cluster node label then not to show that Node Label ? In my mind is, show all node labels no matter they mapped to nodes/queues or not. We can optimize this easily in the future, I prefer to keep completed message before people post their comments. bq. you mean the existing names of metrics entries needs to be appended with (Partition=xxx) and not to show both right ? I think we need to show both (partition-specific and queue general), the only change is append with (Node-Label=xxx). bq. Its great to hear its working fine, but it worked without any modifications to the patch ? Forgot to mention, I modified patch a little bit, removed some avoid-displaying-checking mentioned by you at https://issues.apache.org/jira/browse/YARN-3362?focusedCommentId=14517364page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14517364. Uploading modified patch as well as CS config for you to test. Add node label usage in RM CapacityScheduler web UI --- Key: YARN-3362 URL: https://issues.apache.org/jira/browse/YARN-3362 Project: Hadoop YARN Issue Type: Sub-task Components: capacityscheduler, resourcemanager, webapp Reporter: Wangda Tan Assignee: Naganarasimha G R Attachments: CSWithLabelsView.png, Screen Shot 2015-04-29 at 11.42.17 AM.png, YARN-3362.20150428-3.patch We don't have node label usage in RM CapacityScheduler web UI now, without this, user will be hard to understand what happened to nodes have labels assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14519917#comment-14519917 ] Wangda Tan commented on YARN-3362: -- Hi Naga, Thanks for taking initiative for this, just tried to run the patch locally, looks great! Some comments: 1) Show partition=partition-name in every partition, if the partition is the NO_LABEL partition, show it's a YARN.DEFAULT.PARTITION. 2) I think it's better to show labels are not accessible, especially for the non-exclusive node label case, we can optimize this in future patch. To avoid people ask question like where is my label? This includes all existing avoid displaying items in your existing patch. But it's good to keep avoid showing label when there's no label in your cluster. 3) Showing partition of partition-specific queue metrics, they're: - Used Capacity:0.0% - Absolute Used Capacity: 0.0% - Absolute Capacity:50.0% - Absolute Max Capacity:100.0% - Configured Capacity: 50.0% - Configured Max Capacity: 100.0% I suggest to add a (Partition=xxx) at the end of these metrics. I attached queue hierarchy showing in my local cluster: https://issues.apache.org/jira/secure/attachment/12729256/Screen%20Shot%202015-04-29%20at%2011.42.17%20AM.png. It seems multi hierarchy works well in my environment. Add node label usage in RM CapacityScheduler web UI --- Key: YARN-3362 URL: https://issues.apache.org/jira/browse/YARN-3362 Project: Hadoop YARN Issue Type: Sub-task Components: capacityscheduler, resourcemanager, webapp Reporter: Wangda Tan Assignee: Naganarasimha G R Attachments: CSWithLabelsView.png, Screen Shot 2015-04-29 at 11.42.17 AM.png, YARN-3362.20150428-3.patch We don't have node label usage in RM CapacityScheduler web UI now, without this, user will be hard to understand what happened to nodes have labels assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14519960#comment-14519960 ] Naganarasimha G R commented on YARN-3362: - Thanks [~wangda], for reviewing and testing the patch. bq. partition=partition-name Well i understand that in the later patches we are targetting it more as partition than labels, but in that case shall i modify the same in other locations of WEB like node labels page, in CS page shall i mark it as Accessible Partitions ? bq. But it's good to keep avoid showing label when there's no label in your cluster. you mean if no node is mapped to cluster node label then not to show that Node Label ? bq. Showing partition of partition-specific queue metrics you mean the existing names of metrics entries needs to be appended with (Partition=xxx) and not to show both right ? bq. It seems multi hierarchy works well in my environment. Its great to hear its working fine, but it worked without any modifications to the patch ? If so can you share offline your cluster setup (topology) with CS configuration, so that i can test it further. Add node label usage in RM CapacityScheduler web UI --- Key: YARN-3362 URL: https://issues.apache.org/jira/browse/YARN-3362 Project: Hadoop YARN Issue Type: Sub-task Components: capacityscheduler, resourcemanager, webapp Reporter: Wangda Tan Assignee: Naganarasimha G R Attachments: CSWithLabelsView.png, Screen Shot 2015-04-29 at 11.42.17 AM.png, YARN-3362.20150428-3.patch We don't have node label usage in RM CapacityScheduler web UI now, without this, user will be hard to understand what happened to nodes have labels assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14507949#comment-14507949 ] Wangda Tan commented on YARN-3362: -- Since this patch tends to create a new RM web view, so I suggest to do it in a compatible way so that people will not get lost when using the new view: - When label is not enabled in the cluster or there's only default label, it will not show the label-hierarchy, so it will be very similar to what we have in old RM web UI - All queue's capacities and usages (by partition) can be found in CSQueue.getQueueCapacities/getQueueResourceUsages. - We may not need to consider show app-by-partition with this patch. Filter apps in a queue by partition needs add additional CS interface, which should be addressed in a separated patch. Add node label usage in RM CapacityScheduler web UI --- Key: YARN-3362 URL: https://issues.apache.org/jira/browse/YARN-3362 Project: Hadoop YARN Issue Type: Sub-task Components: capacityscheduler, resourcemanager, webapp Reporter: Wangda Tan Assignee: Naganarasimha G R We don't have node label usage in RM CapacityScheduler web UI now, without this, user will be hard to understand what happened to nodes have labels assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378200#comment-14378200 ] Wangda Tan commented on YARN-3362: -- bq. If this is the case then the approach which you specified makes sense but by can you mean currently its not there and in future it can come in ? Some of them are already existed, like user-limit, and some of them are coming, like am-resource-percent. Sorry I may not understand what's your question, user-limit and queue-limit are just two different limits regardless of node labels, sometimes user-limit higher and sometimes queue-limit higher. Could you explain what's your question (maybe by example)? Thanks, Add node label usage in RM CapacityScheduler web UI --- Key: YARN-3362 URL: https://issues.apache.org/jira/browse/YARN-3362 Project: Hadoop YARN Issue Type: Sub-task Components: capacityscheduler, resourcemanager, webapp Reporter: Wangda Tan Assignee: Naganarasimha G R We don't have node label usage in RM CapacityScheduler web UI now, without this, user will be hard to understand what happened to nodes have labels assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376563#comment-14376563 ] Naganarasimha G R commented on YARN-3362: - Thanks for the feedback [~leftnoteasy], bq. different labels under same queue can have different user-limit/capacity/maximum-capacity/max-am-resource, etc. If this is the case then the approach which you specified makes sense but by can you mean currently its not there and in future it can come in ? More than repeated info, other drawback i can see is suppose for particular label userlimit is not reached but as overall at queue level if the user has reached his limit it will be difficult for user to go through all labels and find out whether user has reached queue limit . Correct me if my understanding on this is wrong . Add node label usage in RM CapacityScheduler web UI --- Key: YARN-3362 URL: https://issues.apache.org/jira/browse/YARN-3362 Project: Hadoop YARN Issue Type: Sub-task Components: capacityscheduler, resourcemanager, webapp Reporter: Wangda Tan Assignee: Naganarasimha G R We don't have node label usage in RM CapacityScheduler web UI now, without this, user will be hard to understand what happened to nodes have labels assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370434#comment-14370434 ] Wangda Tan commented on YARN-3362: -- Hi [~Naganarasimha], Thanks for your comments, bq. There will be some common queue metrics across the labels, wont it get repeated across for each label if a queue is mapped to multiple labels? Some common fields may get repeated (like absolute max capacity, etc.). I think repeat some of them is not a very big issue to me. I think we can show queue-label-metrics + queue-common-metrics for each queue-label bq. IIUC most of the queue Metrics might not be specific to a label, like Capacity, Absolute max capacity, Max apps, Max AM's per user etc... . Correct me if my understanding on this is wrong. Yes, they're, but there're more parameters / metrics in queues for both label/queue, different labels under same queue can have different user-limit/capacity/maximum-capacity/max-am-resource, etc.). We also need to show them to users if possible bq. Apart from the label specific queue metrics like (label capacity, label abs capacity,used) are there any new Label specific queue metrics you have in your mind ? I think above can answer your question. bq. would it be better to list like If we have this view, 1) How you show label-specific metrics? 2) What's the used-resource in queue level means (used-resource make more sense when it's per-label). 3) How to check label-wise resource usage for parent queues. bq. Also if required we can have seperate page (/in the labels page/append at the end of CS page) like I think my proposal is still a little more clear, we need to show label-wise metrics to user. And with that, user can clear understand resource usage for each partition (just check each label's usage. Also parent's label-wise usage can show as well. Add node label usage in RM CapacityScheduler web UI --- Key: YARN-3362 URL: https://issues.apache.org/jira/browse/YARN-3362 Project: Hadoop YARN Issue Type: Sub-task Components: capacityscheduler, resourcemanager, webapp Reporter: Wangda Tan Assignee: Naganarasimha G R We don't have node label usage in RM CapacityScheduler web UI now, without this, user will be hard to understand what happened to nodes have labels assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370438#comment-14370438 ] Wangda Tan commented on YARN-3362: -- For the active-user-info, we need some queue-user-by-label metrics as well, such as used-resource-by-user-and-label. Which can be placed to queue-label metrics table. Add node label usage in RM CapacityScheduler web UI --- Key: YARN-3362 URL: https://issues.apache.org/jira/browse/YARN-3362 Project: Hadoop YARN Issue Type: Sub-task Components: capacityscheduler, resourcemanager, webapp Reporter: Wangda Tan Assignee: Naganarasimha G R We don't have node label usage in RM CapacityScheduler web UI now, without this, user will be hard to understand what happened to nodes have labels assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[
https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368463#comment-14368463
]
Naganarasimha G R commented on YARN-3362:
-
Thanks [~wangda],
Regarding the approach to display i had few concerns :
* There will be some common queue metrics across the labels, wont it get
repeated across for each label if a queue is mapped to multiple labels ?
* IIUC most of the queue Metrics might not be specific to a label, like
Capacity, Absolute max capacity, Max apps, Max AM's per user etc... . Correct
me if my understanding on this is wrong.
* Apart from the label specific queue metrics like (label capacity, label abs
capacity,used) are there any new Label specific queue metrics you have in your
mind ?
* would it be better to list like
{noformat}
+ root [=] 30% used
+ a [===] 75% used
+ a1 [=] 30% used
-
| Queue Metrics |
||
| metrics1 |value1 |
| metrics2 |value2 |
-
| Active Users info (yarn-3273)|
||
| user1 |info|
| user2 |info|
-
| Label Resource usage info |
||
| label_x [=] 30% used |
| label_y [] 20% used |
--
+ a2 [=] 30% used
...
{noformat}
* Also if required we can have seperate page (/in the labels page/append at the
end of CS page) like :
{noformat}
+ label_x [=] 30% used [Actual Resource - Used resource ]
+ root [=] 30% used [Actual Resource - Used
resource ]
+ a [===] 75% used [Actual
Resource - Used resource ]
+ a1 [=] 30% used [Actual Resource - Used resource
]
+ label_y
+ root [...]
+ ...
+ label_z
+ root [...]
{noformat}
YARN-3273, has added more info to the CS page so we need to consider the size
of page and its usability.
Please provide your thoughts on the same
Add node label usage in RM CapacityScheduler web UI
---
Key: YARN-3362
URL: https://issues.apache.org/jira/browse/YARN-3362
Project: Hadoop YARN
Issue Type: Sub-task
Components: capacityscheduler, resourcemanager, webapp
Reporter: Wangda Tan
Assignee: Naganarasimha G R
We don't have node label usage in RM CapacityScheduler web UI now, without
this, user will be hard to understand what happened to nodes have labels
assign to it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366608#comment-14366608 ] Naganarasimha G R commented on YARN-3362: - Hi Wangda, Would like to work on this issue, hence have assigned to my name, if you have already started working on it please feel free to reassign. Add node label usage in RM CapacityScheduler web UI --- Key: YARN-3362 URL: https://issues.apache.org/jira/browse/YARN-3362 Project: Hadoop YARN Issue Type: Sub-task Components: capacityscheduler, resourcemanager, webapp Reporter: Wangda Tan Assignee: Naganarasimha G R We don't have node label usage in RM CapacityScheduler web UI now, without this, user will be hard to understand what happened to nodes have labels assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[ https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366616#comment-14366616 ] Wangda Tan commented on YARN-3362: -- It's yours :). Looking forward your patch. Thanks, Add node label usage in RM CapacityScheduler web UI --- Key: YARN-3362 URL: https://issues.apache.org/jira/browse/YARN-3362 Project: Hadoop YARN Issue Type: Sub-task Components: capacityscheduler, resourcemanager, webapp Reporter: Wangda Tan Assignee: Naganarasimha G R We don't have node label usage in RM CapacityScheduler web UI now, without this, user will be hard to understand what happened to nodes have labels assign to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3362) Add node label usage in RM CapacityScheduler web UI
[
https://issues.apache.org/jira/browse/YARN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366191#comment-14366191
]
Wangda Tan commented on YARN-3362:
--
My proposal is:
For now, RM CapacityScheduler UI looks like
{code}
+ root [==] 50% used
+ a [===] 75% used
- a1 [=] 30% used
-
| Queue Metrics Table |
||
| metrics1 |value1 |
| metrics2 |value2 |
| metrics3 |value3 |
| metrics4 |value4 |
--
+ b [...]
+ c [...]
{code}
We can add one more hierarchy above queue's hierarchy, which are labels can be
accessed and/or labels are being used by the queue, which can looks like
{code}
+ label_x [=] 30% used
+ root [=] 30% used
+ a [===] 75% used
+ a1 [=] 30% used
-
| Queue Metrics Table (For label_x) |
||
| metrics1 |value1 |
| metrics2 |value2 |
| metrics3 |value3 |
| metrics4 |value4 |
--
+ label_y
+ root [...]
+ ...
+ label_z
+ root [...]
+ ...
+ no_label
+ root [...]
+ ...
{code}
To make it backward compatible, when there's no label in the system, it will
not show label-bar, and root is still root-queue.
Please feel free to share your ideas on this!
Add node label usage in RM CapacityScheduler web UI
---
Key: YARN-3362
URL: https://issues.apache.org/jira/browse/YARN-3362
Project: Hadoop YARN
Issue Type: Sub-task
Components: capacityscheduler, resourcemanager, webapp
Reporter: Wangda Tan
We don't have node label usage in RM CapacityScheduler web UI now, without
this, user will be hard to understand what happened to nodes have labels
assign to it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
