[jira] [Commented] (YARN-7116) CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM usage.
[
https://issues.apache.org/jira/browse/YARN-7116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181174#comment-16181174
]
Sunil G commented on YARN-7116:
---
Yes. Its a good addition in branch-2.8
> CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM
> usage.
> -
>
> Key: YARN-7116
> URL: https://issues.apache.org/jira/browse/YARN-7116
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler, webapp
>Affects Versions: 2.9.0, 2.8.1, 3.0.0-alpha4
>Reporter: Wangda Tan
>Assignee: Wangda Tan
> Fix For: 2.9.0, 3.0.0-beta1, 2.8.3
>
> Attachments: YARN-7116.001.patch
>
>
> On CapacityScheduler's web UI, AM usage of different users belong to the same
> queue always shows queue's AM usage.
> The root cause is: under CapacitySchedulerPage.
> {code}
> tbody.tr().td(userInfo.getUsername())
> .td(userInfo.getUserResourceLimit().toString())
> .td(resourcesUsed.toString())
> .td(resourceUsages.getAMLimit().toString())
> .td(amUsed.toString())
> .td(Integer.toString(userInfo.getNumActiveApplications()))
> .td(Integer.toString(userInfo.getNumPendingApplications()))._();
> {code}
> Instead of amUsed.toString(), it should use userInfo.getAmUsed().
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7116) CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM usage.
[
https://issues.apache.org/jira/browse/YARN-7116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16176701#comment-16176701
]
Wangda Tan commented on YARN-7116:
--
[~eepayne], I'm fine with backporting to 2.8
> CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM
> usage.
> -
>
> Key: YARN-7116
> URL: https://issues.apache.org/jira/browse/YARN-7116
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler, webapp
>Affects Versions: 2.9.0, 2.8.1, 3.0.0-alpha4
>Reporter: Wangda Tan
>Assignee: Wangda Tan
> Fix For: 2.9.0, 3.0.0-beta1
>
> Attachments: YARN-7116.001.patch
>
>
> On CapacityScheduler's web UI, AM usage of different users belong to the same
> queue always shows queue's AM usage.
> The root cause is: under CapacitySchedulerPage.
> {code}
> tbody.tr().td(userInfo.getUsername())
> .td(userInfo.getUserResourceLimit().toString())
> .td(resourcesUsed.toString())
> .td(resourceUsages.getAMLimit().toString())
> .td(amUsed.toString())
> .td(Integer.toString(userInfo.getNumActiveApplications()))
> .td(Integer.toString(userInfo.getNumPendingApplications()))._();
> {code}
> Instead of amUsed.toString(), it should use userInfo.getAmUsed().
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7116) CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM usage.
[
https://issues.apache.org/jira/browse/YARN-7116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16176700#comment-16176700
]
Eric Payne commented on YARN-7116:
--
[~leftnoteasy], [~sunilg]
If there are no objections, I'll backport this to 2.8.
> CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM
> usage.
> -
>
> Key: YARN-7116
> URL: https://issues.apache.org/jira/browse/YARN-7116
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler, webapp
>Affects Versions: 2.9.0, 2.8.1, 3.0.0-alpha4
>Reporter: Wangda Tan
>Assignee: Wangda Tan
> Fix For: 2.9.0, 3.0.0-beta1
>
> Attachments: YARN-7116.001.patch
>
>
> On CapacityScheduler's web UI, AM usage of different users belong to the same
> queue always shows queue's AM usage.
> The root cause is: under CapacitySchedulerPage.
> {code}
> tbody.tr().td(userInfo.getUsername())
> .td(userInfo.getUserResourceLimit().toString())
> .td(resourcesUsed.toString())
> .td(resourceUsages.getAMLimit().toString())
> .td(amUsed.toString())
> .td(Integer.toString(userInfo.getNumActiveApplications()))
> .td(Integer.toString(userInfo.getNumPendingApplications()))._();
> {code}
> Instead of amUsed.toString(), it should use userInfo.getAmUsed().
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7116) CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM usage.
[
https://issues.apache.org/jira/browse/YARN-7116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148979#comment-16148979
]
Hudson commented on YARN-7116:
--
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12284 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/12284/])
YARN-7116. CapacityScheduler Web UI: Queue's AM usage is always show on
(sunilg: rev f9e0cc8cdca387bf2811c07beccae4af07362952)
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/CapacitySchedulerPage.java
> CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM
> usage.
> -
>
> Key: YARN-7116
> URL: https://issues.apache.org/jira/browse/YARN-7116
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler, webapp
>Affects Versions: 2.9.0, 2.8.1, 3.0.0-alpha4
>Reporter: Wangda Tan
>Assignee: Wangda Tan
> Fix For: 2.9.0, 3.0.0-beta1
>
> Attachments: YARN-7116.001.patch
>
>
> On CapacityScheduler's web UI, AM usage of different users belong to the same
> queue always shows queue's AM usage.
> The root cause is: under CapacitySchedulerPage.
> {code}
> tbody.tr().td(userInfo.getUsername())
> .td(userInfo.getUserResourceLimit().toString())
> .td(resourcesUsed.toString())
> .td(resourceUsages.getAMLimit().toString())
> .td(amUsed.toString())
> .td(Integer.toString(userInfo.getNumActiveApplications()))
> .td(Integer.toString(userInfo.getNumPendingApplications()))._();
> {code}
> Instead of amUsed.toString(), it should use userInfo.getAmUsed().
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7116) CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM usage.
[
https://issues.apache.org/jira/browse/YARN-7116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146803#comment-16146803
]
Bibin A Chundatt commented on YARN-7116:
[~sunilg]
Don't have any concern regarding setting {{amused}} change.
Was under the impression that AM limit for user was set only for default
partition.
+1 lgtm
> CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM
> usage.
> -
>
> Key: YARN-7116
> URL: https://issues.apache.org/jira/browse/YARN-7116
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler, webapp
>Affects Versions: 2.9.0, 2.8.1, 3.0.0-alpha4
>Reporter: Wangda Tan
>Assignee: Wangda Tan
> Attachments: YARN-7116.001.patch
>
>
> On CapacityScheduler's web UI, AM usage of different users belong to the same
> queue always shows queue's AM usage.
> The root cause is: under CapacitySchedulerPage.
> {code}
> tbody.tr().td(userInfo.getUsername())
> .td(userInfo.getUserResourceLimit().toString())
> .td(resourcesUsed.toString())
> .td(resourceUsages.getAMLimit().toString())
> .td(amUsed.toString())
> .td(Integer.toString(userInfo.getNumActiveApplications()))
> .td(Integer.toString(userInfo.getNumPendingApplications()))._();
> {code}
> Instead of amUsed.toString(), it should use userInfo.getAmUsed().
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7116) CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM usage.
[
https://issues.apache.org/jira/browse/YARN-7116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145128#comment-16145128
]
Sunil G commented on YARN-7116:
---
[~bibinchundatt], I didnt understand your point clearly. As per below code, we
consider all resource usages per label and updating same. Below code is from
trunk. Post this patch, we handle {{amUsed}} more cleanly. Please update me if
I understood your comment wrongly.
{code}
PartitionResourcesInfo resourceUsages = lqinfo
.getResources()
.getPartitionResourceUsageInfo((nodeLabel == null) ? "" :
nodeLabel);
if (nodeLabel != null) {
resourcesUsed = userInfo.getResourceUsageInfo()
.getPartitionResourceUsageInfo(nodeLabel).getUsed();
}
ResourceInfo amUsed = (resourceUsages.getAmUsed() == null)
? new ResourceInfo(Resources.none())
: resourceUsages.getAmUsed();
{code}
> CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM
> usage.
> -
>
> Key: YARN-7116
> URL: https://issues.apache.org/jira/browse/YARN-7116
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler, webapp
>Affects Versions: 2.9.0, 2.8.1, 3.0.0-alpha4
>Reporter: Wangda Tan
>Assignee: Wangda Tan
> Attachments: YARN-7116.001.patch
>
>
> On CapacityScheduler's web UI, AM usage of different users belong to the same
> queue always shows queue's AM usage.
> The root cause is: under CapacitySchedulerPage.
> {code}
> tbody.tr().td(userInfo.getUsername())
> .td(userInfo.getUserResourceLimit().toString())
> .td(resourcesUsed.toString())
> .td(resourceUsages.getAMLimit().toString())
> .td(amUsed.toString())
> .td(Integer.toString(userInfo.getNumActiveApplications()))
> .td(Integer.toString(userInfo.getNumPendingApplications()))._();
> {code}
> Instead of amUsed.toString(), it should use userInfo.getAmUsed().
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7116) CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM usage.
[
https://issues.apache.org/jira/browse/YARN-7116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16144745#comment-16144745
]
Bibin A Chundatt commented on YARN-7116:
Thank you [~leftnoteasy] for patch. IIUC the AM limit shown is of partition
now. AM limit also could be changed. thoughts ??
> CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM
> usage.
> -
>
> Key: YARN-7116
> URL: https://issues.apache.org/jira/browse/YARN-7116
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler, webapp
>Affects Versions: 2.9.0, 2.8.1, 3.0.0-alpha4
>Reporter: Wangda Tan
>Assignee: Wangda Tan
> Attachments: YARN-7116.001.patch
>
>
> On CapacityScheduler's web UI, AM usage of different users belong to the same
> queue always shows queue's AM usage.
> The root cause is: under CapacitySchedulerPage.
> {code}
> tbody.tr().td(userInfo.getUsername())
> .td(userInfo.getUserResourceLimit().toString())
> .td(resourcesUsed.toString())
> .td(resourceUsages.getAMLimit().toString())
> .td(amUsed.toString())
> .td(Integer.toString(userInfo.getNumActiveApplications()))
> .td(Integer.toString(userInfo.getNumPendingApplications()))._();
> {code}
> Instead of amUsed.toString(), it should use userInfo.getAmUsed().
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7116) CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM usage.
[
https://issues.apache.org/jira/browse/YARN-7116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16144594#comment-16144594
]
Hadoop QA commented on YARN-7116:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 45m 49s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 69m 8s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation |
| | hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppStarvation |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-7116 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12884152/YARN-7116.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux a5d046886a2b 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / a1e3f84 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/17168/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/17168/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17168/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> CapacityScheduler Web UI: Queue's AM usage is
[jira] [Commented] (YARN-7116) CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM usage.
[
https://issues.apache.org/jira/browse/YARN-7116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16144524#comment-16144524
]
Wangda Tan commented on YARN-7116:
--
[~sunilg] could you help to review this patch? this should be very
straightforward :)
> CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM
> usage.
> -
>
> Key: YARN-7116
> URL: https://issues.apache.org/jira/browse/YARN-7116
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 2.9.0, 2.8.1, 3.0.0-alpha4
>Reporter: Wangda Tan
>Assignee: Wangda Tan
> Attachments: YARN-7116.001.patch
>
>
> On CapacityScheduler's web UI, AM usage of different users belong to the same
> queue always shows queue's AM usage.
> The root cause is: under CapacitySchedulerPage.
> {code}
> tbody.tr().td(userInfo.getUsername())
> .td(userInfo.getUserResourceLimit().toString())
> .td(resourcesUsed.toString())
> .td(resourceUsages.getAMLimit().toString())
> .td(amUsed.toString())
> .td(Integer.toString(userInfo.getNumActiveApplications()))
> .td(Integer.toString(userInfo.getNumPendingApplications()))._();
> {code}
> Instead of amUsed.toString(), it should use userInfo.getAmUsed().
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7116) CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM usage.
[
https://issues.apache.org/jira/browse/YARN-7116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16144509#comment-16144509
]
Wangda Tan commented on YARN-7116:
--
Credit to Steven Brennan for reporting and reproducing the issue.
> CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM
> usage.
> -
>
> Key: YARN-7116
> URL: https://issues.apache.org/jira/browse/YARN-7116
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 2.9.0, 2.8.1, 3.0.0-alpha4
>Reporter: Wangda Tan
>Assignee: Wangda Tan
>
> On CapacityScheduler's web UI, AM usage of different users belong to the same
> queue always shows queue's AM usage.
> The root cause is: under CapacitySchedulerPage.
> {code}
> tbody.tr().td(userInfo.getUsername())
> .td(userInfo.getUserResourceLimit().toString())
> .td(resourcesUsed.toString())
> .td(resourceUsages.getAMLimit().toString())
> .td(amUsed.toString())
> .td(Integer.toString(userInfo.getNumActiveApplications()))
> .td(Integer.toString(userInfo.getNumPendingApplications()))._();
> {code}
> Instead of amUsed.toString(), it should use userInfo.getAmUsed().
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
