[
https://issues.apache.org/jira/browse/YARN-10353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17160272#comment-17160272
]
Hadoop QA commented on YARN-10353:
----------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 43s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 1m
22s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
20s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 38s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 22m
1s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 87m 2s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://builds.apache.org/job/PreCommit-YARN-Build/26289/artifact/out/Dockerfile
|
| JIRA Issue | YARN-10353 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/13007903/YARN-10353.002.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 16de36e1e524 4.15.0-101-generic #102-Ubuntu SMP Mon May 11
10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / 2ba44a73bf2 |
| Default Java | Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/26289/testReport/ |
| Max. process+thread count | 344 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/26289/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
> Log vcores used and cumulative cpu in containers monitor
> --------------------------------------------------------
>
> Key: YARN-10353
> URL: https://issues.apache.org/jira/browse/YARN-10353
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: yarn
> Affects Versions: 3.4.0
> Reporter: Jim Brennan
> Assignee: Jim Brennan
> Priority: Minor
> Attachments: YARN-10353.001.patch, YARN-10353.002.patch
>
>
> We currently log the percentage/cpu and percentage/cpus-used-by-yarn in the
> Containers Monitor log. It would be useful to also log vcores used vs vcores
> assigned, and total accumulated CPU time.
> For example, currently we have an audit log that looks like this:
> {noformat}
> 2020-07-16 20:33:51,550 DEBUG [Container Monitor] ContainersMonitorImpl.audit
> (ContainersMonitorImpl.java:recordUsage(651)) - Resource usage of ProcessTree
> 809 for container-id container_1594931466123_0002_01_000007: 309.5 MB of 2 GB
> physical memory used; 2.8 GB of 4.2 GB virtual memory used CPU:143.0905
> CPU/core:35.772625
> {noformat}
> The proposal is to add two more fields to show vCores and Cumulative CPU ms:
> {noformat}
> 2020-07-16 20:33:51,550 DEBUG [Container Monitor] ContainersMonitorImpl.audit
> (ContainersMonitorImpl.java:recordUsage(651)) - Resource usage of ProcessTree
> 809 for container-id container_1594931466123_0002_01_000007: 309.5 MB of 2 GB
> physical memory used; 2.8 GB of 4.2 GB virtual memory used CPU:143.0905
> CPU/core:35.772625 vCores:2/1 CPU-ms:4180
> {noformat}
> This is a snippet of a log from one of our clusters running branch-2.8 with a
> similar change.
> {noformat}
> 2020-07-16 21:00:02,240 [Container Monitor] DEBUG
> ContainersMonitorImpl.audit: Memory usage of ProcessTree 5267 for
> container-id container_e04_1594079801456_1397450_01_001992: 1.6 GB of 2.5 GB
> physical memory used; 3.8 GB of 5.3 GB virtual memory used. CPU usage: 18 of
> 10 CPU vCores used. Cumulative CPU time: 157410
> 2020-07-16 21:00:02,269 [Container Monitor] DEBUG
> ContainersMonitorImpl.audit: Memory usage of ProcessTree 18801 for
> container-id container_e04_1594079801456_1390375_01_000019: 413.2 MB of 2.5
> GB physical memory used; 3.8 GB of 5.3 GB virtual memory used. CPU usage: 0
> of 10 CPU vCores used. Cumulative CPU time: 113830
> 2020-07-16 21:00:02,298 [Container Monitor] DEBUG
> ContainersMonitorImpl.audit: Memory usage of ProcessTree 5279 for
> container-id container_e04_1594079801456_1397450_01_001991: 2.2 GB of 2.5 GB
> physical memory used; 3.8 GB of 5.3 GB virtual memory used. CPU usage: 17 of
> 10 CPU vCores used. Cumulative CPU time: 128630
> 2020-07-16 21:00:02,339 [Container Monitor] DEBUG
> ContainersMonitorImpl.audit: Memory usage of ProcessTree 24189 for
> container-id container_e04_1594079801456_1390430_01_000415: 392.7 MB of 2.5
> GB physical memory used; 3.8 GB of 5.3 GB virtual memory used. CPU usage: 0
> of 10 CPU vCores used. Cumulative CPU time: 96060
> 2020-07-16 21:00:02,367 [Container Monitor] DEBUG
> ContainersMonitorImpl.audit: Memory usage of ProcessTree 6751 for
> container-id container_e04_1594079801456_1397923_01_003248: 1.3 GB of 3 GB
> physical memory used; 4.3 GB of 6.3 GB virtual memory used. CPU usage: 12 of
> 10 CPU vCores used. Cumulative CPU time: 116820
> 2020-07-16 21:00:02,396 [Container Monitor] DEBUG
> ContainersMonitorImpl.audit: Memory usage of ProcessTree 12138 for
> container-id container_e04_1594079801456_1397760_01_000044: 4.4 GB of 6 GB
> physical memory used; 6.9 GB of 12.6 GB virtual memory used. CPU usage: 15 of
> 10 CPU vCores used. Cumulative CPU time: 45900
> 2020-07-16 21:00:02,424 [Container Monitor] DEBUG
> ContainersMonitorImpl.audit: Memory usage of ProcessTree 101918 for
> container-id container_e04_1594079801456_1391130_01_002378: 2.4 GB of 4 GB
> physical memory used; 5.8 GB of 8.4 GB virtual memory used. CPU usage: 13 of
> 10 CPU vCores used. Cumulative CPU time: 2572390
> 2020-07-16 21:00:02,456 [Container Monitor] DEBUG
> ContainersMonitorImpl.audit: Memory usage of ProcessTree 26596 for
> container-id container_e04_1594079801456_1390446_01_000665: 418.6 MB of 2.5
> GB physical memory used; 3.8 GB of 5.3 GB virtual memory used. CPU usage: 0
> of 10 CPU vCores used. Cumulative CPU time: 101210
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
