[jira] [Commented] (YARN-4983) JVM and UGI metrics disappear after RM is once transitioned to standby mode
[ https://issues.apache.org/jira/browse/YARN-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15259512#comment-15259512 ] Hudson commented on YARN-4983: -- FAILURE: Integrated in Hadoop-trunk-Commit #9678 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/9678/]) YARN-4983. JVM and UGI metrics disappear after RM transitioned to (jianhe: rev 4beff013546dbc29c004a1bad1b019dc9d2b751f) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/source/JvmMetrics.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMHAMetrics.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java > JVM and UGI metrics disappear after RM is once transitioned to standby mode > --- > > Key: YARN-4983 > URL: https://issues.apache.org/jira/browse/YARN-4983 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Li Lu >Assignee: Li Lu > Fix For: 2.9.0 > > Attachments: YARN-4983-trunk.000.patch, YARN-4983-trunk.001.patch, > YARN-4983-trunk.002.patch, YARN-4983-trunk.003.patch > > > When get transitioned to standby, the RM will shutdown the existing metric > system and relaunch a new one. This will cause the jvm metrics and ugi > metrics to miss in the new metric system. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4983) JVM and UGI metrics disappear after RM is once transitioned to standby mode
[
https://issues.apache.org/jira/browse/YARN-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15259314#comment-15259314
]
Hadoop QA commented on YARN-4983:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 8s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 14s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
57s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 36s
{color} | {color:green} trunk passed with JDK v1.8.0_92 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 4s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 37s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
31s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
45s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 19s
{color} | {color:green} trunk passed with JDK v1.8.0_92 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 33s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 14s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 34s
{color} | {color:green} the patch passed with JDK v1.8.0_92 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 7s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m 7s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m 6s
{color} | {color:red} root: patch generated 1 new + 172 unchanged - 1 fixed =
173 total (was 173) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 33s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 4s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 19s
{color} | {color:green} the patch passed with JDK v1.8.0_92 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 28s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m 54s
{color} | {color:green} hadoop-common in the patch passed with JDK v1.8.0_92.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 77m 31s {color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK
v1.8.0_92. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m 28s
{color} | {color:green} hadoop-common in the patch passed with JDK v1.7.0_95.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 77m 31s {color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK
v1.7.0_95. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 227m 9s {color}
[jira] [Commented] (YARN-4983) JVM and UGI metrics disappear after RM is once transitioned to standby mode
[ https://issues.apache.org/jira/browse/YARN-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15258960#comment-15258960 ] Li Lu commented on YARN-4983: - bq. One pedantic little comment: lest we work against YARN-4977, can we please include a message on the throws tag in the javadoc headers? Sure. I'll update that part. bq. Also, it looks like the test tests the transition from active to standby, but not back to active. Any value in adding that last step? Not necessarily? The problem happens when we try to reinitialize the RM active service. This step happens when we call transitionToStandBy. The actual symptom also happens when the RM becomes standby from active. > JVM and UGI metrics disappear after RM is once transitioned to standby mode > --- > > Key: YARN-4983 > URL: https://issues.apache.org/jira/browse/YARN-4983 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Li Lu >Assignee: Li Lu > Attachments: YARN-4983-trunk.000.patch, YARN-4983-trunk.001.patch, > YARN-4983-trunk.002.patch > > > When get transitioned to standby, the RM will shutdown the existing metric > system and relaunch a new one. This will cause the jvm metrics and ugi > metrics to miss in the new metric system. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4983) JVM and UGI metrics disappear after RM is once transitioned to standby mode
[ https://issues.apache.org/jira/browse/YARN-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15258944#comment-15258944 ] Daniel Templeton commented on YARN-4983: One pedantic little comment: lest we work against YARN-4977, can we please include a message on the throws tag in the javadoc headers? Also, it looks like the test tests the transition from active to standby, but not back to active. Any value in adding that last step? > JVM and UGI metrics disappear after RM is once transitioned to standby mode > --- > > Key: YARN-4983 > URL: https://issues.apache.org/jira/browse/YARN-4983 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Li Lu >Assignee: Li Lu > Attachments: YARN-4983-trunk.000.patch, YARN-4983-trunk.001.patch, > YARN-4983-trunk.002.patch > > > When get transitioned to standby, the RM will shutdown the existing metric > system and relaunch a new one. This will cause the jvm metrics and ugi > metrics to miss in the new metric system. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4983) JVM and UGI metrics disappear after RM is once transitioned to standby mode
[ https://issues.apache.org/jira/browse/YARN-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15258938#comment-15258938 ] Li Lu commented on YARN-4983: - I thought [~templedf] uploaded a patch to another JIRA to this one and then deleted it? Just try to confirm... Thanks. > JVM and UGI metrics disappear after RM is once transitioned to standby mode > --- > > Key: YARN-4983 > URL: https://issues.apache.org/jira/browse/YARN-4983 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Li Lu >Assignee: Li Lu > Attachments: YARN-4983-trunk.000.patch, YARN-4983-trunk.001.patch, > YARN-4983-trunk.002.patch > > > When get transitioned to standby, the RM will shutdown the existing metric > system and relaunch a new one. This will cause the jvm metrics and ugi > metrics to miss in the new metric system. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4983) JVM and UGI metrics disappear after RM is once transitioned to standby mode
[ https://issues.apache.org/jira/browse/YARN-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15258930#comment-15258930 ] Junping Du commented on YARN-4983: -- Hi [~templedf], do you have more comments on the latest patch? If not, I will go ahead to commit this patch which is an important bug fix. > JVM and UGI metrics disappear after RM is once transitioned to standby mode > --- > > Key: YARN-4983 > URL: https://issues.apache.org/jira/browse/YARN-4983 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Li Lu >Assignee: Li Lu > Attachments: YARN-4983-trunk.000.patch, YARN-4983-trunk.001.patch, > YARN-4983-trunk.002.patch > > > When get transitioned to standby, the RM will shutdown the existing metric > system and relaunch a new one. This will cause the jvm metrics and ugi > metrics to miss in the new metric system. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4983) JVM and UGI metrics disappear after RM is once transitioned to standby mode
[ https://issues.apache.org/jira/browse/YARN-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15257749#comment-15257749 ] Junping Du commented on YARN-4983: -- bq. Due to the amount of work to refactor RM HA, maybe we want to resolve that in a separate JIRA? Sounds good. Will file a separate JIRA to track this. Three UT test failures are not related, HADOOP-12687 and YARN-4947 are filed before to track them. +1. Patch LGTM. Will commit it shortly if no further comments from others. > JVM and UGI metrics disappear after RM is once transitioned to standby mode > --- > > Key: YARN-4983 > URL: https://issues.apache.org/jira/browse/YARN-4983 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Li Lu >Assignee: Li Lu > Attachments: YARN-4983-trunk.000.patch, YARN-4983-trunk.001.patch, > YARN-4983-trunk.002.patch > > > When get transitioned to standby, the RM will shutdown the existing metric > system and relaunch a new one. This will cause the jvm metrics and ugi > metrics to miss in the new metric system. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4983) JVM and UGI metrics disappear after RM is once transitioned to standby mode
[ https://issues.apache.org/jira/browse/YARN-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15257168#comment-15257168 ] Li Lu commented on YARN-4983: - Could not reproduce the UT failures locally. At least two UT failures (TestClientRMTokens and TestAMAuthorization) also appears in other unrelated JIRAs, such as YARN-3150 (a documentation patch). > JVM and UGI metrics disappear after RM is once transitioned to standby mode > --- > > Key: YARN-4983 > URL: https://issues.apache.org/jira/browse/YARN-4983 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Li Lu >Assignee: Li Lu > Attachments: YARN-4983-trunk.000.patch, YARN-4983-trunk.001.patch, > YARN-4983-trunk.002.patch > > > When get transitioned to standby, the RM will shutdown the existing metric > system and relaunch a new one. This will cause the jvm metrics and ugi > metrics to miss in the new metric system. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4983) JVM and UGI metrics disappear after RM is once transitioned to standby mode
[
https://issues.apache.org/jira/browse/YARN-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15257107#comment-15257107
]
Hadoop QA commented on YARN-4983:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 6m 52s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 49s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
5s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 18s
{color} | {color:green} trunk passed with JDK v1.8.0_92 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 59s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
7s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 30s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
29s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
43s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 13s
{color} | {color:green} trunk passed with JDK v1.8.0_92 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 30s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 14s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
9s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 38s
{color} | {color:green} the patch passed with JDK v1.8.0_92 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m 38s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 32s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 32s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m 5s
{color} | {color:red} root: patch generated 1 new + 172 unchanged - 1 fixed =
173 total (was 173) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 28s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
29s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 4s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 21s
{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch failed
with JDK v1.8.0_92. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 27s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 6m 48s
{color} | {color:green} hadoop-common in the patch passed with JDK v1.8.0_92.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 76m 0s {color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK
v1.8.0_92. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m 3s
{color} | {color:green} hadoop-common in the patch passed with JDK v1.7.0_95.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 77m 10s {color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK
v1.7.0_95. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color}
[jira] [Commented] (YARN-4983) JVM and UGI metrics disappear after RM is once transitioned to standby mode
[
https://issues.apache.org/jira/browse/YARN-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256754#comment-15256754
]
Li Lu commented on YARN-4983:
-
Thanks for the review [~djp]! I totally agree that we need to refactor the code
of RM HA so that the code is more maintainable. For the boolean flag here,
{{initialize}} and {{fromActive}} are currently decoupled.
{{createAndInitActiveServices}} has two call sites, one in RM#serviceInit,
another in reinitialize. When reinitializing, we're using the flag
{{initialize}} to distinguish if we need to reinitialize the active service.
This method is called when we do a state transition to standby and active.
Therefore, we only need to set fromActive to true when we're reinitializing the
active services and is transitioning from active to standby. Due to the amount
of work to refactor RM HA, maybe we want to resolve that in a separate JIRA? We
can focus on a quick fix for the current symptom in this JIRA? Thanks!
BTW, I kicked another Jenkins run to see if there are any other UT problems.
> JVM and UGI metrics disappear after RM is once transitioned to standby mode
> ---
>
> Key: YARN-4983
> URL: https://issues.apache.org/jira/browse/YARN-4983
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Li Lu
>Assignee: Li Lu
> Attachments: YARN-4983-trunk.000.patch, YARN-4983-trunk.001.patch,
> YARN-4983-trunk.002.patch
>
>
> When get transitioned to standby, the RM will shutdown the existing metric
> system and relaunch a new one. This will cause the jvm metrics and ugi
> metrics to miss in the new metric system.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4983) JVM and UGI metrics disappear after RM is once transitioned to standby mode
[
https://issues.apache.org/jira/browse/YARN-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256481#comment-15256481
]
Junping Du commented on YARN-4983:
--
Thanks [~gtCarrera9] for reporting the issue and delivering the fix.
The patch looks good in overall. There are some questions and comments below.
I think our work here is trying to differentiate the metric
initialization/re-initialization with RM first time start with RM transition
from standby. Isn't it? If so, below code:
{noformat}
@@ -1114,7 +1127,7 @@ void reinitialize(boolean initialize) {
QueueMetrics.clearQueueMetrics();
if (initialize) {
resetDispatcher();
- createAndInitActiveServices();
+ createAndInitActiveServices(true);
}
}
{noformat}
Shall we reuse {{boolean initialize}} instead of set true directly? I saw there
are two caller path: one is from RM.serviceStart() and the other is from
transition to standby from active, marked by boolean value of initialize. I
think we should also differentiate the situation here?
In addition, I think our RM start logic is a bit redundant when RM HA is
enabled: we always do serviceInit() first, then serviceStart(). However, within
serviceStart(), we always put RM to standby mode first and do reinitialize
again. Any special reason that we cannot initialize once to be in standby mode?
CC [~jianhe], [~ka...@cloudera.com].
> JVM and UGI metrics disappear after RM is once transitioned to standby mode
> ---
>
> Key: YARN-4983
> URL: https://issues.apache.org/jira/browse/YARN-4983
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Li Lu
>Assignee: Li Lu
> Attachments: YARN-4983-trunk.000.patch, YARN-4983-trunk.001.patch,
> YARN-4983-trunk.002.patch
>
>
> When get transitioned to standby, the RM will shutdown the existing metric
> system and relaunch a new one. This will cause the jvm metrics and ugi
> metrics to miss in the new metric system.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4983) JVM and UGI metrics disappear after RM is once transitioned to standby mode
[
https://issues.apache.org/jira/browse/YARN-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15254865#comment-15254865
]
Hadoop QA commented on YARN-4983:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 15s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
51s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m 29s
{color} | {color:green} trunk passed with JDK v1.8.0_77 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m 8s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
11s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 45s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
31s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
53s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 39s
{color} | {color:green} trunk passed with JDK v1.8.0_77 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 43s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 16s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m 12s
{color} | {color:green} the patch passed with JDK v1.8.0_77 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m 12s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m 6s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m 6s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m 11s
{color} | {color:red} root: patch generated 1 new + 173 unchanged - 1 fixed =
174 total (was 174) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 39s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
30s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
26s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 31s
{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch failed
with JDK v1.8.0_77. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 40s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 10m 50s
{color} | {color:green} hadoop-common in the patch passed with JDK v1.8.0_77.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 83m 39s {color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK
v1.8.0_77. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 9m 55s {color}
| {color:red} hadoop-common in the patch failed with JDK v1.7.0_95. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 86m 46s {color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK
v1.7.0_95. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
36s {col
[jira] [Commented] (YARN-4983) JVM and UGI metrics disappear after RM is once transitioned to standby mode
[ https://issues.apache.org/jira/browse/YARN-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15254433#comment-15254433 ] Li Lu commented on YARN-4983: - Similar UT failures happened in HADOOP-12563. The patch now get reverted. I'm launching another Jenkins run for this JIRA. > JVM and UGI metrics disappear after RM is once transitioned to standby mode > --- > > Key: YARN-4983 > URL: https://issues.apache.org/jira/browse/YARN-4983 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Li Lu >Assignee: Li Lu > Attachments: YARN-4983-trunk.000.patch, YARN-4983-trunk.001.patch > > > When get transitioned to standby, the RM will shutdown the existing metric > system and relaunch a new one. This will cause the jvm metrics and ugi > metrics to miss in the new metric system. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4983) JVM and UGI metrics disappear after RM is once transitioned to standby mode
[ https://issues.apache.org/jira/browse/YARN-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15254238#comment-15254238 ] Daniel Templeton commented on YARN-4983: I meant there are unit tests that are tripping on the problems you're trying to fix. :) > JVM and UGI metrics disappear after RM is once transitioned to standby mode > --- > > Key: YARN-4983 > URL: https://issues.apache.org/jira/browse/YARN-4983 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Li Lu >Assignee: Li Lu > Attachments: YARN-4983-trunk.000.patch, YARN-4983-trunk.001.patch > > > When get transitioned to standby, the RM will shutdown the existing metric > system and relaunch a new one. This will cause the jvm metrics and ugi > metrics to miss in the new metric system. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4983) JVM and UGI metrics disappear after RM is once transitioned to standby mode
[ https://issues.apache.org/jira/browse/YARN-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15254222#comment-15254222 ] Li Lu commented on YARN-4983: - Thanks [~templedf]! bq. Some of the unit tests that trip on it are only creating disembodied schedulers. Did you mean the failures for the first time, or from the second patch? I'm trying to understand with the second groups of failures, with something complaining from the protobuf level... > JVM and UGI metrics disappear after RM is once transitioned to standby mode > --- > > Key: YARN-4983 > URL: https://issues.apache.org/jira/browse/YARN-4983 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Li Lu >Assignee: Li Lu > Attachments: YARN-4983-trunk.000.patch, YARN-4983-trunk.001.patch > > > When get transitioned to standby, the RM will shutdown the existing metric > system and relaunch a new one. This will cause the jvm metrics and ugi > metrics to miss in the new metric system. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4983) JVM and UGI metrics disappear after RM is once transitioned to standby mode
[ https://issues.apache.org/jira/browse/YARN-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15254206#comment-15254206 ] Daniel Templeton commented on YARN-4983: This also comes up as an issue with some unit tests. I agree that the root issue is the erroneous assumption that the RM and scheduler won't be started a second time from within the same VM. Ideally we'd fix it below the level or the RM. Some of the unit tests that trip on it are only creating disembodied schedulers. > JVM and UGI metrics disappear after RM is once transitioned to standby mode > --- > > Key: YARN-4983 > URL: https://issues.apache.org/jira/browse/YARN-4983 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Li Lu >Assignee: Li Lu > Attachments: YARN-4983-trunk.000.patch, YARN-4983-trunk.001.patch > > > When get transitioned to standby, the RM will shutdown the existing metric > system and relaunch a new one. This will cause the jvm metrics and ugi > metrics to miss in the new metric system. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4983) JVM and UGI metrics disappear after RM is once transitioned to standby mode
[
https://issues.apache.org/jira/browse/YARN-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15253535#comment-15253535
]
Hadoop QA commented on YARN-4983:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 16s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 15s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
34s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 38s
{color} | {color:green} trunk passed with JDK v1.8.0_77 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 37s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 31s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
29s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
42s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 14s
{color} | {color:green} trunk passed with JDK v1.8.0_77 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 33s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 15s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 36s
{color} | {color:green} the patch passed with JDK v1.8.0_77 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 37s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 37s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m 7s
{color} | {color:red} root: patch generated 1 new + 173 unchanged - 1 fixed =
174 total (was 174) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 31s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
29s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 8s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 22s
{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch failed
with JDK v1.8.0_77. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 32s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 6m 45s {color}
| {color:red} hadoop-common in the patch failed with JDK v1.8.0_77. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 80m 3s {color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK
v1.8.0_77. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 13s {color}
| {color:red} hadoop-common in the patch failed with JDK v1.7.0_95. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 81m 37s {color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK
v1.7.0_95. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
33s {color} | {colo
[jira] [Commented] (YARN-4983) JVM and UGI metrics disappear after RM is once transitioned to standby mode
[
https://issues.apache.org/jira/browse/YARN-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15253188#comment-15253188
]
Hadoop QA commented on YARN-4983:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 11s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 15s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
37s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 1s
{color} | {color:green} trunk passed with JDK v1.8.0_77 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 49s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 32s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
40s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 15s
{color} | {color:green} trunk passed with JDK v1.8.0_77 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 32s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 14s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 7s
{color} | {color:green} the patch passed with JDK v1.8.0_77 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 7s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 43s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 43s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m 4s
{color} | {color:red} root: patch generated 2 new + 172 unchanged - 1 fixed =
174 total (was 173) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 30s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 5s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 22s
{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch failed
with JDK v1.8.0_77. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 32s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 43s {color}
| {color:red} hadoop-common in the patch failed with JDK v1.8.0_77. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 68m 27s {color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK
v1.8.0_77. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m 34s
{color} | {color:green} hadoop-common in the patch passed with JDK v1.7.0_95.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 69m 22s {color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK
v1.7.0_95. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s {color}
