[jira] [Commented] (YARN-5094) some YARN container events have timestamp of -1
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16332736#comment-16332736 ] Hudson commented on YARN-5094: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13525 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/13525/]) YARN-5094. some YARN container events have timestamp of -1. (haibochen: rev 4aca4ff759f773135f8a27dbaa9731196fac5233) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/application/TestApplication.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/event/LocalizationEvent.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerEvent.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/application/ApplicationEvent.java > some YARN container events have timestamp of -1 > --- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: 2.9.0, 3.0.0 >Reporter: Sangjin Lee >Assignee: Haibo Chen >Priority: Critical > Labels: YARN-5355 > Attachments: YARN-5094-YARN-2928.001.patch, YARN-5094.00.patch, > YARN-5094.02.patch > > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5094) some YARN container events have timestamp of -1
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16332604#comment-16332604 ] Haibo Chen commented on YARN-5094: -- Thanks [~rohithsharma] for the review. Will commit it shortly. > some YARN container events have timestamp of -1 > --- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: 2.9.0, 3.0.0 >Reporter: Sangjin Lee >Assignee: Haibo Chen >Priority: Critical > Labels: YARN-5355 > Attachments: YARN-5094-YARN-2928.001.patch, YARN-5094.00.patch, > YARN-5094.02.patch > > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5094) some YARN container events have timestamp of -1
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16332598#comment-16332598 ] Rohith Sharma K S commented on YARN-5094: - I cross confirmed that it only happens for NM publishing events. All the entities published by RM are associated with timestamp. I will be able to commit it only tomorrow! [~haibochen] would you like to go ahead committing it today? > some YARN container events have timestamp of -1 > --- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: 2.9.0, 3.0.0 >Reporter: Sangjin Lee >Assignee: Haibo Chen >Priority: Critical > Labels: YARN-5355 > Attachments: YARN-5094-YARN-2928.001.patch, YARN-5094.00.patch, > YARN-5094.02.patch > > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5094) some YARN container events have timestamp of -1
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16332573#comment-16332573 ] Rohith Sharma K S commented on YARN-5094: - Yes, we need this to get in. Patch looks fine to me. I am just wondering is this only in NM events or should we modify in RM events as well? > some YARN container events have timestamp of -1 > --- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: 2.9.0, 3.0.0 >Reporter: Sangjin Lee >Assignee: Haibo Chen >Priority: Critical > Labels: YARN-5355 > Attachments: YARN-5094-YARN-2928.001.patch, YARN-5094.00.patch, > YARN-5094.02.patch > > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5094) some YARN container events have timestamp of -1
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16332482#comment-16332482 ] Haibo Chen commented on YARN-5094: -- +1 on fixing it. [~rohithsharma] [~sunilg] [~vrushalic] [~varun_saxena] want to take a look at the latest patch. > some YARN container events have timestamp of -1 > --- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: 2.9.0, 3.0.0 >Reporter: Sangjin Lee >Assignee: Haibo Chen >Priority: Critical > Labels: YARN-5355 > Attachments: YARN-5094-YARN-2928.001.patch, YARN-5094.00.patch, > YARN-5094.02.patch > > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5094) some YARN container events have timestamp of -1
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16331994#comment-16331994 ] genericqa commented on YARN-5094: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 11s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 49s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 21s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 33s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 10m 12s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 46s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 22s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 30s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 44s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 44s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 17s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 29s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 9m 39s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 48s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 17s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 19m 5s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 19s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 60m 37s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 | | JIRA Issue | YARN-5094 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12871021/YARN-5094.02.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux e968668aff69 4.4.0-64-generic #85-Ubuntu SMP Mon Feb 20 11:50:30 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 9e4f52d | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_151 | | findbugs | v3.1.0-RC1 | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/19347/testReport/ | | Max. process+thread count | 408 (vs. ulimit of 5000) | | modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/19347/console | | Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > some YARN container events have timestamp of -1 >
[jira] [Commented] (YARN-5094) some YARN container events have timestamp of -1
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16331894#comment-16331894 ] Sunil G commented on YARN-5094: --- Container finished time is coming as -1 when new UI pulls container data from ATS v2. This needs to be fixed. > some YARN container events have timestamp of -1 > --- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: 2.9.0, 3.0.0 >Reporter: Sangjin Lee >Assignee: Haibo Chen >Priority: Critical > Labels: YARN-5355 > Attachments: YARN-5094-YARN-2928.001.patch, YARN-5094.00.patch, > YARN-5094.02.patch > > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5094) some YARN container events have timestamp of -1
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16133989#comment-16133989 ] Hadoop QA commented on YARN-5094: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 24s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m 35s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 21s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 33s{color} | {color:green} trunk passed {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 53s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager in trunk has 1 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 22s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 32s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 48s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 48s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 19s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 33s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 59s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 17s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 14m 51s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 20s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 40m 0s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:14b5c93 | | JIRA Issue | YARN-5094 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12871021/YARN-5094.02.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux d60dd45bde23 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 436c263 | | Default Java | 1.8.0_144 | | findbugs | v3.1.0-RC1 | | findbugs | https://builds.apache.org/job/PreCommit-YARN-Build/17009/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/17009/testReport/ | | modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/17009/console | | Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > some YARN container events have timestamp of -1 > --- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issu
[jira] [Commented] (YARN-5094) some YARN container events have timestamp of -1
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16035301#comment-16035301 ] Haibo Chen commented on YARN-5094: -- Will look into it and follow up with a new patch > some YARN container events have timestamp of -1 > --- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Haibo Chen > Labels: YARN-5355 > Attachments: YARN-5094.00.patch, YARN-5094.02.patch, > YARN-5094-YARN-2928.001.patch > > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5094) some YARN container events have timestamp of -1
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16035296#comment-16035296 ] Vrushali C commented on YARN-5094: -- Thanks [~haibochen] for the patch. Do you think we might want to put it in these too: containermanager/AuxServicesEvent.java containermanager/container/ContainerEvent.java containermanager/launcher/ContainersLauncherEvent.java containermanager/localizer/sharedcache/SharedCacheUploadEvent.java containermanager/monitor/ContainersMonitorEvent.java containermanager/scheduler/ContainerSchedulerEvent.java Do you think we can add a basic test in TestTimelineReaderWebServices for this, if it's not too cumbersome to add. > some YARN container events have timestamp of -1 > --- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Haibo Chen > Labels: YARN-5355 > Attachments: YARN-5094.00.patch, YARN-5094.02.patch, > YARN-5094-YARN-2928.001.patch > > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5094) some YARN container events have timestamp of -1
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16035282#comment-16035282 ] Haibo Chen commented on YARN-5094: -- Findbug warnings are unrelated. > some YARN container events have timestamp of -1 > --- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Haibo Chen > Labels: YARN-5355 > Attachments: YARN-5094.00.patch, YARN-5094.02.patch, > YARN-5094-YARN-2928.001.patch > > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5094) some YARN container events have timestamp of -1
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16035259#comment-16035259 ] Hadoop QA commented on YARN-5094: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 22m 28s{color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 33s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 19s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 32s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 18s{color} | {color:green} trunk passed {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 50s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager in trunk has 5 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 21s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 28s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 28s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 28s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 17s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 27s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 14s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 59s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 17s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 14m 29s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 26s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 60m 31s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:14b5c93 | | JIRA Issue | YARN-5094 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12871021/YARN-5094.02.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 32f3886933f8 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 73ecb19 | | Default Java | 1.8.0_131 | | findbugs | v3.1.0-RC1 | | findbugs | https://builds.apache.org/job/PreCommit-YARN-Build/16090/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/16090/testReport/ | | modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/16090/console | | Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > some YARN container events have timestamp of -1 > --- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN
[jira] [Commented] (YARN-5094) some YARN container events have timestamp of -1
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16035116#comment-16035116 ] Haibo Chen commented on YARN-5094: -- Will update the patch to address the unit test failures. > some YARN container events have timestamp of -1 > --- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Haibo Chen > Labels: YARN-5355 > Attachments: YARN-5094.00.patch, YARN-5094-YARN-2928.001.patch > > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5094) some YARN container events have timestamp of -1
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16033935#comment-16033935 ] Haibo Chen commented on YARN-5094: -- Thanks [~gtCarrera9] for your comments. Yes, I have limited the change to only events that we are interested in exposing to ATSv2 for the exact same reason that you pointed. > some YARN container events have timestamp of -1 > --- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Haibo Chen > Labels: YARN-5355 > Attachments: YARN-5094.00.patch, YARN-5094-YARN-2928.001.patch > > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5094) some YARN container events have timestamp of -1
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16033917#comment-16033917 ] Hadoop QA commented on YARN-5094: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s{color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s{color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m 28s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 28s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 17s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 27s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 16s{color} | {color:green} trunk passed {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 42s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager in trunk has 5 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 17s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 24s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 33s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 17s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 27s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 15s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 59s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 12m 56s{color} | {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 20s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 35m 7s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.yarn.server.nodemanager.containermanager.application.TestApplication | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:14b5c93 | | JIRA Issue | YARN-5094 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12870878/YARN-5094.00.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 8490e8ffcae4 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 7101477 | | Default Java | 1.8.0_131 | | findbugs | v3.1.0-RC1 | | findbugs | https://builds.apache.org/job/PreCommit-YARN-Build/16075/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html | | unit | https://builds.apache.org/job/PreCommit-YARN-Build/16075/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/16075/testReport/ | | modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/h
[jira] [Commented] (YARN-5094) some YARN container events have timestamp of -1
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16033890#comment-16033890 ] Li Lu commented on YARN-5094: - Sorry about the delay. Please feel free to take it. Let's not touch AbstractEvent as a whole but treat different events separately? Also, even for NM related events we should be careful about the actual performance. I barely remember that my last conclusion (a year ago) was it's fine (to assign a timestamp for NM events), but let's be careful. > some YARN container events have timestamp of -1 > --- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Haibo Chen > Labels: YARN-5355 > Attachments: YARN-5094.00.patch, YARN-5094-YARN-2928.001.patch > > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5094) some YARN container events have timestamp of -1
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16033878#comment-16033878 ] Haibo Chen commented on YARN-5094: -- Sorry, I just realized that AbstractEvent is used in everywhere. To be conservative, I think we can set the timestamp for events only when we actually need the timestamps (e.g. events exposed in ATSv2) Thus, I am uploading a patch here to only change NM container/localization/application event timestamps. > some YARN container events have timestamp of -1 > --- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Haibo Chen > Labels: YARN-5355 > Attachments: YARN-5094-YARN-2928.001.patch > > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5094) some YARN container events have timestamp of -1 in REST output
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16033262#comment-16033262 ] Haibo Chen commented on YARN-5094: -- As indicated in the comments above, YARN-5169 is the real cause. I am planning to fix the issue there. > some YARN container events have timestamp of -1 in REST output > -- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Haibo Chen > Labels: YARN-5355 > Attachments: YARN-5094-YARN-2928.001.patch > > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5094) some YARN container events have timestamp of -1 in REST output
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16032366#comment-16032366 ] Rohith Sharma K S commented on YARN-5094: - [~haibo.chen] thanks for finding this JIRA. Would you rebase the patch ? > some YARN container events have timestamp of -1 in REST output > -- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Haibo Chen > Labels: YARN-5355 > Attachments: YARN-5094-YARN-2928.001.patch > > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5094) some YARN container events have timestamp of -1 in REST output
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003537#comment-16003537 ] Haibo Chen commented on YARN-5094: -- [~gtCarrera9] Do you mind if I take over? I am looking at YARN-5169, which is the real cause. > some YARN container events have timestamp of -1 in REST output > -- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Li Lu > Labels: YARN-5355 > Attachments: YARN-5094-YARN-2928.001.patch > > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5094) some YARN container events have timestamp of -1 in REST output
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317326#comment-15317326 ] Sangjin Lee commented on YARN-5094: --- I don't believe this should block the merge. Please let me know if you disagree. > some YARN container events have timestamp of -1 in REST output > -- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Li Lu > Labels: yarn-2928-1st-milestone > Attachments: YARN-5094-YARN-2928.001.patch > > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5094) some YARN container events have timestamp of -1 in REST output
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15303432#comment-15303432 ] Sangjin Lee commented on YARN-5094: --- When YARN-5169 is fixed, we can pick it up by rebasing with the trunk. Since RM's {{SystemMetricsPublisher}} is setting the timestamp by hand for the timeline event, we should probably back that out once YARN-5169 is fixed. > some YARN container events have timestamp of -1 in REST output > -- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Li Lu > Labels: yarn-2928-1st-milestone > Attachments: YARN-5094-YARN-2928.001.patch > > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5094) some YARN container events have timestamp of -1 in REST output
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15303431#comment-15303431 ] Sangjin Lee commented on YARN-5094: --- Filed YARN-5169. We discussed this offline, and I believe this is an existing trunk issue and is best addressed by fixing it in the trunk. > some YARN container events have timestamp of -1 in REST output > -- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Li Lu > Labels: yarn-2928-1st-milestone > Attachments: YARN-5094-YARN-2928.001.patch > > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5094) some YARN container events have timestamp of -1 in REST output
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302668#comment-15302668 ] Hadoop QA commented on YARN-5094: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 16m 42s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s {color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 29s {color} | {color:green} YARN-2928 passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 22s {color} | {color:green} YARN-2928 passed with JDK v1.8.0_91 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 27s {color} | {color:green} YARN-2928 passed with JDK v1.7.0_101 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 18s {color} | {color:green} YARN-2928 passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 29s {color} | {color:green} YARN-2928 passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 14s {color} | {color:green} YARN-2928 passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 50s {color} | {color:green} YARN-2928 passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s {color} | {color:green} YARN-2928 passed with JDK v1.8.0_91 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 22s {color} | {color:green} YARN-2928 passed with JDK v1.7.0_101 {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 24s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 19s {color} | {color:green} the patch passed with JDK v1.8.0_91 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 19s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 23s {color} | {color:green} the patch passed with JDK v1.7.0_101 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 23s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 15s {color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: patch generated 1 new + 125 unchanged - 1 fixed = 126 total (was 126) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 26s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 11s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} Patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 0s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 15s {color} | {color:green} the patch passed with JDK v1.8.0_91 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 20s {color} | {color:green} the patch passed with JDK v1.7.0_101 {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 13m 10s {color} | {color:red} hadoop-yarn-server-nodemanager in the patch failed with JDK v1.8.0_91. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 13m 46s {color} | {color:red} hadoop-yarn-server-nodemanager in the patch failed with JDK v1.7.0_101. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 19s {color} | {color:green} Patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 59m 23s {color} | {color:black} {color} | \\ \\ || Reason || Tests || | JDK v1.8.0_91 Failed junit tests | hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager | | JDK v1.7.0_101 Failed junit tests | hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:cf2e
[jira] [Commented] (YARN-5094) some YARN container events have timestamp of -1 in REST output
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302514#comment-15302514 ] Varun Saxena commented on YARN-5094: Personally I would have slightly preferred timestamps to be filled in NMTimelinePublisher code. That would mean we fill timestamps where we need. But then it is unlikely somebody will change the behavior. And we will have a test case too to verify it. As such the code looks fine to me. Changes in NMTimelinePublisher are not necessary by the way. > some YARN container events have timestamp of -1 in REST output > -- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Li Lu > Labels: yarn-2928-1st-milestone > Attachments: YARN-5094-YARN-2928.001.patch > > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5094) some YARN container events have timestamp of -1 in REST output
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15297290#comment-15297290 ] Sangjin Lee commented on YARN-5094: --- Yes, I'm also slightly leaning towards populating the timestamp within the timeline service code rather than changing {{ContainerEvent}} and so on. I can't really think why populating the timestamp instead of -1 would be an issue, but still... > some YARN container events have timestamp of -1 in REST output > -- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Li Lu > Labels: yarn-2928-1st-milestone > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5094) some YARN container events have timestamp of -1 in REST output
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296707#comment-15296707 ] Li Lu commented on YARN-5094: - bq. We can either populate the timestamp in ContainerEvent/ApplicationEvent/LocalizationEvent or take current system time and populate the timestamp in publish***Event methods(or other publish methods) in NMTimelinePublisher as these are direct calls from ContainerManagerImpl/ContainersMonitorImpl. Yes. I'm relatively conservative on the first approach, since there are notes in AbstractEvent saying that: {code} // We're not generating a real timestamp here. It's too expensive. {code} Having seen this and also the SMP design in RM I think passing in timestamp would be a better approach? > some YARN container events have timestamp of -1 in REST output > -- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Li Lu > Labels: yarn-2928-1st-milestone > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5094) some YARN container events have timestamp of -1 in REST output
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296074#comment-15296074 ] Varun Saxena commented on YARN-5094: You are correct. I think timestamp needs to be added for application and localization events as well. We can either populate the timestamp in ContainerEvent/ApplicationEvent/LocalizationEvent or take current system time and populate the timestamp in publish***Event methods(or other publish methods) in NMTimelinePublisher as these are direct calls from ContainerManagerImpl/ContainersMonitorImpl. > some YARN container events have timestamp of -1 in REST output > -- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Li Lu > Labels: yarn-2928-1st-milestone > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5094) some YARN container events have timestamp of -1 in REST output
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294486#comment-15294486 ] Li Lu commented on YARN-5094: - OK, seems like we missed something here: We use the incoming container event's timestamp as our event timestamp. However, {{ContainerEvent}} does not contain valid event timestamp since it's created via AbstractEvent(type), where timestamp is always set to -1: {code} // use this if you DON'T care about the timestamp public AbstractEvent(TYPE type) { this.type = type; // We're not generating a real timestamp here. It's too expensive. timestamp = -1L; } {code} However, we do have the timestamp in RM SMP. This caused the difference. I'm working on a fix now... Probably we need to add timestamp support to {{ContainerEvent}}s so that we can use this information? > some YARN container events have timestamp of -1 in REST output > -- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Li Lu > Labels: yarn-2928-1st-milestone > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5094) some YARN container events have timestamp of -1 in REST output
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15286092#comment-15286092 ] Li Lu commented on YARN-5094: - Found same issue when working on the UI stuff. Let me take a look at it. > some YARN container events have timestamp of -1 in REST output > -- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Li Lu > Labels: yarn-2928-1st-milestone > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org