[jira] [Updated] (YARN-3046) [Event producers] Implement MapReduce AM writing some MR metrics to ATS
[ https://issues.apache.org/jira/browse/YARN-3046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3046: - Attachment: YARN-3046-v5.patch Update patch to address latest comments from [~sjlee0], verify it works fine end-2-end. > [Event producers] Implement MapReduce AM writing some MR metrics to ATS > --- > > Key: YARN-3046 > URL: https://issues.apache.org/jira/browse/YARN-3046 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Sangjin Lee >Assignee: Junping Du > Attachments: YARN-3046-no-test-v2.patch, YARN-3046-no-test.patch, > YARN-3046-v1-rebase.patch, YARN-3046-v1.patch, YARN-3046-v2.patch, > YARN-3046-v3.patch, YARN-3046-v4.patch, YARN-3046-v5.patch > > > Per design in YARN-2928, select a handful of MR metrics (e.g. HDFS bytes > written) and have the MR AM write the framework-specific metrics to ATS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3046) [Event producers] Implement MapReduce AM writing some MR metrics to ATS
[ https://issues.apache.org/jira/browse/YARN-3046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3046: - Attachment: YARN-3046-v4.patch Upload a patch against v3 with removing changes in TimelineEvent. > [Event producers] Implement MapReduce AM writing some MR metrics to ATS > --- > > Key: YARN-3046 > URL: https://issues.apache.org/jira/browse/YARN-3046 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Sangjin Lee >Assignee: Junping Du > Attachments: YARN-3046-no-test-v2.patch, YARN-3046-no-test.patch, > YARN-3046-v1-rebase.patch, YARN-3046-v1.patch, YARN-3046-v2.patch, > YARN-3046-v3.patch, YARN-3046-v4.patch > > > Per design in YARN-2928, select a handful of MR metrics (e.g. HDFS bytes > written) and have the MR AM write the framework-specific metrics to ATS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3046) [Event producers] Implement MapReduce AM writing some MR metrics to ATS
[ https://issues.apache.org/jira/browse/YARN-3046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3046: - Attachment: YARN-3046-v3.patch Incorporate [~zjshen] and [~rkanter]'s comments in v3 patch! Also, identify a NPE issue in previous patch for MiniMRYarnCluster (if not setting auxiliary service explicitly). Verify related tests can pass. > [Event producers] Implement MapReduce AM writing some MR metrics to ATS > --- > > Key: YARN-3046 > URL: https://issues.apache.org/jira/browse/YARN-3046 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Sangjin Lee >Assignee: Junping Du > Attachments: YARN-3046-no-test-v2.patch, YARN-3046-no-test.patch, > YARN-3046-v1-rebase.patch, YARN-3046-v1.patch, YARN-3046-v2.patch, > YARN-3046-v3.patch > > > Per design in YARN-2928, select a handful of MR metrics (e.g. HDFS bytes > written) and have the MR AM write the framework-specific metrics to ATS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3046) [Event producers] Implement MapReduce AM writing some MR metrics to ATS
[ https://issues.apache.org/jira/browse/YARN-3046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3046: - Attachment: YARN-3046-v2.patch Update patch to v2 for incorporate comments from [~zjshen] and [~rkanter]. Filed YARN-3488 against YARN-2928 to address AM get timeline service version/enable info from RM via RPC call. Filed MAPREDUCE-6318 to refactor JobHistoryEventHandler processing on timeline v1 entity after we merge YARN-2928 to trunk and branch-2. > [Event producers] Implement MapReduce AM writing some MR metrics to ATS > --- > > Key: YARN-3046 > URL: https://issues.apache.org/jira/browse/YARN-3046 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Sangjin Lee >Assignee: Junping Du > Attachments: YARN-3046-no-test-v2.patch, YARN-3046-no-test.patch, > YARN-3046-v1-rebase.patch, YARN-3046-v1.patch, YARN-3046-v2.patch > > > Per design in YARN-2928, select a handful of MR metrics (e.g. HDFS bytes > written) and have the MR AM write the framework-specific metrics to ATS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3046) [Event producers] Implement MapReduce AM writing some MR metrics to ATS
[ https://issues.apache.org/jira/browse/YARN-3046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3046: - Attachment: YARN-3046-no-test-v2.patch Forget to add code for finding of TimelineCollectorAddress in previous patch, add it in v2. End 2 end test is still missing in this patch here because some existing tests - TestMRTimelineEventHandling get failed locally even without applying this patch. Still in digging out the issue. > [Event producers] Implement MapReduce AM writing some MR metrics to ATS > --- > > Key: YARN-3046 > URL: https://issues.apache.org/jira/browse/YARN-3046 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Sangjin Lee >Assignee: Junping Du > Attachments: YARN-3046-no-test-v2.patch, YARN-3046-no-test.patch > > > Per design in YARN-2928, select a handful of MR metrics (e.g. HDFS bytes > written) and have the MR AM write the framework-specific metrics to ATS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3046) [Event producers] Implement MapReduce AM writing some MR metrics to ATS
[ https://issues.apache.org/jira/browse/YARN-3046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3046: - Attachment: YARN-3046-no-test.patch Upload the first patch - not including any test yet. Call for a early review. Will add tests soon. > [Event producers] Implement MapReduce AM writing some MR metrics to ATS > --- > > Key: YARN-3046 > URL: https://issues.apache.org/jira/browse/YARN-3046 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Sangjin Lee >Assignee: Junping Du > Attachments: YARN-3046-no-test.patch > > > Per design in YARN-2928, select a handful of MR metrics (e.g. HDFS bytes > written) and have the MR AM write the framework-specific metrics to ATS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3046) [Event producers] Implement MapReduce AM writing some MR metrics to ATS
[ https://issues.apache.org/jira/browse/YARN-3046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated YARN-3046: -- Summary: [Event producers] Implement MapReduce AM writing some MR metrics to ATS (was: implement MapReduce AM writing some MR metrics to ATS) > [Event producers] Implement MapReduce AM writing some MR metrics to ATS > --- > > Key: YARN-3046 > URL: https://issues.apache.org/jira/browse/YARN-3046 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Sangjin Lee >Assignee: Robert Kanter > > Per design in YARN-2928, select a handful of MR metrics (e.g. HDFS bytes > written) and have the MR AM write the framework-specific metrics to ATS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
