[jira] [Commented] (MAPREDUCE-6018) Create a framework specific config to enable timeline server
[ https://issues.apache.org/jira/browse/MAPREDUCE-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14186728#comment-14186728 ] Hudson commented on MAPREDUCE-6018: --- SUCCESS: Integrated in Hadoop-Yarn-trunk #726 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/726/]) MAPREDUCE-6018. Added an MR specific config to enable emitting job history data to the timeline server. Contributed by Robert Kanter. (zjshen: rev 971e91c8c03a23e4613ed3f071b4f982ee5a1b63) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMRTimelineEventHandling.java * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml Create a framework specific config to enable timeline server Key: MAPREDUCE-6018 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6018 Project: Hadoop Map/Reduce Issue Type: Sub-task Reporter: Jonathan Eagles Assignee: Robert Kanter Fix For: 2.6.0 Attachments: MAPREDUCE-6018.patch, MAPREDUCE-6018.patch, MAPREDUCE-6018.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6018) Create a framework specific config to enable timeline server
[ https://issues.apache.org/jira/browse/MAPREDUCE-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14186827#comment-14186827 ] Hudson commented on MAPREDUCE-6018: --- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1940 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1940/]) MAPREDUCE-6018. Added an MR specific config to enable emitting job history data to the timeline server. Contributed by Robert Kanter. (zjshen: rev 971e91c8c03a23e4613ed3f071b4f982ee5a1b63) * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMRTimelineEventHandling.java Create a framework specific config to enable timeline server Key: MAPREDUCE-6018 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6018 Project: Hadoop Map/Reduce Issue Type: Sub-task Reporter: Jonathan Eagles Assignee: Robert Kanter Fix For: 2.6.0 Attachments: MAPREDUCE-6018.patch, MAPREDUCE-6018.patch, MAPREDUCE-6018.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6018) Create a framework specific config to enable timeline server
[ https://issues.apache.org/jira/browse/MAPREDUCE-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14186889#comment-14186889 ] Hudson commented on MAPREDUCE-6018: --- SUCCESS: Integrated in Hadoop-Hdfs-trunk #1915 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1915/]) MAPREDUCE-6018. Added an MR specific config to enable emitting job history data to the timeline server. Contributed by Robert Kanter. (zjshen: rev 971e91c8c03a23e4613ed3f071b4f982ee5a1b63) * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMRTimelineEventHandling.java Create a framework specific config to enable timeline server Key: MAPREDUCE-6018 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6018 Project: Hadoop Map/Reduce Issue Type: Sub-task Reporter: Jonathan Eagles Assignee: Robert Kanter Fix For: 2.6.0 Attachments: MAPREDUCE-6018.patch, MAPREDUCE-6018.patch, MAPREDUCE-6018.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6018) Create a framework specific config to enable timeline server
[ https://issues.apache.org/jira/browse/MAPREDUCE-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14186317#comment-14186317 ] Zhijie Shen commented on MAPREDUCE-6018: Again, Jenkins is temporally broken. The last patch is minor change upon the prior one. Will go ahead to commit it directly. Create a framework specific config to enable timeline server Key: MAPREDUCE-6018 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6018 Project: Hadoop Map/Reduce Issue Type: Sub-task Reporter: Jonathan Eagles Assignee: Robert Kanter Attachments: MAPREDUCE-6018.patch, MAPREDUCE-6018.patch, MAPREDUCE-6018.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6018) Create a framework specific config to enable timeline server
[ https://issues.apache.org/jira/browse/MAPREDUCE-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14186335#comment-14186335 ] Hudson commented on MAPREDUCE-6018: --- SUCCESS: Integrated in Hadoop-trunk-Commit #6363 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/6363/]) MAPREDUCE-6018. Added an MR specific config to enable emitting job history data to the timeline server. Contributed by Robert Kanter. (zjshen: rev 971e91c8c03a23e4613ed3f071b4f982ee5a1b63) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMRTimelineEventHandling.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml Create a framework specific config to enable timeline server Key: MAPREDUCE-6018 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6018 Project: Hadoop Map/Reduce Issue Type: Sub-task Reporter: Jonathan Eagles Assignee: Robert Kanter Fix For: 2.6.0 Attachments: MAPREDUCE-6018.patch, MAPREDUCE-6018.patch, MAPREDUCE-6018.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6018) Create a framework specific config to enable timeline server
[
https://issues.apache.org/jira/browse/MAPREDUCE-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14184611#comment-14184611
]
Vinod Kumar Vavilapalli commented on MAPREDUCE-6018:
mapreduce.job.emit-timeline-data is already a boolean, no need for enabled.
Otherwise we can follow RM's config and say
{{mapreduce.job.timeline-data-publisher.enabled}}?
Create a framework specific config to enable timeline server
Key: MAPREDUCE-6018
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6018
Project: Hadoop Map/Reduce
Issue Type: Sub-task
Reporter: Jonathan Eagles
Assignee: Robert Kanter
Attachments: MAPREDUCE-6018.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6018) Create a framework specific config to enable timeline server
[ https://issues.apache.org/jira/browse/MAPREDUCE-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14184645#comment-14184645 ] Zhijie Shen commented on MAPREDUCE-6018: bq. mapreduce.job.emit-timeline-data Thanks for clarification. I used to think emit-timeline-data means a sub module of job. In MAPREDUCE-5933, we actually didn't introduce an obvious submodule, so let's use mapreduce.job.emit-timeline-data = false. Create a framework specific config to enable timeline server Key: MAPREDUCE-6018 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6018 Project: Hadoop Map/Reduce Issue Type: Sub-task Reporter: Jonathan Eagles Assignee: Robert Kanter Attachments: MAPREDUCE-6018.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6018) Create a framework specific config to enable timeline server
[
https://issues.apache.org/jira/browse/MAPREDUCE-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14184365#comment-14184365
]
Vinod Kumar Vavilapalli commented on MAPREDUCE-6018:
Haven't looked at all of the patch, but one comment: How about
{{mapreduce.job.emit-timeline-data}} (or something similar) instead of
{{mapreduce.job.timeline-service.enabled}}? If you agree, similarly rename the
constants.
Create a framework specific config to enable timeline server
Key: MAPREDUCE-6018
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6018
Project: Hadoop Map/Reduce
Issue Type: Sub-task
Reporter: Jonathan Eagles
Assignee: Robert Kanter
Attachments: MAPREDUCE-6018.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6018) Create a framework specific config to enable timeline server
[ https://issues.apache.org/jira/browse/MAPREDUCE-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14184372#comment-14184372 ] Zhijie Shen commented on MAPREDUCE-6018: Let's say mapreduce.job.emit-timeline-data.enabled? Create a framework specific config to enable timeline server Key: MAPREDUCE-6018 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6018 Project: Hadoop Map/Reduce Issue Type: Sub-task Reporter: Jonathan Eagles Assignee: Robert Kanter Attachments: MAPREDUCE-6018.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6018) Create a framework specific config to enable timeline server
[
https://issues.apache.org/jira/browse/MAPREDUCE-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14183737#comment-14183737
]
Zhijie Shen commented on MAPREDUCE-6018:
Thanks for the patch, [~rkanter]! Some minor comments about it.
1. It's not necessary var. The client is only constructed when the flag is set.
Then we try to use timeline client, just check null first.
{code}
+ private boolean timelineEnabled;
{code}
2. Can you add the config to mapred-default.xml and add a description?
Create a framework specific config to enable timeline server
Key: MAPREDUCE-6018
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6018
Project: Hadoop Map/Reduce
Issue Type: Sub-task
Reporter: Jonathan Eagles
Assignee: Robert Kanter
Attachments: MAPREDUCE-6018.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6018) Create a framework specific config to enable timeline server
[ https://issues.apache.org/jira/browse/MAPREDUCE-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14183744#comment-14183744 ] Zhijie Shen commented on MAPREDUCE-6018: I forgot mentioning how about using *false* as the default value? MR is a long-live and widely used framework, let's be more conservative when introducing a new feature it. Create a framework specific config to enable timeline server Key: MAPREDUCE-6018 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6018 Project: Hadoop Map/Reduce Issue Type: Sub-task Reporter: Jonathan Eagles Assignee: Robert Kanter Attachments: MAPREDUCE-6018.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6018) Create a framework specific config to enable timeline server
[ https://issues.apache.org/jira/browse/MAPREDUCE-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14181868#comment-14181868 ] Zhijie Shen commented on MAPREDUCE-6018: [~rkanter], thanks for taking care of this issue. We could have a MR framework-level config, such as mapreduce.jobhistory.timeline-service.enabled, and it can also be overwritten by each individual MR job by specifying it in CLI args. Only when this flag is set, the logic that history events are posted to the timeline server will be executed. Create a framework specific config to enable timeline server Key: MAPREDUCE-6018 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6018 Project: Hadoop Map/Reduce Issue Type: Sub-task Reporter: Jonathan Eagles Assignee: Robert Kanter -- This message was sent by Atlassian JIRA (v6.3.4#6332)
