[jira] [Commented] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265838#comment-15265838 ] Varun Saxena commented on MAPREDUCE-6688: - 3 of the checkstyle issues can be fixed. Will take care of them in next patch after feedback from next round of review. > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[
https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265827#comment-15265827
]
Hadoop QA commented on MAPREDUCE-6688:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 16s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 36s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
48s {color} | {color:green} YARN-2928 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 25s
{color} | {color:green} YARN-2928 passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 42s
{color} | {color:green} YARN-2928 passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s {color} | {color:green} YARN-2928 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 28s
{color} | {color:green} YARN-2928 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
46s {color} | {color:green} YARN-2928 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 10s
{color} | {color:red}
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
in YARN-2928 has 2 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s
{color} | {color:green} YARN-2928 passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s
{color} | {color:green} YARN-2928 passed with JDK v1.7.0_95 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 28s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 28s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 41s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 41s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 30s
{color} | {color:red} hadoop-mapreduce-project/hadoop-mapreduce-client: patch
generated 34 new + 711 unchanged - 32 fixed = 745 total (was 743) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 22s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
38s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
51s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 43s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 52s
{color} | {color:green} hadoop-mapreduce-client-core in the patch passed with
JDK v1.8.0_91. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m 8s
{color} | {color:green} hadoop-mapreduce-client-app in the patch passed with
JDK v1.8.0_91. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 104m 54s
{color} | {color:red} hadoop-mapreduce-client-jobclient in the patch failed
with JDK v1.8.0_91. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 16s
{color} | {color:green} hadoop-mapreduce-client-core in the patch passed w
[jira] [Commented] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265759#comment-15265759 ] Varun Saxena commented on MAPREDUCE-6688: - Well I was of the opinion that right now, we are not quite sure of the approach we might take in YARN-3401 and the solution would emerge more from the security model we decide upon. Which we haven't given a great deal of thought on. It is hence, possible we later decide that we will not push events from JobHistoryEventHandler from RM, that is, if we even decide to route it via RM. And yes, that may involve further refactoring. So I was of the opinion let us first go with what is best for current code and then refactor later, if required, based on decision taken. Thoughts ? For me, there will be certain events which would require to be pushed immediately and for that we will have a sync call(for instance, JOB_FINISHED) and others will be async calls. So that should be clean enough. Correct ? Or you are thinking of some other parameters on which we will make this decision ? > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[
https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265752#comment-15265752
]
Naganarasimha G R commented on MAPREDUCE-6688:
--
Hi [~varun_saxena],
Earlier when we had discussed the solution offline i did not get the complete
picture but after putting more thought on it and seeing your patch, had my
reservations on the approach which i mentioned
{{HistoryEvent.getTimelineEntitiesToPublish}}, In my view overall design across
the history events seems to have specific modifications which are required for
JobFinished and JobSubmitted Event. Also if we plan to push later through RM
then i am not sure we will try to push events from EventHandler to RM. And also
configuration would be difficult to understand to the user as it considers the
implementation specific logic.
Even if we take [~gtCarrera9]'s approach {{HistoryEvent.addDataToEntity}} will
also have its own set of issues.
So, IMO if its important to capture configuration as part of System Entities
and if its important split the configs then i am fine with the current approach
provided if others are ok to refactor it again based on the approach taken in
YARN-3401.
> Store job configurations in Timeline Service v2
> ---
>
> Key: MAPREDUCE-6688
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster
>Affects Versions: YARN-2928
>Reporter: Junping Du
>Assignee: Varun Saxena
> Labels: yarn-2928-1st-milestone
> Attachments: MAPREDUCE-6688-YARN-2928.01.patch,
> YARN-3959-YARN-2928.01.patch
>
>
> We already have configuration field in HBase schema for application entity.
> We need to make sure AM write it out when it get launched.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265751#comment-15265751 ] Varun Saxena commented on MAPREDUCE-6688: - Replaced the previous patch as this one is equivalent to the previous patch except that there were some conflicts due to MAPREDUCE-6424 going in. > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated MAPREDUCE-6688: Affects Version/s: YARN-2928 Status: Patch Available (was: In Progress) > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated MAPREDUCE-6688: Attachment: (was: MAPREDUCE-6688-YARN-2928.01.patch) > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Affects Versions: YARN-2928 >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated MAPREDUCE-6688: Attachment: MAPREDUCE-6688-YARN-2928.01.patch > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > MAPREDUCE-6688-YARN-2928.01.patch, YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6424) Store MR counters as timeline metrics instead of event
[ https://issues.apache.org/jira/browse/MAPREDUCE-6424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265744#comment-15265744 ] Naganarasimha G R commented on MAPREDUCE-6424: -- Thanks [~varun_saxena] for reviewing and committing the patch and [~sjlee0] & [~gtCarrera9] for the review . > Store MR counters as timeline metrics instead of event > -- > > Key: MAPREDUCE-6424 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6424 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Junping Du >Assignee: Naganarasimha G R > Labels: yarn-2928-1st-milestone > Fix For: YARN-2928 > > Attachments: MAPREDUCE-6424-YARN-2928.v1.001.patch, > MAPREDUCE-6424-YARN-2928.v1.002.patch, MAPREDUCE-6424-YARN-2928.v1.003.patch, > MAPREDUCE-6424-YARN-2928.v1.004.patch, MAPREDUCE-6424-YARN-2928.v1.005.patch > > > In MAPREDUCE-6327, we make map/reduce counters get encoded from > JobFinishedEvent as timeline events with counters details in JSON format. > We need to store framework specific counters as metrics in timeline service > to support query, aggregation, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6424) Store MR counters as timeline metrics instead of event
[ https://issues.apache.org/jira/browse/MAPREDUCE-6424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265741#comment-15265741 ] Varun Saxena commented on MAPREDUCE-6424: - Committed this to YARN-2928 branch. Thanks [~Naganarasimha] for your contribution. Thanks [~sjlee0] and [~gtCarrera9] for your reviews. And thanks [~djp] for filing this JIRA. > Store MR counters as timeline metrics instead of event > -- > > Key: MAPREDUCE-6424 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6424 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Junping Du >Assignee: Naganarasimha G R > Labels: yarn-2928-1st-milestone > Fix For: YARN-2928 > > Attachments: MAPREDUCE-6424-YARN-2928.v1.001.patch, > MAPREDUCE-6424-YARN-2928.v1.002.patch, MAPREDUCE-6424-YARN-2928.v1.003.patch, > MAPREDUCE-6424-YARN-2928.v1.004.patch, MAPREDUCE-6424-YARN-2928.v1.005.patch > > > In MAPREDUCE-6327, we make map/reduce counters get encoded from > JobFinishedEvent as timeline events with counters details in JSON format. > We need to store framework specific counters as metrics in timeline service > to support query, aggregation, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6424) Store MR counters as timeline metrics instead of event
[ https://issues.apache.org/jira/browse/MAPREDUCE-6424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated MAPREDUCE-6424: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: YARN-2928 Status: Resolved (was: Patch Available) > Store MR counters as timeline metrics instead of event > -- > > Key: MAPREDUCE-6424 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6424 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Junping Du >Assignee: Naganarasimha G R > Labels: yarn-2928-1st-milestone > Fix For: YARN-2928 > > Attachments: MAPREDUCE-6424-YARN-2928.v1.001.patch, > MAPREDUCE-6424-YARN-2928.v1.002.patch, MAPREDUCE-6424-YARN-2928.v1.003.patch, > MAPREDUCE-6424-YARN-2928.v1.004.patch, MAPREDUCE-6424-YARN-2928.v1.005.patch > > > In MAPREDUCE-6327, we make map/reduce counters get encoded from > JobFinishedEvent as timeline events with counters details in JSON format. > We need to store framework specific counters as metrics in timeline service > to support query, aggregation, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6424) Store MR counters as timeline metrics instead of event
[ https://issues.apache.org/jira/browse/MAPREDUCE-6424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265734#comment-15265734 ] Varun Saxena commented on MAPREDUCE-6424: - bq. Could you guys review it, and commit it if you are +1 too? [~sjlee0], you meant go ahead and commit it as well ? Will do so. That will help me invoke build for MAPREDUCE-6688 and have a QA report by the time its morning there. > Store MR counters as timeline metrics instead of event > -- > > Key: MAPREDUCE-6424 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6424 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Junping Du >Assignee: Naganarasimha G R > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6424-YARN-2928.v1.001.patch, > MAPREDUCE-6424-YARN-2928.v1.002.patch, MAPREDUCE-6424-YARN-2928.v1.003.patch, > MAPREDUCE-6424-YARN-2928.v1.004.patch, MAPREDUCE-6424-YARN-2928.v1.005.patch > > > In MAPREDUCE-6327, we make map/reduce counters get encoded from > JobFinishedEvent as timeline events with counters details in JSON format. > We need to store framework specific counters as metrics in timeline service > to support query, aggregation, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265654#comment-15265654 ] Varun Saxena commented on MAPREDUCE-6688: - Anyways we need to think of whether we should allow and if not, how not to allow AM publishing YARN entities directly from AM. > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
[ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265651#comment-15265651 ] Varun Saxena commented on MAPREDUCE-6688: - [~sjlee0], I think we can just take a call on what the optimum value size can be(while keeping in mind that 10 async entities go together) as code has already been written for it. Breaking up entities this way should work for 1st milestone atleast. If people have divergent views on the issue, we can raise a new JIRA to address those issue later, say make it configurable or handle it differently. Also I would like [~gtCarrera9] to have a look as well because he suggested refactoring on MAPREDUCE-6424 and I have taken care of that in this JIRA. > Store job configurations in Timeline Service v2 > --- > > Key: MAPREDUCE-6688 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster >Reporter: Junping Du >Assignee: Varun Saxena > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6688-YARN-2928.01.patch, > YARN-3959-YARN-2928.01.patch > > > We already have configuration field in HBase schema for application entity. > We need to make sure AM write it out when it get launched. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6424) Store MR counters as timeline metrics instead of event
[ https://issues.apache.org/jira/browse/MAPREDUCE-6424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265650#comment-15265650 ] Varun Saxena commented on MAPREDUCE-6424: - LGTM too... > Store MR counters as timeline metrics instead of event > -- > > Key: MAPREDUCE-6424 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6424 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Junping Du >Assignee: Naganarasimha G R > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6424-YARN-2928.v1.001.patch, > MAPREDUCE-6424-YARN-2928.v1.002.patch, MAPREDUCE-6424-YARN-2928.v1.003.patch, > MAPREDUCE-6424-YARN-2928.v1.004.patch, MAPREDUCE-6424-YARN-2928.v1.005.patch > > > In MAPREDUCE-6327, we make map/reduce counters get encoded from > JobFinishedEvent as timeline events with counters details in JSON format. > We need to store framework specific counters as metrics in timeline service > to support query, aggregation, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6424) Store MR counters as timeline metrics instead of event
[ https://issues.apache.org/jira/browse/MAPREDUCE-6424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265647#comment-15265647 ] Li Lu commented on MAPREDUCE-6424: -- +1. Latest patch lgtm. > Store MR counters as timeline metrics instead of event > -- > > Key: MAPREDUCE-6424 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6424 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Junping Du >Assignee: Naganarasimha G R > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6424-YARN-2928.v1.001.patch, > MAPREDUCE-6424-YARN-2928.v1.002.patch, MAPREDUCE-6424-YARN-2928.v1.003.patch, > MAPREDUCE-6424-YARN-2928.v1.004.patch, MAPREDUCE-6424-YARN-2928.v1.005.patch > > > In MAPREDUCE-6327, we make map/reduce counters get encoded from > JobFinishedEvent as timeline events with counters details in JSON format. > We need to store framework specific counters as metrics in timeline service > to support query, aggregation, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
