[ https://issues.apache.org/jira/browse/MAPREDUCE-6424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15259240#comment-15259240 ]
Sangjin Lee commented on MAPREDUCE-6424: ---------------------------------------- Thanks [~Naganarasimha] for your patch! I think we said it is OK to emit them as is and capture the work of emitting them periodically in a later JIRA. One thing to add, though: I mentioned it elsewhere but we also need to emit the metrics at the *YARN application* level if we want them aggregated to the flows. Therefore, we should add code that emits the metrics for the YARN application entity in addition to the MR job entity. I'd say that's actually even more important than metrics at the job entity. The metrics at the MR-specific entities are there to view, but they will not be aggregated. On the other hand, the YARN application entity is the focal point of real time flow aggregation. > Store MR counters as timeline metrics instead of event > ------------------------------------------------------ > > Key: MAPREDUCE-6424 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6424 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Reporter: Junping Du > Assignee: Naganarasimha G R > Labels: yarn-2928-1st-milestone > Attachments: MAPREDUCE-6424-YARN-2928.v1.001.patch > > > In MAPREDUCE-6327, we make map/reduce counters get encoded from > JobFinishedEvent as timeline events with counters details in JSON format. > We need to store framework specific counters as metrics in timeline service > to support query, aggregation, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)