[
https://issues.apache.org/jira/browse/HIVE-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14235308#comment-14235308
]
Lefty Leverenz commented on HIVE-7438:
--------------------------------------
Does the wiki need to be updated? (If so, add a TODOC-SPARK label.)
> Counters, statistics, and metrics [Spark Branch]
> ------------------------------------------------
>
> Key: HIVE-7438
> URL: https://issues.apache.org/jira/browse/HIVE-7438
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
> Reporter: Xuefu Zhang
> Assignee: Chengxiang Li
> Labels: Spark-M3
> Attachments: hive on spark job statistic design.docx
>
>
> Hive makes use of MapReduce counters for statistics and possibly for other
> purposes. For Hive on Spark, we should achieve the same functionality using
> Spark's accumulators.
> Hive also collects metrics from MapReduce jobs traditionally. Spark job very
> likely publishes a different set of metrics, which, if made available, would
> help user to get insights into their spark jobs. Thus, we should obtain the
> metrics and make them available as we do for MapReduce.
> This task therefore includes:
> # identify Hive's existing functionality w.r.t. counters, statistics, and
> metrics;
> # design and implement the same functionality in Spark.
> Please refer to the design document for more information.
> https://cwiki.apache.org/confluence/display/Hive/Hive+on+Spark#HiveonSpark-CountersandMetrics
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
