[
https://issues.apache.org/jira/browse/OOZIE-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14574235#comment-14574235
]
Laxman commented on OOZIE-2195:
-------------------------------
+1 (non-binding) for the idea.
[~jaydeepvishwakarma], how about having some consolidated counters as well.
For ex: total number of map tasks, total number of reduce tasks, total records
written, total bytes written, bags spilled, records spilled and anything that
makes sense of pig script performance.
Following are some counters emitted in log.
{noformat}
Counters:
Total records written : 1258632411
Total bytes written : 49810853810
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 16
Total records proactively spilled: 447456
{noformat}
> Expose Pig read/write bytes details as counters in Launcher
> -----------------------------------------------------------
>
> Key: OOZIE-2195
> URL: https://issues.apache.org/jira/browse/OOZIE-2195
> Project: Oozie
> Issue Type: New Feature
> Reporter: Jaydeep Vishwakarma
> Assignee: Narayan Periwal
>
> Pig action spawns multiple mr jobs and all the aggregated read/write bytes
> metrics are emitted in logs only. It would be really helpful if we expose
> those information in Launcher as counters. This should not be the default
> behaviour. User can enable this by job level or global
> configuration.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
