[
https://issues.apache.org/jira/browse/PIG-1829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12988203#action_12988203
]
Olga Natkovich commented on PIG-1829:
-------------------------------------
I agree that having standard API is useful. I don't think I like the idea of
reading current structure because this will make it difficult to run against
concurrent versions of Hadoop.
I think we need to do further investigation and find other alternatives or just
say that this will resolve once we have a reasonable support form Hadoop
> "0" value seen in PigStat's map/reduce runtime, even when the job is
> successful
> -------------------------------------------------------------------------------
>
> Key: PIG-1829
> URL: https://issues.apache.org/jira/browse/PIG-1829
> Project: Pig
> Issue Type: Improvement
> Affects Versions: 0.8.0
> Reporter: Thejas M Nair
> Fix For: 0.9.0
>
>
> Pig runtime calls JobClient.getMapTaskReports(jobId) and
> JobClient.getReduceTaskReports(jobId) to get statistics about numbers of
> maps/reducers, as well as max/min/avg time of these tasks. But from time to
> time, these calls return empty lists. When that happens pig is reports 0
> values for the stats.
> The jobtracker keeps the stats information only for a limited duration based
> on the configuration parameters mapred.jobtracker.completeuserjobs.maximum
> and mapred.job.tracker.retiredjobs.cache.size. Since pig collects the stats
> after jobs have finished running, it is possible that the stats for the
> initial jobs are no longer available. To have better chances of getting the
> stats, it should be collected as soon as the job is over.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
