[
https://issues.apache.org/jira/browse/PIG-1829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12988320#action_12988320
]
Santhosh Srinivasan commented on PIG-1829:
------------------------------------------
bq. bq. What are the bounds on the reasonable amount of time?
bq. The JT has a configured limit on #jobs in memory and disk. So, one can
customize it per-installation.
Any range in practical usage, 1 minute, 5 minutes, 1 hour, etc? It will help to
set expectation with the user.
> "0" value seen in PigStat's map/reduce runtime, even when the job is
> successful
> -------------------------------------------------------------------------------
>
> Key: PIG-1829
> URL: https://issues.apache.org/jira/browse/PIG-1829
> Project: Pig
> Issue Type: Improvement
> Affects Versions: 0.8.0
> Reporter: Thejas M Nair
> Fix For: 0.9.0
>
>
> Pig runtime calls JobClient.getMapTaskReports(jobId) and
> JobClient.getReduceTaskReports(jobId) to get statistics about numbers of
> maps/reducers, as well as max/min/avg time of these tasks. But from time to
> time, these calls return empty lists. When that happens pig is reports 0
> values for the stats.
> The jobtracker keeps the stats information only for a limited duration based
> on the configuration parameters mapred.jobtracker.completeuserjobs.maximum
> and mapred.job.tracker.retiredjobs.cache.size. Since pig collects the stats
> after jobs have finished running, it is possible that the stats for the
> initial jobs are no longer available. To have better chances of getting the
> stats, it should be collected as soon as the job is over.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
