[ http://issues.apache.org/jira/browse/HADOOP-492?page=comments#action_12431736 ] Milind Bhandarkar commented on HADOOP-492: ------------------------------------------
We need separate global counters because there is no way to programmatically add new metrics to the TaskMetrics class, which will (after I resubmit my TaskMetrics patch) be accumulated in JobTracker via heartbeats. OTOH, TaskMetrics could be implemented in terms of global counters. > Global counters > --------------- > > Key: HADOOP-492 > URL: http://issues.apache.org/jira/browse/HADOOP-492 > Project: Hadoop > Issue Type: New Feature > Components: mapred > Reporter: arkady borkovsky > > It would be nice to have map / reduce job keep aggregated counts for > arbitrary events occuring in its tasks -- the numer of records processed, the > numer of exceptions of a specific type, the number of sentences in passive > voice, whatever the jobs finds useful. > This can be implemented by tasks periodically sending <name, value> pairs to > the jobtracker (in some implementations such messages are piggy-backed on the > heartbeats), so that the job tracker stores all the latests values from each > task and aggregates them on a request. It should also make the aggregated > values available at the job end. The value for a task would be flushed when > the task fails. > #491 and #490 may be related to this one. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
