[
https://issues.apache.org/jira/browse/HADOOP-1048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12476375
]
Doug Cutting commented on HADOOP-1048:
--------------------------------------
Another possible optimization in JobInProgress#updateTaskStatus might be to
keep, for each task, its previous counter values, then compute the difference
between those and the new values, and update global counters incrementally
rather than recomputing them from scratch each time. This might not actually
reduce the amount of computation much, but it might greatly decrease the amount
of allocated objects, and object allocations are considerably more expensive
than arithmetic.
> Sort500 failing since counters patch went in
> --------------------------------------------
>
> Key: HADOOP-1048
> URL: https://issues.apache.org/jira/browse/HADOOP-1048
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Reporter: David Bowen
>
> Nigel wrote:
> We have a problem
> Looks like counters patch is "breaking" things
> Sort500 has been failing since the patch went in
> Looking at the JT logs, there are 0 "Call queue overflow" message before
> the patch went in and 15000+ after the patch went in
> Looks like the counters stuff is overwhelming the JT
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
