[
https://issues.apache.org/jira/browse/MAPREDUCE-4366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660470#comment-13660470
]
Arun C Murthy commented on MAPREDUCE-4366:
------------------------------------------
bq. Waiting maps and reduces are updated in the job tracker metrics every time
that a task is launched is fails/completes, so this would not work unless I am
missing something.
We are both right and wrong, simultaneously.
The problem is that I was following call-heirarchy for JT metrics
(JIP.jobtracker.getInstrumentation), as opposed to queue metrics
(JIP.queueMetrics).
JIP.jobtracker metrics are borked - a bigger bug, sigh. These aren't updated
ala JIP.queueMetrics for task launch & fail/completion.
Another thing to fix?
----
{quote}
My job has two maps. Attempts are run for both of them. One map gets a
speculative attempt because it's running slow. The other map's attempt fails.
The speculative one completes. initialMaps=2 + speculativeMaps=0 -
runningMaps=1 - finishedMaps=1 - failedMaps=0. So pendingMaps is now 0 ...
{quote}
No, speculativeMaps == 1, by which we get pendingMaps as 1 since initialMaps=2
+ speculativeMaps=1 - runningMaps=1 - finishedMaps=1 - failedMaps=0. Correct?
Am I missing something else? I may well be...
> mapred metrics shows negative count of waiting maps and reduces
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-4366
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4366
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 1.0.2
> Reporter: Thomas Graves
> Assignee: Sandy Ryza
> Attachments: MAPREDUCE-4366-branch-1-1.patch,
> MAPREDUCE-4366-branch-1.patch
>
>
> Negative waiting_maps and waiting_reduces count is observed in the mapred
> metrics. MAPREDUCE-1238 partially fixed this but it appears there is still
> issues as we are seeing it, but not as bad.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
