[
https://issues.apache.org/jira/browse/MAPREDUCE-2124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12927739#action_12927739
]
Amar Kamat commented on MAPREDUCE-2124:
---------------------------------------
Scott,
If I understand correctly, you intend to display/store the amount of time spent
by the reducer during shuffle, sort and reduce phase in the task/attempt
counter for completed tasks. Currently, once the task is done, shuffle and sort
times are stored in the job history (via {{TaskStatus}}). Why can't you simply
use this, instead of counters, to display shuffle and sort times for reducers
on the webui (and store it in {{TaskReport}})?
> Add job counters for measuring time spent in three different phases in
> reducers
> -------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2124
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2124
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.22.0
> Reporter: Scott Chen
> Assignee: Scott Chen
> Priority: Minor
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-2124-v2.txt, MAPREDUCE-2124.txt,
> test-result.txt
>
>
> We currently have SLOTS_MILLIS_REDUCES which measures the total slot time of
> reducer.
> It will be useful if we have
> {code}
> SLOTS_MILLIS_REDUCES_COPY
> SLOTS_MILLIS_REDUCES_SORT
> SLOTS_MILLIS_REDUCES_REDUCE
> {code}
> which measures three different phases of a reducer.
> This will help us identify the bottleneck of the reducers.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
