[
https://issues.apache.org/jira/browse/MAPREDUCE-1463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831894#action_12831894
]
Amar Kamat commented on MAPREDUCE-1463:
---------------------------------------
What should be the behavior where total number of maps and reducers are less
(i.e a small job for now) but takes huge amount of time to finish. For example
the map takes a day to run while the reduces are also compute intensive. In
such a case would we still consider the job as small job? I think what we want
to capture is the job behavior (fast *finishing* job versus others). Using task
counts might not be sufficient.
Scott, wouldn't this problem be solved if you set
'mapreduce.job.reduce.slowstart.completedmaps' to a default value of 0 (instead
of 0.5) for all your users?
> Reducer should start faster for smaller jobs
> --------------------------------------------
>
> Key: MAPREDUCE-1463
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1463
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: contrib/fair-share
> Reporter: Scott Chen
> Assignee: Scott Chen
> Attachments: MAPREDUCE-1463-v1.patch, MAPREDUCE-1463-v2.patch,
> MAPREDUCE-1463-v3.patch
>
>
> Our users often complain about the slowness of smaller ad-hoc jobs.
> The overhead to wait for the reducers to start in this case is significant.
> It will be good if we can start the reducer sooner in this case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
