[
https://issues.apache.org/jira/browse/HADOOP-5160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12671241#action_12671241
]
Nathan Marz commented on HADOOP-5160:
-------------------------------------
Reducing the number of reduce slots to 1 per box is impractical. This job that
requires the 1 reducer per machine configuration runs as part of a much, much
larger workflow, where every other job benefits from having multiple reducers
on each machine.
> Hadoop reduce scheduler sometimes leaves machines idle
> ------------------------------------------------------
>
> Key: HADOOP-5160
> URL: https://issues.apache.org/jira/browse/HADOOP-5160
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Nathan Marz
>
> I have a MapReduce application with number of reducers equal to the number of
> machines in the cluster (and with speculative execution turned off). However,
> Hadoop schedules multiple reduces to run on single machines and leaves other
> machines idle. This causes contention and seriously slows down the job.
> Hadoop should employ the simple heuristic of utilizing as many machines as
> possible when scheduling reduces.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
