[
https://issues.apache.org/jira/browse/MAPREDUCE-1379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12800605#action_12800605
]
Matei Zaharia commented on MAPREDUCE-1379:
------------------------------------------
Is the goal to be able to set "total.tasks.maximum" to less than
map.tasks.maximum + reduce.tasks.maximum? If so, you should watch out because
you might possibly starve tasks of one type. For example, if you set max map
slots to 5, max reduce slots to 5, and max total slots to 5, most of the
current job schedulers will probably cause reduce tasks to be starved, because
they always look for a map to launch before looking for a reduce. In fact,
given that schedulers ultimately decide which tasks to launch anyway, I
wouldn't add a "max total slots" concept in the TaskTracker to begin with; I
would add this functionality to one of the schedulers.
> Limit both numMapTasks and numReduceTasks
> -----------------------------------------
>
> Key: MAPREDUCE-1379
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1379
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: tasktracker
> Affects Versions: 0.22.0
> Reporter: Bochun Bai
> Priority: Trivial
> Attachments: limit-both-numMapTasks-and-numReduceTasks.patch
>
>
> In some environment, the number of concurrent running process is very
> sensitive.
> mapreduce.tasktracker.map.tasks.maximum
> and
> mapreduce.tasktracker.reduce.tasks.maximum
> limit tasks running on each tasktracker separately.
> This patch limits them together, using
> mapreduce.tasktracker.total.tasks.maximum
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
