[
https://issues.apache.org/jira/browse/MAPREDUCE-1998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12896134#action_12896134
]
Allen Wittenauer commented on MAPREDUCE-1998:
---------------------------------------------
So the way I imagine this working is:
admin configures 3 queues:
jobs with > 1000 tasks
jobs with <999 >100 tasks
jobs with < 99 tasks
user submits a job to default, which triggers sized-based queuing.
hadoop determines map tasks: lets say 100, and user requested 20 reduces.
Total task count: 120.
JobTracker places job in the middle tier queue since there are 120 tasks.
> Size-based queuing for capacity scheduler
> -----------------------------------------
>
> Key: MAPREDUCE-1998
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1998
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: contrib/capacity-sched
> Reporter: Allen Wittenauer
>
> On job submission, it would be useful if the capacity scheduler could pick a
> queue based on the # of maps and reduces. This way one could have queues
> based on job-size without users having to pick the queue prior to submission.
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
