[
https://issues.apache.org/jira/browse/MAPREDUCE-3859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13618429#comment-13618429
]
Arun C Murthy commented on MAPREDUCE-3859:
------------------------------------------
[~sergeant] Sorry, I lost track of this. I think you are right...
A bit of history: The CS, initially, did not support multiple-slots per task
i.e. HighRAM jobs. In that case, the check 'if (queueSlotsOccupied <
queueCapacity)' was correct, since in that case queueCapacity would at least be
(queueSlotsOccupied + 1), there-by satisfying the current ask. Obviously, that
breaks with multiple slots per job... and hence the bug.
The bug exists in YARN (i.e. MR2) CapacityScheduler too - could you please
provide a fix for that too?
I'll go ahead and commit them both... thanks!
Apologies again for losing track of this! My bad.
> CapacityScheduler incorrectly utilizes extra-resources of queue for
> high-memory jobs
> ------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3859
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3859
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: capacity-sched
> Affects Versions: 1.0.0
> Environment: CDH3u1
> Reporter: Sergey Tryuber
> Attachments: test-to-fail.patch.txt
>
>
> Imagine, we have a queue A with capacity 10 slots and 20 as extra-capacity,
> jobs which use 3 map slots will never consume more than 9 slots, regardless
> how many free slots on a cluster.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
