[
https://issues.apache.org/jira/browse/MAPREDUCE-3859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660501#comment-13660501
]
Arun C Murthy commented on MAPREDUCE-3859:
------------------------------------------
[~sergeant] Sorry! I lost track of this again. Sincere apologies.
The same bug exists in CS in YARN:
{noformat}
Resource currentCapacity =
Resources.lessThan(resourceCalculator, clusterResource,
usedResources, queueCapacity) ?
queueCapacity : Resources.add(usedResources, required);
{noformat}
Do you mind providing a patch for that too? Something like:
{noformat}
index 64f7114..7d8800a 100644
---
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
+++
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
@@ -997,7 +997,7 @@ private Resource computeUserLimit(FiCaSchedulerApp
application,
Resource currentCapacity =
Resources.lessThan(resourceCalculator, clusterResource,
- usedResources, queueCapacity) ?
+ Resources.add(usedResources, required), queueCapacity) ?
queueCapacity : Resources.add(usedResources, required);
// Never allow a single user to take more than the
{noformat}
Guilty as charged on mocks for testing in YARN... *smile*
However, it does result in our tests running in ~20mins rather than 5-6hrs it
takes in hadoop-1!
> CapacityScheduler incorrectly utilizes extra-resources of queue for
> high-memory jobs
> ------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3859
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3859
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: capacity-sched
> Affects Versions: 1.0.0
> Environment: CDH3u1
> Reporter: Sergey Tryuber
> Assignee: Sergey Tryuber
> Attachments: test-to-fail.patch.txt
>
>
> Imagine, we have a queue A with capacity 10 slots and 20 as extra-capacity,
> jobs which use 3 map slots will never consume more than 9 slots, regardless
> how many free slots on a cluster.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
