[
https://issues.apache.org/jira/browse/MAPREDUCE-3483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169438#comment-13169438
]
Thomas Graves commented on MAPREDUCE-3483:
------------------------------------------
I believe the queue capacity was actually 4.8G - 12% of 40G cluster. The UI
displayed capacity at 208% usage when this occurred.
2011-11-29 20:51:53,948 DEBUG
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue:
User limit
computation for userfoo in queue default userLimit=100 userLimitFactor=1.0
required: memory: 8192 consumed: memory:
10240 limit: 8192 queueCapacity: 8192 qconsumed: 10240 currentCapacity: 18432
activeUsers: 1 clusterCapacity: 40960
2011-11-29 20:51:53,948 DEBUG
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue:
User userfoo
in queue default will exceed limit - consumed: memory: 10240 limit: memory:
8192
> CapacityScheduler reserves container on same node as AM but can't ever use
> due to never enough avail memory
> -----------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3483
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3483
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Arun C Murthy
> Priority: Blocker
>
> Saw a case where a job was stuck trying to get reducers. The issue is the
> capacity scheduler reserved a container on the same node as the application
> master but there wasn't ever enough memory to run the reducer on that node.
> Node total memory was 8G, Reducer needed 8G, AM was using 2G. This
> particular job had 10 reducers and it was stuck waiting on the one because
> the AM + reserved reducer memory was already over the queue limit.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
