[jira] [Updated] (MAPREDUCE-3483) CapacityScheduler reserves container on same node as AM but can't ever use due to never enough avail memory
[ https://issues.apache.org/jira/browse/MAPREDUCE-3483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated MAPREDUCE-3483: Target Version/s: 2.6.0 (was: 3.0.0, 2.5.0) > CapacityScheduler reserves container on same node as AM but can't ever use > due to never enough avail memory > --- > > Key: MAPREDUCE-3483 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3483 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 2.3.0 >Reporter: Thomas Graves >Assignee: Arun C Murthy > > Saw a case where a job was stuck trying to get reducers. The issue is the > capacity scheduler reserved a container on the same node as the application > master but there wasn't ever enough memory to run the reducer on that node. > Node total memory was 8G, Reducer needed 8G, AM was using 2G. This > particular job had 10 reducers and it was stuck waiting on the one because > the AM + reserved reducer memory was already over the queue limit. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (MAPREDUCE-3483) CapacityScheduler reserves container on same node as AM but can't ever use due to never enough avail memory
[ https://issues.apache.org/jira/browse/MAPREDUCE-3483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated MAPREDUCE-3483: - Target Version/s: 3.0.0, 2.5.0 (was: 0.23.0) > CapacityScheduler reserves container on same node as AM but can't ever use > due to never enough avail memory > --- > > Key: MAPREDUCE-3483 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3483 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 2.3.0 >Reporter: Thomas Graves >Assignee: Arun C Murthy > > Saw a case where a job was stuck trying to get reducers. The issue is the > capacity scheduler reserved a container on the same node as the application > master but there wasn't ever enough memory to run the reducer on that node. > Node total memory was 8G, Reducer needed 8G, AM was using 2G. This > particular job had 10 reducers and it was stuck waiting on the one because > the AM + reserved reducer memory was already over the queue limit. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (MAPREDUCE-3483) CapacityScheduler reserves container on same node as AM but can't ever use due to never enough avail memory
[ https://issues.apache.org/jira/browse/MAPREDUCE-3483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated MAPREDUCE-3483: - Affects Version/s: (was: 0.23.0) 2.3.0 > CapacityScheduler reserves container on same node as AM but can't ever use > due to never enough avail memory > --- > > Key: MAPREDUCE-3483 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3483 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 2.3.0 >Reporter: Thomas Graves >Assignee: Arun C Murthy > > Saw a case where a job was stuck trying to get reducers. The issue is the > capacity scheduler reserved a container on the same node as the application > master but there wasn't ever enough memory to run the reducer on that node. > Node total memory was 8G, Reducer needed 8G, AM was using 2G. This > particular job had 10 reducers and it was stuck waiting on the one because > the AM + reserved reducer memory was already over the queue limit. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (MAPREDUCE-3483) CapacityScheduler reserves container on same node as AM but can't ever use due to never enough avail memory
[ https://issues.apache.org/jira/browse/MAPREDUCE-3483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3483: - Priority: Major (was: Blocker) Thanks Thomas. The way the CS is setup, this is an extremely rare corner case which is hard to fix. The essential problem here is that in your setup the queue has so little capacity that it can't get more than one reduce slot... For now, I'll downgrade as this is a corner case as I think about ways to fix this. > CapacityScheduler reserves container on same node as AM but can't ever use > due to never enough avail memory > --- > > Key: MAPREDUCE-3483 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3483 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Arun C Murthy > > Saw a case where a job was stuck trying to get reducers. The issue is the > capacity scheduler reserved a container on the same node as the application > master but there wasn't ever enough memory to run the reducer on that node. > Node total memory was 8G, Reducer needed 8G, AM was using 2G. This > particular job had 10 reducers and it was stuck waiting on the one because > the AM + reserved reducer memory was already over the queue limit. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira