[jira] [Updated] (MAPREDUCE-3897) capacity scheduler - maxActiveApplicationsPerUser calculation can be wrong
[ https://issues.apache.org/jira/browse/MAPREDUCE-3897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3897: - Resolution: Fixed Fix Version/s: 0.23.2 Status: Resolved (was: Patch Available) I just committed this. Thanks Eric! > capacity scheduler - maxActiveApplicationsPerUser calculation can be wrong > -- > > Key: MAPREDUCE-3897 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3897 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Eric Payne >Priority: Critical > Fix For: 0.23.2 > > Attachments: MAPREDUCE-3897-1.txt, MAPREDUCE-3897-1.txt, > MAPREDUCE-3897-2.txt > > > The capacity scheduler calculates the maxActiveApplications and the > maxActiveApplicationsPerUser based on the config > yarn.scheduler.capacity.maximum-applications or default 1. > MaxActiveApplications = max ( ceil ( clusterMemory/minAllocation * > maxAMResource% * absoluteMaxCapacity), 1) > MaxActiveAppsPerUser = max( ceil (maxActiveApplicationsComputedAbove * > (userLimit%/100) * userLimitFactor), 1) > maxActiveApplications is already multiplied by the queue absolute MAXIMUM > capacity, so if max capacity > capacity and if you have user limit factor 1 > (which is the default) and only 1 user is running, that user will not be > allowed to use over the queue capacity, so having it relative to MAX capacity > doesn't make sense. That user could easily end up in a deadlock and all its > space used by application masters. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3897) capacity scheduler - maxActiveApplicationsPerUser calculation can be wrong
[ https://issues.apache.org/jira/browse/MAPREDUCE-3897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated MAPREDUCE-3897: -- Status: Patch Available (was: Open) > capacity scheduler - maxActiveApplicationsPerUser calculation can be wrong > -- > > Key: MAPREDUCE-3897 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3897 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Eric Payne >Priority: Critical > Attachments: MAPREDUCE-3897-1.txt, MAPREDUCE-3897-1.txt, > MAPREDUCE-3897-2.txt > > > The capacity scheduler calculates the maxActiveApplications and the > maxActiveApplicationsPerUser based on the config > yarn.scheduler.capacity.maximum-applications or default 1. > MaxActiveApplications = max ( ceil ( clusterMemory/minAllocation * > maxAMResource% * absoluteMaxCapacity), 1) > MaxActiveAppsPerUser = max( ceil (maxActiveApplicationsComputedAbove * > (userLimit%/100) * userLimitFactor), 1) > maxActiveApplications is already multiplied by the queue absolute MAXIMUM > capacity, so if max capacity > capacity and if you have user limit factor 1 > (which is the default) and only 1 user is running, that user will not be > allowed to use over the queue capacity, so having it relative to MAX capacity > doesn't make sense. That user could easily end up in a deadlock and all its > space used by application masters. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3897) capacity scheduler - maxActiveApplicationsPerUser calculation can be wrong
[ https://issues.apache.org/jira/browse/MAPREDUCE-3897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated MAPREDUCE-3897: -- Attachment: MAPREDUCE-3897-2.txt Updated patch so that maxActiveApplications remains calculated based on absoluteMaxCapacity but modified so that only maxActiveApplicationsPerUser is calculated based on absoluteCapacity. This will limit the number of active apps for a single user to a percetage corresponding the absolute capacity of the queue unless the user limit factor is increased for that queue. > capacity scheduler - maxActiveApplicationsPerUser calculation can be wrong > -- > > Key: MAPREDUCE-3897 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3897 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Eric Payne >Priority: Critical > Attachments: MAPREDUCE-3897-1.txt, MAPREDUCE-3897-1.txt, > MAPREDUCE-3897-2.txt > > > The capacity scheduler calculates the maxActiveApplications and the > maxActiveApplicationsPerUser based on the config > yarn.scheduler.capacity.maximum-applications or default 1. > MaxActiveApplications = max ( ceil ( clusterMemory/minAllocation * > maxAMResource% * absoluteMaxCapacity), 1) > MaxActiveAppsPerUser = max( ceil (maxActiveApplicationsComputedAbove * > (userLimit%/100) * userLimitFactor), 1) > maxActiveApplications is already multiplied by the queue absolute MAXIMUM > capacity, so if max capacity > capacity and if you have user limit factor 1 > (which is the default) and only 1 user is running, that user will not be > allowed to use over the queue capacity, so having it relative to MAX capacity > doesn't make sense. That user could easily end up in a deadlock and all its > space used by application masters. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3897) capacity scheduler - maxActiveApplicationsPerUser calculation can be wrong
[ https://issues.apache.org/jira/browse/MAPREDUCE-3897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated MAPREDUCE-3897: -- Status: Open (was: Patch Available) > capacity scheduler - maxActiveApplicationsPerUser calculation can be wrong > -- > > Key: MAPREDUCE-3897 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3897 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Eric Payne >Priority: Critical > Attachments: MAPREDUCE-3897-1.txt, MAPREDUCE-3897-1.txt > > > The capacity scheduler calculates the maxActiveApplications and the > maxActiveApplicationsPerUser based on the config > yarn.scheduler.capacity.maximum-applications or default 1. > MaxActiveApplications = max ( ceil ( clusterMemory/minAllocation * > maxAMResource% * absoluteMaxCapacity), 1) > MaxActiveAppsPerUser = max( ceil (maxActiveApplicationsComputedAbove * > (userLimit%/100) * userLimitFactor), 1) > maxActiveApplications is already multiplied by the queue absolute MAXIMUM > capacity, so if max capacity > capacity and if you have user limit factor 1 > (which is the default) and only 1 user is running, that user will not be > allowed to use over the queue capacity, so having it relative to MAX capacity > doesn't make sense. That user could easily end up in a deadlock and all its > space used by application masters. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3897) capacity scheduler - maxActiveApplicationsPerUser calculation can be wrong
[ https://issues.apache.org/jira/browse/MAPREDUCE-3897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated MAPREDUCE-3897: -- Attachment: MAPREDUCE-3897-1.txt > capacity scheduler - maxActiveApplicationsPerUser calculation can be wrong > -- > > Key: MAPREDUCE-3897 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3897 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Eric Payne >Priority: Critical > Attachments: MAPREDUCE-3897-1.txt, MAPREDUCE-3897-1.txt > > > The capacity scheduler calculates the maxActiveApplications and the > maxActiveApplicationsPerUser based on the config > yarn.scheduler.capacity.maximum-applications or default 1. > MaxActiveApplications = max ( ceil ( clusterMemory/minAllocation * > maxAMResource% * absoluteMaxCapacity), 1) > MaxActiveAppsPerUser = max( ceil (maxActiveApplicationsComputedAbove * > (userLimit%/100) * userLimitFactor), 1) > maxActiveApplications is already multiplied by the queue absolute MAXIMUM > capacity, so if max capacity > capacity and if you have user limit factor 1 > (which is the default) and only 1 user is running, that user will not be > allowed to use over the queue capacity, so having it relative to MAX capacity > doesn't make sense. That user could easily end up in a deadlock and all its > space used by application masters. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3897) capacity scheduler - maxActiveApplicationsPerUser calculation can be wrong
[ https://issues.apache.org/jira/browse/MAPREDUCE-3897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated MAPREDUCE-3897: -- Status: Patch Available (was: Open) > capacity scheduler - maxActiveApplicationsPerUser calculation can be wrong > -- > > Key: MAPREDUCE-3897 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3897 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Eric Payne >Priority: Critical > Attachments: MAPREDUCE-3897-1.txt, MAPREDUCE-3897-1.txt > > > The capacity scheduler calculates the maxActiveApplications and the > maxActiveApplicationsPerUser based on the config > yarn.scheduler.capacity.maximum-applications or default 1. > MaxActiveApplications = max ( ceil ( clusterMemory/minAllocation * > maxAMResource% * absoluteMaxCapacity), 1) > MaxActiveAppsPerUser = max( ceil (maxActiveApplicationsComputedAbove * > (userLimit%/100) * userLimitFactor), 1) > maxActiveApplications is already multiplied by the queue absolute MAXIMUM > capacity, so if max capacity > capacity and if you have user limit factor 1 > (which is the default) and only 1 user is running, that user will not be > allowed to use over the queue capacity, so having it relative to MAX capacity > doesn't make sense. That user could easily end up in a deadlock and all its > space used by application masters. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3897) capacity scheduler - maxActiveApplicationsPerUser calculation can be wrong
[ https://issues.apache.org/jira/browse/MAPREDUCE-3897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated MAPREDUCE-3897: -- Attachment: MAPREDUCE-3897-1.txt Changed calculation for maxActiveApplications. Old: maxActiveApplications = Maxtotal cluster mem) / minimumAllocation) * maxAMResourcePercent * absoluteMaxCapacity),1) New: maxActiveApplications = Maxtotal cluster mem) / minimumAllocation) * maxAMResourcePercent * absoluteCapacity),1) By using absoluteCapacity rather than absoluteMaxCapacity, the result is based on a per-queue capacity and prevents a small queue from hitting the deadlock condition of being filled with all AMs. This fix will be further enhanced when it is possible to specify maxAMResourcePercent on a per-queue basis (see MAPREDUCE:3893) > capacity scheduler - maxActiveApplicationsPerUser calculation can be wrong > -- > > Key: MAPREDUCE-3897 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3897 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Eric Payne >Priority: Critical > Attachments: MAPREDUCE-3897-1.txt > > > The capacity scheduler calculates the maxActiveApplications and the > maxActiveApplicationsPerUser based on the config > yarn.scheduler.capacity.maximum-applications or default 1. > MaxActiveApplications = max ( ceil ( clusterMemory/minAllocation * > maxAMResource% * absoluteMaxCapacity), 1) > MaxActiveAppsPerUser = max( ceil (maxActiveApplicationsComputedAbove * > (userLimit%/100) * userLimitFactor), 1) > maxActiveApplications is already multiplied by the queue absolute MAXIMUM > capacity, so if max capacity > capacity and if you have user limit factor 1 > (which is the default) and only 1 user is running, that user will not be > allowed to use over the queue capacity, so having it relative to MAX capacity > doesn't make sense. That user could easily end up in a deadlock and all its > space used by application masters. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
