[ https://issues.apache.org/jira/browse/MAPREDUCE-3341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13147266#comment-13147266 ]
jirapos...@reviews.apache.org commented on MAPREDUCE-3341: ---------------------------------------------------------- bq. On 2011-11-08 17:27:47, Robert Evans wrote: bq. > Mostly looks good, just one minor nit. Thanks Bobby for noticing that! Below is an example log output. I will upload a new patch removing the unnecssary whitespace. 2011-11-09 20:20:19,309 INFO capacity.LeafQueue (LeafQueue.java:setupQueueConfigs(250)) - Initializing default capacity = 0.4[ = (float) configuredCapacity / 100 ] asboluteCapacity = 0.4[ = parentAbsoluteCapacity * capacity ] maxCapacity = -0.01[ = configuredMaxCapacity ] absoluteMaxCapacity = 3.4028235E38[ = Float.MAX_VALUE if maximumCapacity undefined, = (parentAbsoluteCapacity * maximumCapacity) / 100 otherwise ] userLimit = 100[ = configuredUserLimit ] userLimitFactor = 1.0[ = configuredUserLimitFactor ] maxApplications = 4000[ = (int)(configuredMaximumSystemApplications * absoluteCapacity) ] maxApplicationsPerUser = 4000[ = (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ] maxActiveApplications = 1[ = max( (int)((clusterResourceMemory / (float)DEFAULT_AM_RESOURCE) * maxAMResourcePercent * absoluteCapacity), 1) ] maxActiveApplicationsPerUser = 1[ = (int)(maxActiveApplications * (userLimit / 100.0f) * userLimitFactor) ] utilization = 0.0[ = usedResourcesMemory / queueLimit ] usedCapacity = 0.0[ = usedResourcesMemory / (clusterResourceMemory * capacity) ] maxAMResourcePercent = 0.1[ = configuredMaximumAMResourcePercent ] minimumAllocationFactor = 0.9[ = (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ] numContainers = 0[ = currentNumContainers ] state = RUNNING[ = configuredState ] acls = ADMINISTER_QUEUE:*SUBMIT_APPLICATIONS:*[ = configuredAcls ] - Anupam ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2775/#review3112 ----------------------------------------------------------- On 2011-11-09 20:34:10, Anupam Seth wrote: bq. bq. ----------------------------------------------------------- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/2775/ bq. ----------------------------------------------------------- bq. bq. (Updated 2011-11-09 20:34:10) bq. bq. bq. Review request for Robert Evans and Jonathan Eagles. bq. bq. bq. Summary bq. ------- bq. bq. Enhance logging for queues in capacity scheduler bq. bq. bq. This addresses bug MAPREDUCE-3341. bq. https://issues.apache.org/jira/browse/MAPREDUCE-3341 bq. bq. bq. Diffs bq. ----- bq. bq. branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java 1198816 bq. bq. Diff: https://reviews.apache.org/r/2775/diff bq. bq. bq. Testing bq. ------- bq. bq. bq. Thanks, bq. bq. Anupam bq. bq. > Enhance logging of initalized queue limit values > ------------------------------------------------ > > Key: MAPREDUCE-3341 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3341 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 > Affects Versions: 0.23.0 > Reporter: Anupam Seth > Assignee: Anupam Seth > Attachments: MAPREDUCE-3341-branch_0_23.patch > > > Currently the RM log shows only a partial set of the limits that are > configured when a queue is initialized / reinitialized. > For example, this is what is currently shown in the RM log for an initialized > queue: > # <datestamp> INFO > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue: > Initializing > default, capacity=0.25, asboluteCapacity=0.25, maxCapacity=25.0, > asboluteMaxCapacity=0.25, userLimit=100, > userLimitFactor=20.0, maxApplications=2500, maxApplicationsPerUser=50000, > state=RUNNING, > acls=ADMINISTER_QUEUE:*SUBMIT_JOB:*ADMINISTER_JOBS:* > Breaking down the line above, shows: > capacity=0.25 > asboluteCapacity=0.25 > maxCapacity=25.0 > asboluteMaxCapacity=0.25 > userLimit=100 > userLimitFactor=20.0 > maxApplications=2500 > maxApplicationsPerUser=50000 > It might be nice if we could include more information such as > maxActiveApplications, maxActiveApplicationsPerUser, utilization, and > usedCapacity along with information on how each of these is computed (i.e. > formulae used) (Thanks to Phil Su for requesting this). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira