[ https://issues.apache.org/jira/browse/YARN-7535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16266791#comment-16266791 ]
Wilfred Spiegelenburg commented on YARN-7535: --------------------------------------------- The demand for an application is not limited and is we do not stop adding up demand at any limit for an application. This is purely about the maximum demand a queue is assigned for the purpose of scheduling. Showing a demand in a queue that is higher than what could ever be allocated in the queue also does not make sense from a scheduling perspective. I agree that it is one of the possible factors you look at from a tuning perspective. Demand should not be the one and only number to base a queue size on. An applications demand is just the requested resources for the Application Master until the application runs. When the AM is not started there is no request for any other resources beside that one AM container. There is thus no demand beside that one container. This means that the demand in a queue is of limited value for tuning the size of a queue. Non runnable applications always skew the demand number. If you run into the maximum size of a queue demand will be skewed unless all applications in all queues are running. Most likely it will give the wrong info and your tuning would be off. If you are not running into the maximum queue size and still have unsatisfied demand something else is going on. Again demand will not be the value you are tuning on in that case. > We should display origin value of demand in fair scheduler page > --------------------------------------------------------------- > > Key: YARN-7535 > URL: https://issues.apache.org/jira/browse/YARN-7535 > Project: Hadoop YARN > Issue Type: Bug > Components: scheduler > Reporter: YunFan Zhou > Assignee: YunFan Zhou > > The value of *demand* of leaf queue that we now view on the fair scheduler > page shows only the value of *maxResources* when the demand value is greater > than *maxResources*. It doesn't reflect the real situation. Most of the time, > when we expand the queue, we often rely on seeing the current demand real > value. > {code:java} > private void updateDemandForApp(FSAppAttempt sched, Resource maxRes) { > sched.updateDemand(); > Resource toAdd = sched.getDemand(); > if (LOG.isDebugEnabled()) { > LOG.debug("Counting resource from " + sched.getName() + " " + toAdd > + "; Total resource consumption for " + getName() + " now " > + demand); > } > demand = Resources.add(demand, toAdd); > demand = Resources.componentwiseMin(demand, maxRes); > } > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org