[ https://issues.apache.org/jira/browse/YARN-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14154057#comment-14154057 ]
Craig Welch commented on YARN-1198: ----------------------------------- Sorry, the above was off - the conversation happened offline, here was the tweak to .7 that Jian suggested: Hi Craig, I looked at your patch again. It's similar to what I thought. One thing is that now that headRoom is not application specific, it doesn't belong to application any more. We may make a member of LeafQueue#User. From CapacityScheduler#allocate, directly call LeafQueue #getAndCalculateHeadRoom , not going through SchedulerApplicationAttempt route to get the HeadRoom. I think this is simpler. do you think this will work? > We may make a member of LeafQueue#User. To clarify: make the headRoom a > variable of LeafQueue#User, and remove that from SchedulerAttempt we might, in this approach, do what we are doing in .7 but without the HeadroomProvider at all... I'm going to give a go at this... > Capacity Scheduler headroom calculation does not work as expected > ----------------------------------------------------------------- > > Key: YARN-1198 > URL: https://issues.apache.org/jira/browse/YARN-1198 > Project: Hadoop YARN > Issue Type: Bug > Reporter: Omkar Vinit Joshi > Assignee: Craig Welch > Attachments: YARN-1198.1.patch, YARN-1198.2.patch, YARN-1198.3.patch, > YARN-1198.4.patch, YARN-1198.5.patch, YARN-1198.6.patch, YARN-1198.7.patch, > YARN-1198.8.patch > > > Today headroom calculation (for the app) takes place only when > * New node is added/removed from the cluster > * New container is getting assigned to the application. > However there are potentially lot of situations which are not considered for > this calculation > * If a container finishes then headroom for that application will change and > should be notified to the AM accordingly. > * If a single user has submitted multiple applications (app1 and app2) to the > same queue then > ** If app1's container finishes then not only app1's but also app2's AM > should be notified about the change in headroom. > ** Similarly if a container is assigned to any applications app1/app2 then > both AM should be notified about their headroom. > ** To simplify the whole communication process it is ideal to keep headroom > per User per LeafQueue so that everyone gets the same picture (apps belonging > to same user and submitted in same queue). > * If a new user submits an application to the queue then all applications > submitted by all users in that queue should be notified of the headroom > change. > * Also today headroom is an absolute number ( I think it should be normalized > but then this is going to be not backward compatible..) > * Also when admin user refreshes queue headroom has to be updated. > These all are the potential bugs in headroom calculations -- This message was sent by Atlassian JIRA (v6.3.4#6332)