[ https://issues.apache.org/jira/browse/YARN-5889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15812187#comment-15812187 ]
Wangda Tan commented on YARN-5889: ---------------------------------- Thanks [~sunilg] for providing the patch and reviews from [~eepayne]. Quickly scanned the patch, If everybody agrees with approach in my comment: https://issues.apache.org/jira/browse/YARN-5889?focusedCommentId=15749129&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15749129 {code} active-user-limit = max( resource-used-by-active-users / #active-users, queue-capacity * MULP ) {code} We can: 1) Update {{resource-used-by-active-users}} when any user enters/exits set of active users. 2) Update #active-users (using approaches like ActiveUsersManager approach) Both of #1/#2 can be done within the same thread, and since we will have an identical UL for all the active users, I think this can be done without adding a new thread. Please correct me if I'm wrong. It looks like Sunil and Eric have different implementation suggestions, I have looked at details of the two approaches. However I think the goals are: a. Can always get up-to-date UL b. Avoid re-computation of UL as much as possible. Any approach can achieve above goals is good to me. And in addition, we can add a UsersManager to LQ to manager all user-related information, such as user-limit / active-user-limit / #active-users. Ideally, LQ/PCPP should get activeUserLimit / userLimit from UsersManager And for last, we can look at logics for #active-users per partition, now we have an identical #active-users for all partition, which causes some problems. Since we will make a major changes to logics around this, it could be a good chance to fix the problem as well. > Improve user-limit calculation in capacity scheduler > ---------------------------------------------------- > > Key: YARN-5889 > URL: https://issues.apache.org/jira/browse/YARN-5889 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler > Reporter: Sunil G > Assignee: Sunil G > Attachments: YARN-5889.0001.patch, > YARN-5889.0001.suggested.patchnotes, YARN-5889.v0.patch, YARN-5889.v1.patch, > YARN-5889.v2.patch > > > Currently user-limit is computed during every heartbeat allocation cycle with > a write lock. To improve performance, this tickets is focussing on moving > user-limit calculation out of heartbeat allocation flow. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org