[ https://issues.apache.org/jira/browse/YARN-5889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15712097#comment-15712097 ]
Eric Payne commented on YARN-5889: ---------------------------------- bq. Any change in resource (allocation and release of container etc) for a given user could set a state variable. This will set off by the computation thread if next cycle falls immediate. Just so we're on the same page and what I'm suggesting is more clear. I'm suggesting that we remove the computation thread ({{ComputeUserLimitAsyncThread}}), and when {{LeafQueue#assignContainers}} or {{LeafQueue#releaseResource}} is called, we can set the flag in {{UserToPartitionRecord}}. This flag should also be set within queues are refreshed. {{getComputedUserLimit}} would then be called as it is now, and compute user limit resource if the flag is set. bq. Its not ideal to ask allocation thread to hold till computation. So by seeing this state variable, we might need to compute user-limit in same allocation thread. I recognize that my suggested approach does the calculation during the allocation thread. My assertion would be that # this calculation is being done currently in the allocation thread # my suggested approach is an optimization that would decrease (sometimes greatly) the number of times the calculation happens # my suggested approach would be more accurate than with a separate computation thread. > Improve user-limit calculation in capacity scheduler > ---------------------------------------------------- > > Key: YARN-5889 > URL: https://issues.apache.org/jira/browse/YARN-5889 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler > Reporter: Sunil G > Assignee: Sunil G > Attachments: YARN-5889.v0.patch, YARN-5889.v1.patch, > YARN-5889.v2.patch > > > Currently user-limit is computed during every heartbeat allocation cycle with > a write lock. To improve performance, this tickets is focussing on moving > user-limit calculation out of heartbeat allocation flow. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org