[ https://issues.apache.org/jira/browse/YARN-8509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16568258#comment-16568258 ]
Eric Payne commented on YARN-8509: ---------------------------------- [~Zian Chen]. Unfortunately, this code cannot go in as it is. This change causes very many unnecessary preemptions, which causes a lot of lost work. {code} Resource minpendingConsideringUserLimit = Resources.componentwiseMin( userNameToHeadroom.get(userName), pending); Resources.addTo(totalPendingConsideringUserLimit, - minpendingConsideringUserLimit); + pending); Resources.subtractFrom(userNameToHeadroom.get(userName), minpendingConsideringUserLimit); } {code} The purpose of doing a min between the user's pending and the user's headroom is to prevent a preemption that cannot be given to the user because the user has reached its headroom limit. > Fix UserLimit calculation for preemption to balance scenario after queue > satisfied > ------------------------------------------------------------------------------------ > > Key: YARN-8509 > URL: https://issues.apache.org/jira/browse/YARN-8509 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn > Reporter: Zian Chen > Assignee: Zian Chen > Priority: Major > Attachments: YARN-8509.001.patch, YARN-8509.002.patch, > YARN-8509.003.patch > > > In LeafQueue#getTotalPendingResourcesConsideringUserLimit, we calculate total > pending resource based on user-limit percent and user-limit factor which will > cap pending resource for each user to the minimum of user-limit pending and > actual pending. This will prevent queue from taking more pending resource to > achieve queue balance after all queue satisfied with its ideal allocation. > > We need to change the logic to let queue pending can go beyond userlimit. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org