[ https://issues.apache.org/jira/browse/YARN-5889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15824442#comment-15824442 ]
Wangda Tan commented on YARN-5889: ---------------------------------- Thanks [~sunilg] for updating the patch. Some minor comments: 1) I made a mistake of my previous comment: recalculateQueueUsageRatio should belong to LQ, and it invokes UM#calculateUserUsageRatio. 2) UM#latestVersionOfUserCount is a little bit confusing, since it doesn't only consider user count, how about rename it to latestVersionOfUsersState? (And update comments, getters/setters accordingly). 3) ActiveUM#isUserAddedOrRemoved, parameter is not required since it is always false. 4) Locks of UM: - Add writeLock for following code block: {code} long latestVersionOfUserCount = getLatestVersionOfUserCount(); if (isRecomputeNeeded(user, userLimitPerSchedulingMode, latestVersionOfUserCount, schedulingMode)) { // recompute userLimitPerSchedulingMode = reComputeUserLimits(rc, userName, nodePartition, clusterResource, schedulingMode, true); // update user count to cache so that we can avoid recompute if no major // changes. user.setLocalVersionOfUserCount(latestVersionOfUserCount); } {code} For getComputedResourceLimitFor*Users. This can make sure update of UL can be sequentially done and version increases while updating UL can always trigger another update. 5) Regarding to UserToPartitionRecord and computeUserLimit: My gut feeling is that we can optimize/simplify this part a little bit, let's revisit it after this patch: This patch could lead to wrong REST UL computation (returns all user limit instead of active user limit), and it recomputes UL for every user instead of every partition. > Improve user-limit calculation in capacity scheduler > ---------------------------------------------------- > > Key: YARN-5889 > URL: https://issues.apache.org/jira/browse/YARN-5889 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler > Reporter: Sunil G > Assignee: Sunil G > Attachments: YARN-5889.0001.patch, > YARN-5889.0001.suggested.patchnotes, YARN-5889.0002.patch, > YARN-5889.0003.patch, YARN-5889.0004.patch, YARN-5889.v0.patch, > YARN-5889.v1.patch, YARN-5889.v2.patch > > > Currently user-limit is computed during every heartbeat allocation cycle with > a write lock. To improve performance, this tickets is focussing on moving > user-limit calculation out of heartbeat allocation flow. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org