[
https://issues.apache.org/jira/browse/YARN-5889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819482#comment-15819482
]
Eric Payne commented on YARN-5889:
----------------------------------
[~sunilg],
- Shoule {{resetUserAddedOrRemoved}} just be {{setUserAddedOrRemoved}}
- {{LeafQueue}}: I think {{totalUserConsumedRatio}} should be removed, since it'
s not used.
- {{LeafQueue#recalculateULCount}} / {{UsersManager#User#cachedULCount}}: I
know I came up with the name originally, but I think a better name would be
{{recalculateUL}}
- {{getComputedActiveUserLimit}} / {{getComputedUserLimit}}: User's
{{cachedULCount}} needs to be updated when the UL is recomputed or else it will
always be out of sync and will always be recomputed:
{code}
if (userLimitPerSchedulingMode == null
|| user.getCachedULCount() != lQueue.getRecalculateULCount()) {
userLimitPerSchedulingMode = reComputeUserLimits(rc, userName,
nodePartition, clusterResource, false);
user.setCachedULCount(lQueue.getRecalculateULCount());
}
{code}
[~leftnoteasy],
bq. User#setCachedCount, should we invalidateUL for the user who
allocates/releases containers, or we should invalidate all user limit? I think
the latter one is more safe to me.
Yes, unfortunately, I think that once the queue goes above its guarantee, the
ratio will change when containers are allocated or released. We may be able to
do an optimization to only reset the specific user's count when the queue is
under its guarantee and all users when it is over, but that may not be worth
the added complexity.
> Improve user-limit calculation in capacity scheduler
> ----------------------------------------------------
>
> Key: YARN-5889
> URL: https://issues.apache.org/jira/browse/YARN-5889
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
> Reporter: Sunil G
> Assignee: Sunil G
> Attachments: YARN-5889.0001.patch,
> YARN-5889.0001.suggested.patchnotes, YARN-5889.0002.patch,
> YARN-5889.v0.patch, YARN-5889.v1.patch, YARN-5889.v2.patch
>
>
> Currently user-limit is computed during every heartbeat allocation cycle with
> a write lock. To improve performance, this tickets is focussing on moving
> user-limit calculation out of heartbeat allocation flow.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
