[
https://issues.apache.org/jira/browse/YARN-5889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated YARN-5889:
--------------------------
Attachment: YARN-5889.0009.patch
Thanks [~leftnoteasy] for helping to review the patch thoroughly. Updating a
new patch
bq.4) isRecomputeNeeded:
I am slightly confused here. I think we might need null check. I ll help to
share detailed view for that.
Assume that there are no precomputed user-limit at the start when RM is started
or queue is refreshed. So all cache ill be empty, and we ll do our first
computation when a container request comes.
So in this case, userLimitPerSchedulingMode will be null. And we ll do a
recompute and then userLimitPerSchedulingMode will have some entires. So a null
check is needed at the very beginning scenario. I can see whether this check
can be done outside or note. Am i missing something here? pls help to share
your view.
bq.And also, we don't need latestVersionOfUserCount, instead we should call
latestVersionOfUsersState.get().
userLimitNeedsRecompute or getLatestVersionOfUsersState are not writeLock
protected. Hence in getComputedResourceLimitForAll/ActiveUsers , it may be
possible that latestVersionOfUsersState may change within writeLock block while
operating. Since we save the saved version of latestVersionOfUserCount to
update local cache (per partition nd sch mode), even though some other thread
changed the real latestVersionOfUsersState, cache will be invalidate it
correctly. Pls pool in your thoughts.
> Improve user-limit calculation in capacity scheduler
> ----------------------------------------------------
>
> Key: YARN-5889
> URL: https://issues.apache.org/jira/browse/YARN-5889
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
> Reporter: Sunil G
> Assignee: Sunil G
> Attachments: YARN-5889.0001.patch,
> YARN-5889.0001.suggested.patchnotes, YARN-5889.0002.patch,
> YARN-5889.0003.patch, YARN-5889.0004.patch, YARN-5889.0005.patch,
> YARN-5889.0006.patch, YARN-5889.0007.patch, YARN-5889.0008.patch,
> YARN-5889.0009.patch, YARN-5889.v0.patch, YARN-5889.v1.patch,
> YARN-5889.v2.patch
>
>
> Currently user-limit is computed during every heartbeat allocation cycle with
> a write lock. To improve performance, this tickets is focussing on moving
> user-limit calculation out of heartbeat allocation flow.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
