[
https://issues.apache.org/jira/browse/YARN-10674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17298156#comment-17298156
]
Peter Bacsko commented on YARN-10674:
-------------------------------------
[~zhuqi] this is very interesting. If we set RM Monitors to enabled, it means
that system-wide preemption is always enabled, too:
AbstractCSQueue:
{noformat}
private boolean isQueueHierarchyPreemptionDisabled(CSQueue q,
CapacitySchedulerConfiguration configuration) {
boolean systemWidePreemption =
csContext.getConfiguration()
.getBoolean(YarnConfiguration.RM_SCHEDULER_ENABLE_MONITORS,
YarnConfiguration.DEFAULT_RM_SCHEDULER_ENABLE_MONITORS);
CSQueue parentQ = q.getParent();
// If the system-wide preemption switch is turned off, all of the queues in
// the qPath hierarchy have preemption disabled, so return true.
if (!systemWidePreemption) return true;
{noformat}
However, you already added a policy in YARN-10623, so looks like this property
always has to be enabled in weight mode. But what if we convert an FS
configuration which disabled preemption completely?
I think the best thing we can do right now is that we disable preemption for
"root", which will propagate to all other parent queues.
So I suggest the following approach:
1. In percentage conversion mode, do not enable RM monitors by default,
because it's not needed.
2. In weight mode (which is the default now), we have to enable it. But if
"yarn.scheduler.fair.preemption" is false, then
"yarn.scheduler.capacity.root.disable_preemption" must be set to true, but only
for "root". This can be done in {{FSQueueConverter}}.
cc [~bteke] [~gandras] [~snemeth], not sure if this is a good approach, but I
can't see anything better.
> fs2cs: should support auto created queue deletion.
> --------------------------------------------------
>
> Key: YARN-10674
> URL: https://issues.apache.org/jira/browse/YARN-10674
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Qi Zhu
> Assignee: Qi Zhu
> Priority: Major
> Labels: fs2cs
> Attachments: YARN-10674.001.patch, YARN-10674.002.patch
>
>
> In FS the auto deletion check interval is 10s.
> {code:java}
> @Override
> public void onCheck() {
> queueMgr.removeEmptyDynamicQueues();
> queueMgr.removePendingIncompatibleQueues();
> }
> while (running) {
> try {
> synchronized (this) {
> reloadListener.onCheck();
> }
> ...
> Thread.sleep(reloadIntervalMs);
> }
> /** Time to wait between checks of the allocation file */
> public static final long ALLOC_RELOAD_INTERVAL_MS = 10 * 1000;{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
