[
https://issues.apache.org/jira/browse/YARN-10674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17299602#comment-17299602
]
Peter Bacsko edited comment on YARN-10674 at 3/11/21, 3:25 PM:
---------------------------------------------------------------
Ok, I did some research, I think we have 3 options to completely disable
preemption:
1) Set disable_preemption to "root", which will propagate down to other queues.
2) Remove "ProportionalCapacityPreemptionPolicy" from the list of policies.
3) Enable "observe_only" property.
I think #1 is not really good, because it relies on a side-effect (propagation
of a setting). The intention is not clear.
#2 is perfectly acceptable and this goes to {{yarn-site.xml}} so it should be
in {{FSYarnSiteConverter}}.
#3 is also OK, but that goes to {{capacity-scheduler.xml}} and NOT to
{{yarn-site.xml}}, I just verified it. So this should be placed somewhere else.
So we can do:
1) Vote for what's best
2) Introduce a command line switch like "-dp" "\-\-disable-preemption" with
values like "nopolicy" or "observeonly" and we pick a default value, eg.
"nopolicy". So we can do something like:
{noformat}
yarn fs2cs --disable-preemption observeonly --yarnsiteconfig
/path/to/yarn-site.xml ....
{noformat}
[~gandras] [~zhuqi] what do you think?
was (Author: pbacsko):
Ok, I did some research, I think we 3 options to completely disable preemption:
1) Set disable_preemption to "root", which will propagate down to other queues.
2) Remove "ProportionalCapacityPreemptionPolicy" from the list of policies.
3) Enable "observe_only" property.
I think #1 is not really good, because it relies on a side-effect (propagation
of a setting). The intention is not clear.
#2 is perfectly acceptable and this goes to {{yarn-site.xml}} so it should be
in {{FSYarnSiteConverter}}.
#3 is also OK, but that goes to {{capacity-scheduler.xml}} and NOT to
{{yarn-site.xml}}, I just verified it. So this should be placed somewhere else.
So we can do:
1) Vote for what's best
2) Introduce a command line switch like "-dp" "\-\-disable-preemption" with
values like "nopolicy" or "observeonly" and we pick a default value, eg.
"nopolicy". So we can do something like:
{noformat}
yarn fs2cs --disable-preemption observeonly --yarnsiteconfig
/path/to/yarn-site.xml ....
{noformat}
[~gandras] [~zhuqi] what do you think?
> fs2cs: should support auto created queue deletion.
> --------------------------------------------------
>
> Key: YARN-10674
> URL: https://issues.apache.org/jira/browse/YARN-10674
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Qi Zhu
> Assignee: Qi Zhu
> Priority: Major
> Labels: fs2cs
> Attachments: YARN-10674.001.patch, YARN-10674.002.patch,
> YARN-10674.003.patch, YARN-10674.004.patch, YARN-10674.005.patch,
> YARN-10674.006.patch
>
>
> In FS the auto deletion check interval is 10s.
> {code:java}
> @Override
> public void onCheck() {
> queueMgr.removeEmptyDynamicQueues();
> queueMgr.removePendingIncompatibleQueues();
> }
> while (running) {
> try {
> synchronized (this) {
> reloadListener.onCheck();
> }
> ...
> Thread.sleep(reloadIntervalMs);
> }
> /** Time to wait between checks of the allocation file */
> public static final long ALLOC_RELOAD_INTERVAL_MS = 10 * 1000;{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
