[ 
https://issues.apache.org/jira/browse/YARN-10674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17299602#comment-17299602
 ] 

Peter Bacsko edited comment on YARN-10674 at 3/11/21, 2:43 PM:
---------------------------------------------------------------

Ok, I did some research, I think we 3 options to completely disable preemption:

1) Set disable_preemption to "root", which will propagate down to other queues.
2) Remove "ProportionalCapacityPreemptionPolicy" from the list of policies.
3) Enable "observe_only" property.

I think #1 is not really good, because it relies on a side-effect (propagation 
of a setting). The intention is not clear.

#2 is perfectly acceptable and this goes to {{yarn-site.xml}} so it should be 
in {{FSYarnSiteConverter}}.
#3 is also OK, but that goes to {{capacity-scheduler.xml}} and NOT to 
{{yarn-site.xml}}, I just verified it. So this should be placed somewhere else.

So we can do:
1) Vote for what's best
2) Introduce a command line switch like "-dp" "\-\-disable-preemption" with 
values like "nopolicy" or "observeonly" and we pick a default value, eg. 
"nopolicy". So we can do something like:
{noformat}
yarn fs2cs --disable-preemption observeonly --yarnsiteconfig 
/path/to/yarn-site.xml ....
{noformat}

[~gandras] [~zhuqi] what do you think?


was (Author: pbacsko):
Ok, I did some research, I think we 3 options to completely disable preemption:

1) Set disable_preemption to "root", which will propagate down to other queues.
2) Remove "ProportionalCapacityPreemptionPolicy" from the list of policies.
3) Enable "observe_only" property.

I think #1 is not really good, because it relies on a side-effect (propagation 
of a setting). The intention is not clear.

#2 is perfectly acceptable and this goes to {{yarn-site.xml}} so it should be 
in {{FSYarnSiteConverter}}.
#3 is also OK, but that goes to {{capacity-scheduler.xml}} and NOT in 
{{yarn-site.xml}}, I just verified it. So this should be placed somewhere else.

So we can do:
1) Vote for what's best
2) Introduce a command line switch like "-dp" "\-\-disable-preemption" with 
values like "nopolicy" or "observeonly" and we pick a default value, eg. 
"nopolicy". So we can do something like:
{noformat}
yarn fs2cs --disable-preemption observeonly --yarnsiteconfig 
/path/to/yarn-site.xml ....
{noformat}

[~gandras] [~zhuqi] what do you think?

> fs2cs: should support auto created queue deletion.
> --------------------------------------------------
>
>                 Key: YARN-10674
>                 URL: https://issues.apache.org/jira/browse/YARN-10674
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Qi Zhu
>            Assignee: Qi Zhu
>            Priority: Major
>              Labels: fs2cs
>         Attachments: YARN-10674.001.patch, YARN-10674.002.patch, 
> YARN-10674.003.patch, YARN-10674.004.patch, YARN-10674.005.patch, 
> YARN-10674.006.patch
>
>
> In FS the auto deletion check interval is 10s.
> {code:java}
> @Override
> public void onCheck() {
>   queueMgr.removeEmptyDynamicQueues();
>   queueMgr.removePendingIncompatibleQueues();
> }
> while (running) {
>   try {
>     synchronized (this) {
>       reloadListener.onCheck();
>     }
> ...
> Thread.sleep(reloadIntervalMs);
> }
> /** Time to wait between checks of the allocation file */
> public static final long ALLOC_RELOAD_INTERVAL_MS = 10 * 1000;{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

Reply via email to