[ https://issues.apache.org/jira/browse/YARN-10674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17299602#comment-17299602 ]
Peter Bacsko edited comment on YARN-10674 at 3/11/21, 2:43 PM: --------------------------------------------------------------- Ok, I did some research, I think we 3 options to completely disable preemption: 1) Set disable_preemption to "root", which will propagate down to other queues. 2) Remove "ProportionalCapacityPreemptionPolicy" from the list of policies. 3) Enable "observe_only" property. I think #1 is not really good, because it relies on a side-effect (propagation of a setting). The intention is not clear. #2 is perfectly acceptable and this goes to {{yarn-site.xml}} so it should be in {{FSYarnSiteConverter}}. #3 is also OK, but that goes to {{capacity-scheduler.xml}} and NOT to {{yarn-site.xml}}, I just verified it. So this should be placed somewhere else. So we can do: 1) Vote for what's best 2) Introduce a command line switch like "-dp" "\-\-disable-preemption" with values like "nopolicy" or "observeonly" and we pick a default value, eg. "nopolicy". So we can do something like: {noformat} yarn fs2cs --disable-preemption observeonly --yarnsiteconfig /path/to/yarn-site.xml .... {noformat} [~gandras] [~zhuqi] what do you think? was (Author: pbacsko): Ok, I did some research, I think we 3 options to completely disable preemption: 1) Set disable_preemption to "root", which will propagate down to other queues. 2) Remove "ProportionalCapacityPreemptionPolicy" from the list of policies. 3) Enable "observe_only" property. I think #1 is not really good, because it relies on a side-effect (propagation of a setting). The intention is not clear. #2 is perfectly acceptable and this goes to {{yarn-site.xml}} so it should be in {{FSYarnSiteConverter}}. #3 is also OK, but that goes to {{capacity-scheduler.xml}} and NOT in {{yarn-site.xml}}, I just verified it. So this should be placed somewhere else. So we can do: 1) Vote for what's best 2) Introduce a command line switch like "-dp" "\-\-disable-preemption" with values like "nopolicy" or "observeonly" and we pick a default value, eg. "nopolicy". So we can do something like: {noformat} yarn fs2cs --disable-preemption observeonly --yarnsiteconfig /path/to/yarn-site.xml .... {noformat} [~gandras] [~zhuqi] what do you think? > fs2cs: should support auto created queue deletion. > -------------------------------------------------- > > Key: YARN-10674 > URL: https://issues.apache.org/jira/browse/YARN-10674 > Project: Hadoop YARN > Issue Type: Sub-task > Reporter: Qi Zhu > Assignee: Qi Zhu > Priority: Major > Labels: fs2cs > Attachments: YARN-10674.001.patch, YARN-10674.002.patch, > YARN-10674.003.patch, YARN-10674.004.patch, YARN-10674.005.patch, > YARN-10674.006.patch > > > In FS the auto deletion check interval is 10s. > {code:java} > @Override > public void onCheck() { > queueMgr.removeEmptyDynamicQueues(); > queueMgr.removePendingIncompatibleQueues(); > } > while (running) { > try { > synchronized (this) { > reloadListener.onCheck(); > } > ... > Thread.sleep(reloadIntervalMs); > } > /** Time to wait between checks of the allocation file */ > public static final long ALLOC_RELOAD_INTERVAL_MS = 10 * 1000;{code} -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org