[
https://issues.apache.org/jira/browse/YARN-10674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17298842#comment-17298842
]
Peter Bacsko commented on YARN-10674:
-------------------------------------
Ok, here is what I found:
1. {{RM_SCHEDULER_ENABLE_MONITORS}} --> ok, this can be set to "true" in all
cases.
2. If FS preemption is disabled --> there is a property which is better than
configuring the "root" queue. If FS preemption is disabled
({{yarn.scheduler.fair.preemption}} = {{false}}),
then we should generate
{{yarn.resourcemanager.monitor.capacity.preemption.observe_only}} = {{true}}.
This means that we have the monitor thread running but we don't do any
preemption. So we don't need to set "root.disable_preemption".
3. As I mentioned, the {{Configuration}} object is empty. The problem is, in
order to use the preemption, we need to set the preemption policy, which is
missing right now. So, if FS preemption is enabled, this line must be added:
{noformat}
if (conf.getBoolean(FairSchedulerConfiguration.PREEMPTION,
FairSchedulerConfiguration.DEFAULT_PREEMPTION)) {
yarnSiteConfig.set(YarnConfiguration.RM_SCHEDULER_MONITOR_POLICIES,
ProportionalCapacityPreemptionPolicy.class.getCanonicalName();
...
{noformat}
So, the modified code should look like this:
{noformat}
yarnSiteConfig.setBoolean(
YarnConfiguration.RM_SCHEDULER_ENABLE_MONITORS, true);
if (conf.getBoolean(FairSchedulerConfiguration.PREEMPTION,
FairSchedulerConfiguration.DEFAULT_PREEMPTION)) {
yarnSiteConfig.set(YarnConfiguration.RM_SCHEDULER_MONITOR_POLICIES,
ProportionalCapacityPreemptionPolicy.class.getCanonicalName();
...
} else {
// no preemption
yarnSiteConfig.setBoolean(CapacitySchedulerConfiguration.PREEMPTION_OBSERVE_ONLY,
true);
}
// new code comes here
if (!userPercentage) {
String policies =
yarnSiteConfig.get(YarnConfiguration.RM_SCHEDULER_MONITOR_POLICIES);
if (policies == null) {
...
{noformat}
Please modify the test cases accordingly and the checkstyle issues also.
> fs2cs: should support auto created queue deletion.
> --------------------------------------------------
>
> Key: YARN-10674
> URL: https://issues.apache.org/jira/browse/YARN-10674
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Qi Zhu
> Assignee: Qi Zhu
> Priority: Major
> Labels: fs2cs
> Attachments: YARN-10674.001.patch, YARN-10674.002.patch,
> YARN-10674.003.patch
>
>
> In FS the auto deletion check interval is 10s.
> {code:java}
> @Override
> public void onCheck() {
> queueMgr.removeEmptyDynamicQueues();
> queueMgr.removePendingIncompatibleQueues();
> }
> while (running) {
> try {
> synchronized (this) {
> reloadListener.onCheck();
> }
> ...
> Thread.sleep(reloadIntervalMs);
> }
> /** Time to wait between checks of the allocation file */
> public static final long ALLOC_RELOAD_INTERVAL_MS = 10 * 1000;{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
