[
https://issues.apache.org/jira/browse/YARN-10022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023840#comment-17023840
]
Prabhu Joseph commented on YARN-10022:
--------------------------------------
[~kmarton] On further testing, have found an issue. The below code starts a
CapacityScheduler instance with schedulerConf which only has scheduler related
configs and no yarn configs. And hence CS has default yarn configs and reads CS
config from local capacity-scheduler.xml instead of one from ZK Config Store.
The stale config in local CS config file causes inconsistent behavior on some
queue opertaions.
+ CapacitySchedulerConfigValidator.validateCSConfiguration(schedulerConf,
+ newConfig, rm.getRMContext());
*Repro:*
{code:java}
1. Have a queue root.v1 with ordering-policy fifo in both local
capacity-scheduler.xml and ZK Config Store.
2. Add a queue root.v1.v2
3. Add a queue root.v1.v2.v3
The step 3 fails with below
Validation failed for Add queue operation! Error message: CapacityScheduler
configuration validation failed:
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Unable to construct
queue ordering policy=fifo queue=root.v1
{code}
The fix is to pass the current yarn config to CS so that it uses scheduler
config from ZK store.
Configuration yarnConf = ((CapacityScheduler) scheduler).getConf();
CapacitySchedulerConfigValidator.validateCSConfiguration(yarnConf, newConfig,
rm.getRMContext());
Have attached [^YARN-10022.003.patch] with the fix.
> Create RM Rest API to validate a CapacityScheduler Configuration
> ----------------------------------------------------------------
>
> Key: YARN-10022
> URL: https://issues.apache.org/jira/browse/YARN-10022
> Project: Hadoop YARN
> Issue Type: New Feature
> Reporter: Kinga Marton
> Assignee: Kinga Marton
> Priority: Major
> Attachments: YARN-10022.001.patch, YARN-10022.002.patch,
> YARN-10022.003.patch, YARN-10022.WIP.patch, YARN-10022.WIP2.patch
>
>
> RMWebService should expose a new api which gets a CapacityScheduler
> Configuration as an input, validates it and returns success / failure.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
