[
https://issues.apache.org/jira/browse/YARN-6840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16167196#comment-16167196
]
Jonathan Hung edited comment on YARN-6840 at 9/15/17 1:20 AM:
--------------------------------------------------------------
Thanks [~leftnoteasy] uploaded 006 patch:
bq. Then I suggest to add this as a Javadoc to the base class's method, this
should be respected by all future implementations.
Added
bq. Avoid call CapacityScheduler methods from provider.
Done. Also exposed the ConfigurationMutationACLPolicy in
MutableConfigurationProvider, since the policy calls cs.getQueue to check queue
ACLs. (ACL checking is also done in RMWebServices just like the
logging/refreshing/confirming.)
bq. Discard pending mutation during recovery. (Make sure that
AdminService#refreshAll won't include the pending mutation too).
Done. Also got rid of the transaction id logic, since there is only one pending
mutation at any time. Also added some stuff in
TestZKConfigurationStore#testFailoverReadsFromUpdatedStore to ensure pending
mutation is not read on failover.
bq. MutableScheduler#updateConfiguration can be removed. Instead we need an API
to get confProvider.
Done, added aclPolicy/logging/confirming API to MutableConfigurationProvider
Also since the pendingMutation/YarnConfigurationStore APIs changed, it's
possible some of the LeveldbConfigurationStore logic broke, but I will address
this in YARN-7046.
was (Author: jhung):
Thanks [~leftnoteasy],
bq. Then I suggest to add this as a Javadoc to the base class's method, this
should be respected by all future implementations.
Added
bq. Avoid call CapacityScheduler methods from provider.
Done. Also exposed the ConfigurationMutationACLPolicy in
MutableConfigurationProvider, since the policy calls cs.getQueue to check queue
ACLs. (ACL checking is also done in RMWebServices just like the
logging/refreshing/confirming.)
bq. Discard pending mutation during recovery. (Make sure that
AdminService#refreshAll won't include the pending mutation too).
Done. Also got rid of the transaction id logic, since there is only one pending
mutation at any time. Also added some stuff in
TestZKConfigurationStore#testFailoverReadsFromUpdatedStore to ensure pending
mutation is not read on failover.
bq. MutableScheduler#updateConfiguration can be removed. Instead we need an API
to get confProvider.
Done, added aclPolicy/logging/confirming API to MutableConfigurationProvider
Also since the pendingMutation/YarnConfigurationStore APIs changed, it's
possible some of the LeveldbConfigurationStore logic broke, but I will address
this in YARN-7046.
> Implement zookeeper based store for scheduler configuration updates
> -------------------------------------------------------------------
>
> Key: YARN-6840
> URL: https://issues.apache.org/jira/browse/YARN-6840
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Wangda Tan
> Assignee: Jonathan Hung
> Attachments: YARN-6840-YARN-5734.001.patch,
> YARN-6840-YARN-5734.002.patch, YARN-6840-YARN-5734.003.patch,
> YARN-6840-YARN-5734.004.patch, YARN-6840-YARN-5734.005.patch,
> YARN-6840-YARN-5734.006.patch
>
>
> Right now there is only in-memory and leveldb based configuration store
> supported. Need one which supports RM HA.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
