[
https://issues.apache.org/jira/browse/YARN-1679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13908767#comment-13908767
]
Xuan Gong commented on YARN-1679:
---------------------------------
If I understand correctly, Fair Scheduler can refresh its queues in two ways:
1. Admin call refreshQueues from AdminService
2. There is separate thread called AllocationFileLoaderService that will
check lastmodifiedTime for the configuration file periodically.
Both of them will call reloadAllocations() to refresh queues.
Now, we have FileSystemBasedConfigurationProvider that allowed users to upload
configuration files to hdfs, and can be shared by all rms. Noted that the
ConfigurationProvider can return InputStream for the configuration files. To
make it work for Fair Scheduler, and according to current design, we should
also at least have lastmodifiedTime information (unless we can find a way to
get lastmodifiedTime from inputStream).
Here is my proposal:
Create a more complex data structure named as ConfigurationContext that can
contain:
• String configurationName
• InputStream contentInputStream
• Long lastModifiedTime
We can also add more information when we need, such as permission, user,
groups, etc. And the configurationProvider can return ConfigurationContext
instead of InputStream.
For Fair Scheduler, when reloadAllocations() is called, we will use
ConfigurationProvider#getConfigurationContext() to get ConfigurationContext
which will contains all the information we need.
> Make admin refresh of Fair scheduler configuration work across RM failover
> --------------------------------------------------------------------------
>
> Key: YARN-1679
> URL: https://issues.apache.org/jira/browse/YARN-1679
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Xuan Gong
> Assignee: Xuan Gong
>
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
