[
https://issues.apache.org/jira/browse/YARN-2923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14605238#comment-14605238
]
Dian Fu commented on YARN-2923:
-------------------------------
Hi [~Naganarasimha],
Sorry to come to this JIRA late. Thanks a lot for working on this. Most codes
LGTM. Just have the following rough thoughts:
1. Classes such as {{AbstractNodeLabelsProvider}} are located in directory
{{hadoop-yarn-server-nodemanager}}, I'm think if we can move it to
{{hadoop-yarn-server-common}}. This would ease the work of adding similar node
label provider mechanism at RM side in the future if possible. There may need
some minor code reactor work to do, such as it accesses NM side configuration
{{NM_NODE_LABELS_PROVIDER_FETCH_INTERVAL_MS}}.
2. For YARN-2923 and YARN-2729, both {{ScriptBasedNodeLabelsProvider}} and
{{ConfigurationNodeLabelsProvider}} have a mechanism to periodically execute a
script or read configuration to fetch labels. Is it possible to merge
{{ScriptBasedNodeLabelsProvider}} and {{ConfigurationNodeLabelsProvider}} into
one class? This class can provide the function to periodically fetch labels and
the actual work of how to fetch labels can be configured in a plugable way. The
benefits of this approach are: a) remove duplicate code in
{{ScriptBasedNodeLabelsProvider}} and {{ConfigurationNodeLabelsProvider}}; b)
ease third parts to add their own node label provider.
Not sure if the above thoughts are clear and make sense to you? I can help to
do some POC based on YARN-2923 and YARN-2729 if needed.
> Support configuration based NodeLabelsProvider Service in Distributed Node
> Label Configuration Setup
> -----------------------------------------------------------------------------------------------------
>
> Key: YARN-2923
> URL: https://issues.apache.org/jira/browse/YARN-2923
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager
> Reporter: Naganarasimha G R
> Assignee: Naganarasimha G R
> Fix For: 2.8.0
>
> Attachments: YARN-2923.20141204-1.patch, YARN-2923.20141210-1.patch,
> YARN-2923.20150328-1.patch, YARN-2923.20150404-1.patch,
> YARN-2923.20150517-1.patch
>
>
> As part of Distributed Node Labels configuration we need to support Node
> labels to be configured in Yarn-site.xml. And on modification of Node Labels
> configuration in yarn-site.xml, NM should be able to get modified Node labels
> from this NodeLabelsprovider service without NM restart
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
