[
https://issues.apache.org/jira/browse/YARN-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243171#comment-14243171
]
Craig Welch commented on YARN-2495:
-----------------------------------
Sorry if I'm jumping in late and asking redundant questions as a result, but
I've gone through the various related jiras and the design documents (incl
updates) ( and the patch :-) ) and I have some requirements related questions
as a result. Just a bit of background to make sure I understand things - it
appears that we've settled on two different but related features here:
1. To enable node labels to be added or removed for a given node without
validation against a centralized list of node labels (not on this patch, but
relevant to the discussion) and
2. To enable node managers to specify their node labels based on local
configuration and scripting (this patch is specific to that feature).
These are, strictly speaking, orthogonal, but may be used together and will
provide something more in a 'combined feature'
A couple things about this feature (2) - I don't believe that it is necessary
to add the node label configuration to the local configuration (yarn-site) or
the heartbeat as such to enable configuration of labels for a node from the
node in a decentralized fashion (e.g. a script on the node saying "put these
labels on me"). This can already be accomplished using the admin cli from a
script or calling a web service from the node (most likely the former, but
either is possible...), so I don't think we need this change to support the
script case, it's already possible to write a script to add a label to a node
on the fly today without any changes. To make this dynamic we would need
feature 1, from the above, but that's not covered in this patch / is a separate
discussion / and also does not require this change. Also, I don't see how this
change allows a script to dynamically configure labels unless it was changing
the yarn-site or the like (I may have missed it, but I don't see that logic
here) - and in any case, it would not be necessary to add this logic to support
that sort of configuration as I pointed out. Is this all just to support
putting labels into the node-managers configuration file and introducing them
that way? Do we have a solid need for that? It's not needed for the dynamic
script case, which is all I've seen discussed here from a "requirements"
perspective (putting it into the config file / adding it to the heartbeat is
implementation, I don't see a requirement for it as such).
In a nutshell - do we need change 2 (this), or do we really just need change 1
(eliminating validation of labels against a centralized list, at least as a
configurable option)?
> Allow admin specify labels from each NM (Distributed configuration)
> -------------------------------------------------------------------
>
> Key: YARN-2495
> URL: https://issues.apache.org/jira/browse/YARN-2495
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
> Reporter: Wangda Tan
> Assignee: Naganarasimha G R
> Attachments: YARN-2495.20141023-1.patch, YARN-2495.20141024-1.patch,
> YARN-2495.20141030-1.patch, YARN-2495.20141031-1.patch,
> YARN-2495.20141119-1.patch, YARN-2495.20141126-1.patch,
> YARN-2495.20141204-1.patch, YARN-2495.20141208-1.patch,
> YARN-2495_20141022.1.patch
>
>
> Target of this JIRA is to allow admin specify labels in each NM, this covers
> - User can set labels in each NM (by setting yarn-site.xml (YARN-2923) or
> using script suggested by [~aw] (YARN-2729) )
> - NM will send labels to RM via ResourceTracker API
> - RM will set labels in NodeLabelManager when NM register/update labels
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
