[
https://issues.apache.org/jira/browse/YARN-1008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13738175#comment-13738175
]
Alejandro Abdelnur commented on YARN-1008:
------------------------------------------
What the patch does:
Introduces a new configuration property in the RM,
{{RM_SCHEDULER_USE_PORT_FOR_NODE_NAME}} (defaulting to {{false}}).
This is an RM property but it is to be used by the scheduler implementations
when matching a resourcerequest to a node.
If the property is set to {{false}} things work as today, the matching is done
using only the HOSTNAME obtained from the NodeId.
If the property is set to {{true}}, the matching is done using the
HOSTNAME:PORT obtained from the NodeId.
There are not changes in the NM or AM side.
If the property is set to {{true}}, the AM must be aware of such setting and
when creating resource requests, it must set the location as HOSTNAME:PORT of
the node instead of just HOSTNAME.
The renaming of {{SchedulerNode#getHostName()}} to
{{SchedulerNode#getNodeName()}} is to make obvious to developers that may not
be HOSTNAME. Added javadocs explaing this clearly.
This works with all 3 schedulers.
The main motivation for this change is to be able to use yarn minicluster with
multiple NMs and being able to target a specific NM instance.
We could expose this for production use if there is a need. For that we would
need to:
* Expose via ApplicationReport the node matching mode: HOSTNAME or HOSTNAME:PORT
* Provide a mechanism for AMs only aware of HOSTNAME matching mode to work with
HOSTNAME:PORT mode
If there is a usecase for this in a real deployment we should follow up this
with another JIRA.
> MiniYARNCluster with multiple nodemanagers, all nodes have same key for
> allocations
> -----------------------------------------------------------------------------------
>
> Key: YARN-1008
> URL: https://issues.apache.org/jira/browse/YARN-1008
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Affects Versions: 2.1.0-beta
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Attachments: YARN-1008.patch, YARN-1008.patch, YARN-1008.patch
>
>
> While the NMs are keyed using the NodeId, the allocation is done based on the
> hostname.
> This makes the different nodes indistinguishable to the scheduler.
> There should be an option to enabled the host:port instead just port for
> allocations. The nodes reported to the AM should report the 'key' (host or
> host:port).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
