[
https://issues.apache.org/jira/browse/YARN-1042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13763848#comment-13763848
]
Junping Du commented on YARN-1042:
----------------------------------
Thanks for comments. Luke!
bq. Although you can do a lot at the app side with container filtering,
protocol and scheduler support will make it more efficient. I guess the
intention of the jira is more for the latter, that affinity support should be
app independent.
Oh. It remind me that intention of JIRA may be on RM side (so the title may be
replaced from container requests to resource request) as long lived services
may have different AppMaster from default one that I change here. Also, I agree
that do it in RM side may be more efficient as no need to return containers in
app side for against affinity rules.
However, my concern is it may take extra complexity to RM as it make RM aware
the affinity/anti-affinity group of tasks (or resource request). IMO, one
simplicity and beauty for YARN is: RM only take care abstracted resource
request, and do container allocation accordingly. I am not sure if putting
resource request into affinity/anti-affinity groups and tracking resource
request relationship hurt this beauty. Thoughts?
> add ability to specify affinity/anti-affinity in container requests
> -------------------------------------------------------------------
>
> Key: YARN-1042
> URL: https://issues.apache.org/jira/browse/YARN-1042
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: resourcemanager
> Affects Versions: 3.0.0
> Reporter: Steve Loughran
> Assignee: Junping Du
> Attachments: YARN-1042-demo.patch
>
>
> container requests to the AM should be able to request anti-affinity to
> ensure that things like Region Servers don't come up on the same failure
> zones.
> Similarly, you may be able to want to specify affinity to same host or rack
> without specifying which specific host/rack. Example: bringing up a small
> giraph cluster in a large YARN cluster would benefit from having the
> processes in the same rack purely for bandwidth reasons.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
