[
https://issues.apache.org/jira/browse/SLIDER-82?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14963114#comment-14963114
]
Steve Loughran commented on SLIDER-82:
--------------------------------------
It turns out that the AM doesn't get a node map via the node map update
callback except when the NM changes, and you don't get an initial map on AM
registration (which would be the obvious place).
What to do?
# 2.8+ get the RM/AM protocol to include the list of nodes on registration.
Best long term strategy for all. Slider on Hadoop <2.8 would have to handle the
missing list, perhaps by not handling anti-affinity there.
# get the node list from the YARN client on launch. Works for short-lived
services, and could be maintained for the life of the AM —but would be lost on
AM restart.
# bring up a YARN client in the RM. Not completely implausible, as the keytab
to do this will come up, and for keytabless operation, the slider client could
ask for a token to talk to the RM as the principal.
> Support ANTI_AFFINITY_REQUIRED option
> -------------------------------------
>
> Key: SLIDER-82
> URL: https://issues.apache.org/jira/browse/SLIDER-82
> Project: Slider
> Issue Type: Task
> Components: appmaster
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Fix For: Slider 2.0.0
>
> Attachments: SLIDER-82.002.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> slider has an anti-affinity flag in roles (visible in resources.json?), which
> is ignored.
> YARN-1042 promises this for YARN, slider will need
> # flag in resources.json
> # use in container requests
> we may also want two policies: anti-affinity-desired, and -required. Then if
> required nodes get >1 container for the same component type on the same node,
> it'd have to request a new one and return the old one (Risk: getting the same
> one back).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
