[
https://issues.apache.org/jira/browse/YARN-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15380918#comment-15380918
]
Carlo Curino commented on YARN-5325:
------------------------------------
The first patch provides 2 implementations for the
{{ARMRMProxyFederationPolicy}}:
# a simple broadcast policy that replicate the user asks to all sub-clusters
# a sophisticated policy ({{LocalityMulticastAMRMProxyFederationPolicy}} that:
## Directs node are rack local asks to sub-clusters that own the node and the
rack. (in some deployments we stripe across racks, so the racks are not
resolved to one subcluster and are thus treated as an ANY)
## Directs ANY that correspond to node/rack local requests to the
corresponding sub-clusters by "splitting" the ANY according to how many
containers are requested (with locality) to each sub-clusters.
## Directs ANY that are not associated with a localized request to subclusters
according to a linear combination of policy weights and headroom information.
An {{alpha}} parameter is used to choose how much headroom plays a role here.
## Directs ANY requesting zero containers (user canceling their asks) to all
sub-clusters we interacted with so far.
## This policy minimizes the work done by RMs by ensuring that the sum of the
request we send out is close to the user ask (might exceed by at most
|sub-clusters| due to rounding to the closest integral number of containers).
The tests verify much of this behavior (though more should be done in testing).
Per discussions with [~subru] we should probably work on optimizing the policy
by factoring out sum-of-weight recomputations, etc...
> Stateless ARMRMProxy policies implementation
> --------------------------------------------
>
> Key: YARN-5325
> URL: https://issues.apache.org/jira/browse/YARN-5325
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
> Reporter: Carlo Curino
> Assignee: Carlo Curino
> Attachments: YARN-5325.01.patch
>
>
> This JIRA tracks policies in the AMRMProxy that decide how to forward
> ResourceRequests, without maintaining substantial state across decissions
> (e.g., broadcast).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
