[
https://issues.apache.org/jira/browse/MAPREDUCE-5622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13821434#comment-13821434
]
Jason Lowe commented on MAPREDUCE-5622:
---------------------------------------
This stems from the fact that in general AMs can't always correlate container
assignments to container requests, and in this case the AM is mistakenly
mis-matching the assignments to the tasks that made the requests.
Ideally YARN should allow AMs to associate a piece of app-specific data to
container requests and have that returned in the corresponding container
assignment. That would make it trivial for AMs to associate requests with
assignments and therefore not screw up the scheduling on their end. However
that's a significant impact on the protocol and RM scheduler, as it currently
collates all requests for an application at a particular priority as a simple
count and therefore cannot distinguish which request corresponds to a granted
container.
> MRAppMaster doesn't assign all allocated NODE_LOCAL containers to node-local
> maps
> ---------------------------------------------------------------------------------
>
> Key: MAPREDUCE-5622
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5622
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster
> Affects Versions: 2.2.0
> Reporter: Devaraj K
>
> MRAppMaster will request containers for all the splits to launch map tasks,
> RM will give Node Local containers for all these if available. When the RM
> gives all containers as Node Local, MR AM may assign these NODE-LOCAL
> containers to non-local maps.
> \\
> \\
> |node1|split1|split2| |split4|
> |node2| |split2|split3| |
> |node3|split1|split2|split3|split4|
> |node4|split1| |split3|split4|
> \\
> Consider this instance, assume RM has given one NODE LOCAL container on each
> node to process all the splits as local maps. While assigning, if the AM
> gives node1-container for split1, node2-container for split3, node3-container
> for split3 and node4-container can be given to only split2 which is not
> local.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
