[
https://issues.apache.org/jira/browse/MAPREDUCE-6529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14982611#comment-14982611
]
Jason Lowe commented on MAPREDUCE-6529:
---------------------------------------
I'm a little confused based on the wording of the summary and the description.
The summary implies there's a bug, but the description sounds like we're trying
to add a feature where the MapReduce AM will voluntarily give up an assigned
container because it's not "good enough." I'm assuming the latter, please
correct me if I'm wrong.
Could you elaborate more on the use-case for this? Seems to me in general it
is not a good idea for the AM to second-guess the RM scheduler when it comes to
container placement. The AM already conveyed to the RM where it wants
containers, and the RM granted some containers. If the AM doesn't like where
those containers were placed, how likely is it that giving them up will result
in a better placement in a reasonable timeframe? The RM scheduler (at least
the CapacityScheduler) already implements a delay to try to achieve node
locality (see YARN-80), so at least in that setup it would seem a detriment to
the job to give up a usable container now in hopes a better one comes along
soon. The RM has already waited around for a better one and decided to give a
suboptimal one instead. Unless it is very costly for the task to run off-node
or off-rack, it will be worse to give up the assigned container than to just
use it because it is unlikely a better container coming along soon.
> AppMaster will not retry to request resource if AppMaster happens to decide
> to not use the resource
> ---------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-6529
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6529
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mr-am
> Affects Versions: 2.6.0
> Reporter: Wei Chen
>
> I am viewing code in RMContainerAllocator.java. I want to do some
> improvement so that the AppMaster could give up some containers that may not
> be optimal when it receives new assigned containers. But I found that if
> AppMaster give up the containers, it will not retry to request the resource
> again.
> int RMContainerRequestor.java, Set<ResourceRequest> ask is used to ask
> resource from ResourceManager. I found each container could only be requested
> once. It mean ask can be filled by addResourceRequestToAsk(ResourceRequest
> remoteRequest[]), but it can only added for once for each container. If we
> give up one assigned container, It will never request again
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
