[
https://issues.apache.org/jira/browse/YARN-4390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15025079#comment-15025079
]
Sunil G commented on YARN-4390:
-------------------------------
Hi [~eepayne]
Thank you for raising this one. We also ran in to many of this use cases while
testing preemption. And such cases are annoying.
Adding to the use case, these 8 selected container can possibly run in multiple
nodes too. This will result reservation for the requesting app and further set
of preemption in next round.
As I see, YARN-4108 is trying to do a lazy preemption approach. If I am not
wrong, scheduler will be able to detect whether a certain set of preemption to
satisfy one huge request will be acceptable or not. If not, another set of
preemption unit to be considered. preemption unit means collection of
potentially to-be-preempted containers.
May be I feel, YARN-4108 is a common solution for all such cases. Could you
also please check that.
cc/[~leftnoteasy] for clarifying.
> Consider container request size during CS preemption
> ----------------------------------------------------
>
> Key: YARN-4390
> URL: https://issues.apache.org/jira/browse/YARN-4390
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
> Affects Versions: 3.0.0, 2.8.0, 2.7.3
> Reporter: Eric Payne
> Assignee: Eric Payne
>
> There are multiple reasons why preemption could unnecessarily preempt
> containers. One is that an app could be requesting a large container (say
> 8-GB), and the preemption monitor could conceivably preempt multiple
> containers (say 8, 1-GB containers) in order to fill the large container
> request. These smaller containers would then be rejected by the requesting AM
> and potentially given right back to the preempted app.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
