[
https://issues.apache.org/jira/browse/YARN-7019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16128415#comment-16128415
]
Sunil G commented on YARN-7019:
-------------------------------
In ideal cases, lowest priority containers are selected for preemption from an
application which in turn is selected based on (fifo + priority / fair ordering
policy). These selected containers of an application may just about to finish
or have some critical data etc. So given a container selection policy, it will
be easier to manage these container selections as well in long run.
As suggested here, along with container priority, *score* which is derived from
reused/checkpointed could give a better candidate selection. In YARN-3784, we
were trying to send preemption timeout to AMs, so could AM take an immediate
action about checkpointing OR even send a feedback to RM with alternative
container to preempt instead of selected one?
> Ability for applications to notify YARN about container reuse
> -------------------------------------------------------------
>
> Key: YARN-7019
> URL: https://issues.apache.org/jira/browse/YARN-7019
> Project: Hadoop YARN
> Issue Type: New Feature
> Reporter: Jason Lowe
>
> During preemption calculations YARN can try to reduce the amount of work lost
> by considering how long a container has been running. However when an
> application framework like Tez reuses a container across multiple tasks it
> changes the work lost calculation since the container has essentially
> checkpointed between task assignments. It would be nice if applications
> could inform YARN when a container has been reused/checkpointed and therefore
> is a better candidate for preemption wrt. lost work than other, younger
> containers.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
