[
https://issues.apache.org/jira/browse/YARN-8292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16477941#comment-16477941
]
Jason Lowe commented on YARN-8292:
----------------------------------
I'm still confused about the Resources.isAnyMajorResourceZero(rc,
toObtainAfterPreemption) clause in the doPreempt conditional. If we add a
rarely-requested resource dimension, it is likely to be often zero in a queue's
usage and therefore zero in toObtainAfterPremption.
Resources.isAnyMajorResourceZero(rc, toObtainAfterPreemption) will then be
always true, and that seems irrelevant to whether we want to keep preempting or
not.
If I understand the proposal correctly, I think the check for a zero resource
can be dropped and it simplifies to the toObtainAfterPreemption component-wise
max'd with zero is less than the amount to obtain from the partition (after
being max'd with zero). In other words, we want to preempt as long as we have
some resources we want to obtain from the partition and preempting the
container makes progress on at least one of the resource dimensions being
requested from the partition.
> Fix the dominant resource preemption cannot happen when some of the resource
> vector becomes negative
> ----------------------------------------------------------------------------------------------------
>
> Key: YARN-8292
> URL: https://issues.apache.org/jira/browse/YARN-8292
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
> Reporter: Sumana Sathish
> Assignee: Wangda Tan
> Priority: Critical
> Fix For: 3.2.0, 3.1.1
>
> Attachments: YARN-8292.001.patch
>
>
>
> This is an example of the problem: (Same if we have more than 2 resources)
>
> Let's say we have 3 queues A/B/C. All containers with equal size <2,3>
>
> ||Queue||Guaranteed||Used ||Pending||
> |A|<20, 10>|<20,30>| |
> |B|<20, 10>|0|0|
> |C|<20, 10>|0|<20, 30>|
> | | | | |
>
> Under current logic, A's calculated to-preempt (how much resource other queue
> can preempt) will be <0, 20>. The preemption will not happen. However, under
> the context of DRC, queue A is using more resource than guaranteed, so queue
> C will be starved
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
