[
https://issues.apache.org/jira/browse/YARN-8149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16436366#comment-16436366
]
Thomas Graves commented on YARN-8149:
-------------------------------------
thinking about this a little more, even with the current preemption on, I don't
think preemption is smart enough to keep starvation from happening. If
preemption was smart enough to kill enough containers on a reserved node to
make it so the big container actually gets scheduled there that might be ok.
But last time I checked it doesn't do that.
Without that or having another way to prevent starvation I wouldn't want to
remove this. I think adding a config would be alright but if anyone finds it
useful you can't remove and would just be an extra config.
If we have other ideas to simply or make this better, great we should look at.
Or if there is a way for us to get stats on if this is useful we could add
those and run and determine if we should remove.
> Revisit behavior of Re-Reservation in Capacity Scheduler
> --------------------------------------------------------
>
> Key: YARN-8149
> URL: https://issues.apache.org/jira/browse/YARN-8149
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Wangda Tan
> Priority: Critical
>
> Frankly speaking, I'm not sure why we need the re-reservation. The formula is
> not that easy to understand:
> Inside:
> {{org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.allocator.RegularContainerAllocator#shouldAllocOrReserveNewContainer}}
> {code:java}
> starvation = re-reservation / (#reserved-container *
> (1 - min(requested-resource / max-alloc,
> max-alloc - min-alloc / max-alloc))
> should_allocate = starvation + requiredContainers - reservedContainers >
> 0{code}
> I think we should be able to remove the starvation computation, just to check
> requiredContainers > reservedContainers should be enough.
> In a large cluster, we can easily overflow re-reservation to MAX_INT, see
> YARN-7636.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
