[
https://issues.apache.org/jira/browse/YARN-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16524101#comment-16524101
]
Wangda Tan commented on YARN-1013:
----------------------------------
Just took a very quick look at YARN-1015. IIUC, scheduler allocates O
containers when node uses more than guaranteed resource.
In my mind, problem of this approach is it cannot guarantee that allocated
containers satisfy user's requirement. It doesn't check getExecutionTypeRequest
of user's ResourceRequest, and it doesn't consider each app's pending O
resource request, and queue's pending O resource request, etc. What if user
doesn't want O containers? Similarly, YARN-6794 randomly promotes O container
even if user doesn't care about container execution type.
The syntax of YARN-8178 is much simpler, application can avoid get O resource
request if the resource is not preemptable. I like a proposal from [~curino]
that we should add a flag to indicate resource request is Guaranteed and
non-preemptable. Once we have that, we can get G container even if queue is
preemptable.
Considering all CS features (user-limit, node partition, application priority,
queue priority) may interact with O containers, I'm not sure how much effort
required to cleanly support this in CS. Simply porting YARN-1015 to CS might be
oversimplified to me.
> CS should watch resource utilization of containers and allocate speculative
> containers if appropriate
> -----------------------------------------------------------------------------------------------------
>
> Key: YARN-1013
> URL: https://issues.apache.org/jira/browse/YARN-1013
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Arun C Murthy
> Assignee: Weiwei Yang
> Priority: Major
>
> CS should watch resource utilization of containers (provided by NM in
> heartbeat) and allocate speculative containers (at lower OS priority) if
> appropriate.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
