[
https://issues.apache.org/jira/browse/YARN-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15677774#comment-15677774
]
Wei Chen commented on YARN-5886:
--------------------------------
hi,
This feature is very interesting. And also Microsoft also published a paper in
ATC to talk about this feature. Here is some of my concerns.
1. How the local NM CotnaienrScheduler coordinate with global scheduler. since
global scheduler will try to keep fair and grarantee share across the
applications(queue).
2. Nodemanger may not know(or estimate) the runtime for queued container.
Falsely estimation(mistake a long-running as a short-running) may cause serious
results.(inverse priority?)
> Dynamically prioritize execution of opportunistic containers (NM queue
> reordering)
> ----------------------------------------------------------------------------------
>
> Key: YARN-5886
> URL: https://issues.apache.org/jira/browse/YARN-5886
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Konstantinos Karanasos
> Assignee: Konstantinos Karanasos
>
> Currently the {{ContainerScheduler}} in the NM picks the next queued
> opportunistic container to be executed in a FIFO manner. That is, we first
> execute containers that arrived first at the NM.
> This JIRA proposes to add pluggable queue reordering strategies at the NM
> that will dynamically determine which opportunistic container will be
> executed next.
> For example, we can choose to prioritize containers that belong to jobs which
> are closer to completion, or containers that are short-running (if such
> information is available).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
