[
https://issues.apache.org/jira/browse/YARN-1011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15119825#comment-15119825
]
Karthik Kambatla commented on YARN-1011:
----------------------------------------
[~nroberts] - interesting ideas. The notion of two schedulers definitely frees
us from the confines we have today.
How about doing the following on node update (along the lines of two scheduler
suggestion):
{code}
process_newly_launched_containers();
process_completed_containers();
while (guaranteed_resources_on_node_are_still_available) {
app = pickApp();
if (app.hasOpportunisticContainersRunningOnTheNode()) {
promote_container_to_guaranteed();
} else {
allocate_guaranteed_container_as_we_do_today(); // includes reservation
}
}
while (opportunistic_resources_on_node_are_still_available) {
app = pickAppOkayWithOpportunisticContainers();
allocate_opportunistic_container(); // reservations are not allowed
}
{code}
This way:
# Apps that can't tolerate using opportunistic containers can state their
preference to have only guaranteed containers.
# For apps that are okay with opportunistic containers, a container is promoted
only when the app deserves guaranteed containers.
> [Umbrella] Schedule containers based on utilization of currently allocated
> containers
> -------------------------------------------------------------------------------------
>
> Key: YARN-1011
> URL: https://issues.apache.org/jira/browse/YARN-1011
> Project: Hadoop YARN
> Issue Type: New Feature
> Reporter: Arun C Murthy
> Attachments: yarn-1011-design-v0.pdf, yarn-1011-design-v1.pdf,
> yarn-1011-design-v2.pdf
>
>
> Currently RM allocates containers and assumes resources allocated are
> utilized.
> RM can, and should, get to a point where it measures utilization of allocated
> containers and, if appropriate, allocate more (speculative?) containers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
