[
https://issues.apache.org/jira/browse/YARN-1051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898677#comment-13898677
]
Carlo Curino commented on YARN-1051:
------------------------------------
Correct... You got the flow right.
A couple more notes for clarity:
For flexibility we are decoupling the creation of a reservation from what
applications are ran into it. I could imagine to have something like Oozie
to submit a reservation request (expressing the skyline of resource it will
need for a pipeline of jobs), get back a session id (practically a queue name),
and then submit the various jobs to it. Dynamically the Plan follower will
ensure the queue exists,and has whatever capacity the admission control
dedicated to it in every moment in time.
We plan to handle sudden collapses in cluster capacity (rack gone bad), by
reconsidering the plan as a whole (for now simple greedy replanner, deciding
what sessions to kill/reposition). This is to be able to express reservations
in absolute terms (100 containers),
instead of relative (10% of capacity)... this is particularly important for
gang jobs like Giraph that cannot deal with partial allocations well.
> YARN Admission Control/Planner: enhancing the resource allocation model with
> time.
> ----------------------------------------------------------------------------------
>
> Key: YARN-1051
> URL: https://issues.apache.org/jira/browse/YARN-1051
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacityscheduler, resourcemanager, scheduler
> Reporter: Carlo Curino
> Assignee: Carlo Curino
> Attachments: YARN-1051-design.pdf, curino_MSR-TR-2013-108.pdf
>
>
> In this umbrella JIRA we propose to extend the YARN RM to handle time
> explicitly, allowing users to "reserve" capacity over time. This is an
> important step towards SLAs, long-running services, workflows, and helps for
> gang scheduling.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
