[
https://issues.apache.org/jira/browse/YARN-3318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14392211#comment-14392211
]
Craig Welch commented on YARN-3318:
-----------------------------------
[~leftnoteasy]
SchedulerProcessEvents replaced with containerAllocated and containerReleased
Serial and SerialEpoch replaced with compareInputOrderTo(), which is the option
2 for addressing it which we settled on offline
Added addSchedulerProcess/removeSchedulerProcess/addAllSchedulerProcesses
Changed configuration so that
yarn.scheduler.capacity.root.default.ordering-policy=fair
will setup the fair configuration, "fifo" will setup fifo, "fair+fifo" will
setup compound fair + fifo, etc. It is possible to setup a custom ordering
policy class using a different configuration, but the base one will handle the
"friendly" setup.
[~vinodkv]
bq. It is not entirely clear how the ordering and limits work together - as one
policy with multiple facets or multiple policy types
This should be modeled as different types of policies, so that they can each
focus on their particular purpose and avoid a repetition of the intermingling
which has made it difficult to mix, match, and share capabilities. Having
multiple policy types is essential to make it easy to combine them as needed.
bq. let's split the patch that exposes this to the client side / web UI and in
the API records into its own JIRA...premature to support this as a publicly
supportable configuration...
The goal is to make this available quickly but iteratively, keeping the changes
small but making them available for use and feedback. Clearly we can mark
things "unstable", communicate that they are not fully mature/subject to
change/should be used gently, but we will need to make it possible to activate
the feature and use it in order to accomplish the use and feedback. We should
grow it organically, gradually, iteratively, think of it is a facet of the
policy framework hooked up and available but with more to follow
bq. ...SchedulableEntity better... well, I'd actually talked [~leftnoteasy]
into SchedulerProcess :-) So, we can chew on this a bit more & see where we go
bq. You add/remove applications to/from LeafQueue's policy but addition/removal
of containers is an event...
This has been factored differently along [~leftnoteasy]'s suggestion, it should
now be consistent
bq. The notion of a comparator doesn't make sense to an admin. It is simply a
policy...
Have modeled "policy configuration" differently, so "comparator" is out of
sight (see above).
bq. Depending on how ordering and limits come together, they may become
properties of a policy
I expect them to be distinct, this is specifically an "ordering-policy", limits
will be other types of "limit-policy"(ies)
patch with these changes to follow in a few...
> Create Initial OrderingPolicy Framework, integrate with CapacityScheduler
> LeafQueue supporting present behavior
> ---------------------------------------------------------------------------------------------------------------
>
> Key: YARN-3318
> URL: https://issues.apache.org/jira/browse/YARN-3318
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: scheduler
> Reporter: Craig Welch
> Assignee: Craig Welch
> Attachments: YARN-3318.13.patch, YARN-3318.14.patch,
> YARN-3318.17.patch, YARN-3318.34.patch, YARN-3318.35.patch, YARN-3318.36.patch
>
>
> Create the initial framework required for using OrderingPolicies with
> SchedulerApplicaitonAttempts and integrate with the CapacityScheduler. This
> will include an implementation which is compatible with current FIFO behavior.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
