[
https://issues.apache.org/jira/browse/YARN-3635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14629139#comment-14629139
]
Sandy Ryza commented on YARN-3635:
----------------------------------
[~leftnoteasy], apologies for this quick drive-by review - I am currently
traveling.
The JIRA appears to be lacking a design-doc and I wasn't able to find
documentation in the patch. The patch should ultimately include some detailed
documentation, but I don't want to ask this of you before OKing the approach.
In light of this, a few questions:
* What steps are required for the Fair Scheduler to integrate with this?
* Is a common way of configuration proposed?
* How does this differ from the current Fair Scheduler model? To summarize:
** The FS model consists of a sequence of placement rules that the app is
passed through.
** Each placement rule gets the chance to assign the app to a queue, reject the
app, or pass. If it passes, the next rule gets a chance.
** A placement rule can base its decision on:
*** The submitting user.
*** The set of groups the submitting user belongs to.
*** The queue requested in the app submission.
*** A set of configuration options that are specific to the rule.
*** The set of queues given in the Fair Scheduler configuration.
** Rules are marked as "terminal" if they will never pass. This helps to avoid
misconfigurations where users place rules after terminal rules.
** Rules have a "create" attribute which determines whether they can create a
new queue or whether they must assign to existing queues.
** Currently the set of placement rules is limited to what's implemented in
YARN. I.e. there's no public pluggable rule support.
I noticed from Vinod's comment that this patch follows a similar structure.
Are there places where my summary would not describe what's going on in this
patch?
> Get-queue-mapping should be a common interface of YarnScheduler
> ---------------------------------------------------------------
>
> Key: YARN-3635
> URL: https://issues.apache.org/jira/browse/YARN-3635
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: scheduler
> Reporter: Wangda Tan
> Assignee: Tan, Wangda
> Attachments: YARN-3635.1.patch, YARN-3635.2.patch, YARN-3635.3.patch,
> YARN-3635.4.patch, YARN-3635.5.patch, YARN-3635.6.patch
>
>
> Currently, both of fair/capacity scheduler support queue mapping, which makes
> scheduler can change queue of an application after submitted to scheduler.
> One issue of doing this in specific scheduler is: If the queue after mapping
> has different maximum_allocation/default-node-label-expression of the
> original queue, {{validateAndCreateResourceRequest}} in RMAppManager checks
> the wrong queue.
> I propose to make the queue mapping as a common interface of scheduler, and
> RMAppManager set the queue after mapping before doing validations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
