[
https://issues.apache.org/jira/browse/YARN-2257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060982#comment-14060982
]
Sandy Ryza commented on YARN-2257:
----------------------------------
Wangda,
That policy would work well for a some situations, but I don't think it covers
many reasonable scenarios. For example, we might want to ignore the queue that
the user defines entirely. Or admins might want to be able to just send apps
to queues named with the user's group, instead of specifying the mapping for
every group. Or a subdivision in an organization might want to make placements
based on group, while a different subdivision using the same cluster might want
to make placements based on user.
Would you mind taking a look at the "Automatically placing applications in
queues" section and corresponding configuration example in
http://hadoop.apache.org/docs/r2.3.0/hadoop-yarn/hadoop-yarn-site/FairScheduler.html
My opinion is that this is a good fit for YARN in general.
> Add user to queue mappings to automatically place users' apps into specific
> queues
> ----------------------------------------------------------------------------------
>
> Key: YARN-2257
> URL: https://issues.apache.org/jira/browse/YARN-2257
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: scheduler
> Reporter: Patrick Liu
> Assignee: Vinod Kumar Vavilapalli
> Labels: features
>
> Currently, the fair-scheduler supports two modes, default queue or individual
> queue for each user.
> Apparently, the default queue is not a good option, because the resources
> cannot be managed for each user or group.
> However, individual queue for each user is not good enough. Especially when
> connecting yarn with hive. There will be increasing hive users in a corporate
> environment. If we create a queue for a user, the resource management will be
> hard to maintain.
> I think the problem can be solved like this:
> 1. Define user->queue mapping in Fair-Scheduler.xml. Inside each queue, use
> aclSubmitApps to control user's ability.
> 2. Each time a user submit an app to yarn, if the user has mapped to a queue,
> the app will be scheduled to that queue; otherwise, the app will be submitted
> to default queue.
> 3. If the user cannot pass aclSubmitApps limits, the app will not be accepted.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
