[ https://issues.apache.org/jira/browse/YARN-5892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15881614#comment-15881614 ]
Wangda Tan commented on YARN-5892: ---------------------------------- Hi [~eepayne], Apologize for my late responses, finally get some time to look at this. I'm a little concerned about semantics of this feature. (I've to admit that existing user-limit semantics is not perfect first) For existing MULP, to me it has two guarantees: #1, if there're N (N <= 100 / MULP) users are consuming resource in a queue, each of them can get at least MULP / 100 * queue-configured-capacity. #2, when doing preemption, if an user uses <= MULP / 100 * queue-configured-capacity, we will not preempt resource from such users. If we introduced this feature, per my understanding, #1 will be break. For example, a queue with MULP = 33, and user=admin has MULP = 75. Assume we have 3 users: Jack, Alice, Admin are running jobs in the queue, there's no guarantee that any user can use at much as their MULP. To make a more clear semantics, in my mind there're some alternative solutions: a. Create queue just for such vip users, and we can plan resources, (for example root.users.admin = 30%, and root.users.others = 50%). b. In addition to that, special queue mapping rules can be implemented based on YARN-3635, which can do more flexible queue mappings for use cases like vip users. Please let me know you thoughts. > Capacity Scheduler: Support user-specific minimum user limit percent > -------------------------------------------------------------------- > > Key: YARN-5892 > URL: https://issues.apache.org/jira/browse/YARN-5892 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacityscheduler > Reporter: Eric Payne > Assignee: Eric Payne > Attachments: YARN-5892.001.patch, YARN-5892.002.patch > > > Currently, in the capacity scheduler, the {{minimum-user-limit-percent}} > property is per queue. A cluster admin should be able to set the minimum user > limit percent on a per-user basis within the queue. > This functionality is needed so that when intra-queue preemption is enabled > (YARN-4945 / YARN-2113), some users can be deemed as more important than > other users, and resources from VIP users won't be as likely to be preempted. > For example, if the {{getstuffdone}} queue has a MULP of 25 percent, but user > {{jane}} is a power user of queue {{getstuffdone}} and needs to be guaranteed > 75 percent, the properties for {{getstuffdone}} and {{jane}} would look like > this: > {code} > <property> > > <name>yarn.scheduler.capacity.root.getstuffdone.minimum-user-limit-percent</name> > <value>25</value> > </property> > <property> > > <name>yarn.scheduler.capacity.root.getstuffdone.jane.minimum-user-limit-percent</name> > <value>75</value> > </property> > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org