[
https://issues.apache.org/jira/browse/YARN-10652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17291583#comment-17291583
]
Siddharth Ahuja edited comment on YARN-10652 at 2/26/21, 11:24 AM:
-------------------------------------------------------------------
Thanks [~rreti] & [~pbacsko] for your comments. However, please correct me if I
am wrong (and I must admit that I am primarily familiar with FairScheduler as
well), however, the
arn.scheduler.capacity.<queue-path>.user-settings.<user-name>.weight setting
has nothing to do with queue placement. We are not trying to have the user ->
firstname.lastname placed into its own queue like root.firstname.lastname here.
What we are instead trying to solve is that if a user-> firstname.lastname
wants to submit a job to the root.default queue as an example, then, that user
can certainly do that today (kindly see the attached screenshots - there is
nothing stopping that today) and considering it can do that, the user weight
setting that decides the user's share in the root.default (not
root.firstname.lastname) queue should be able to cater for this user, that's
all. Giving firstname.lastname user it's share through this setting does not
trigger any placements and as such should not have any cause of concern. Again,
please correct me if I am wrong [~pbacsko].
was (Author: sahuja):
Thanks [~rreti] & [~pbacsko] for your comments. However, please correct me if I
am wrong (and I must admit that I am primarily familiar with FairScheduler as
well), however,
{{yarn.scheduler.capacity.<queue-path>.user-settings.<user-name>.weight}}
setting has nothing to do with queue placement. We are not trying to have the
user -> firstname.lastname placed into its own queue like
root.firstname.lastname here. What we are instead trying to solve is that if a
user-> firstname.lastname wants to submit a job to the root.default queue as an
example, then, that user can certainly do that today (kindly see the attached
screenshots - there is nothing stopping that today) and considering it can do
that, the user weight setting that decides the user's share in the root.default
(not root.firstname.lastname) queue should be able to cater for this user,
that's all. Giving firstname.lastname user it's share through this setting does
not trigger any placements and as such should not have any cause of concern.
Again, please correct me if I am wrong [~pbacsko].
> Capacity Scheduler fails to handle user weights for a user that has a "."
> (dot) in it
> -------------------------------------------------------------------------------------
>
> Key: YARN-10652
> URL: https://issues.apache.org/jira/browse/YARN-10652
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
> Affects Versions: 3.3.0
> Reporter: Siddharth Ahuja
> Assignee: Siddharth Ahuja
> Priority: Major
> Attachments: Correct user weight of 0.76 picked up for the user with
> a dot after the patch.png, Incorrect default user weight of 1.0 being picked
> for the user with a dot before the patch.png, YARN-10652.001.patch
>
>
> AD usernames can have a "." (dot) in them i.e. they can be of the format ->
> {{firstname.lastname}}. However, if you specify a username with this format
> against the Capacity Scheduler setting ->
> {{yarn.scheduler.capacity.root.default.user-settings.firstname.lastname.weight}},
> it fails to be applied and is instead assigned the default of 1.0f weight.
> This renders the user weight feature (being used as a means of setting user
> priorities for a queue) unusable for such users.
> This limitation comes from [1]. From [1], only word characters (A word
> character: [a-zA-Z_0-9]) (see [2]) are permissible at the moment which is no
> good for AD names that contain a "." (dot).
> Similar discussion has been had in a few HADOOP jiras e.g. HADOOP-7050 and
> HADOOP-15395 and the outcome was to use non-whitespace characters i.e.
> instead of {{\w+}}, use {{\S+}}.
> We could go down similar path and unblock this feature for the AD usernames
> with a "." (dot) in them.
> [1]
> https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java#L1953
> [2]
> https://docs.oracle.com/javase/tutorial/essential/regex/pre_char_classes.html
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
