[
https://issues.apache.org/jira/browse/YARN-10652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17297089#comment-17297089
]
Wilfred Spiegelenburg commented on YARN-10652:
----------------------------------------------
I completely agree with your assessment [~pbacsko]. This is nowhere near a full
fix for the dot problem at all. That needs to be tackled one issue at a time.
We should not do all of it now. We can take multiple jiras to fix these issues.
I thus second the case by case solution approach. Fixing this one outside of
placement rule changes is one step. Introducing a standard way for the property
resolution for all properties that use the <userName> would be a *nice* to
have, again not needed now. The property introduced for max apps resolves
without an issue even with dots in the name.
Placement rules are complex, I would not recommend that this Jira should look
at it at all.
[~snemeth] & [~shuzirra]: based on the fact that we need to fix this
irrespective of what is done in placement rules I would like to proceed with
the commit for this. The change allows the administrator to just use the
existing user name in the configuration similar to the "max-parallel-apps"
setting. When and if a solution is implemented for the placement rules to
support dots in user and group names, which are part of the queue path, new
fixes might be needed for this issue and YARN-9930. We might even leave these
two as is. That is not a decision we need to make now.
> Capacity Scheduler fails to handle user weights for a user that has a "."
> (dot) in it
> -------------------------------------------------------------------------------------
>
> Key: YARN-10652
> URL: https://issues.apache.org/jira/browse/YARN-10652
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
> Affects Versions: 3.3.0
> Reporter: Siddharth Ahuja
> Assignee: Siddharth Ahuja
> Priority: Major
> Attachments: Correct user weight of 0.76 picked up for the user with
> a dot after the patch.png, Incorrect default user weight of 1.0 being picked
> for the user with a dot before the patch.png, YARN-10652.001.patch
>
>
> AD usernames can have a "." (dot) in them i.e. they can be of the format ->
> {{firstname.lastname}}. However, if you specify a username with this format
> against the Capacity Scheduler setting ->
> {{yarn.scheduler.capacity.root.default.user-settings.firstname.lastname.weight}},
> it fails to be applied and is instead assigned the default of 1.0f weight.
> This renders the user weight feature (being used as a means of setting user
> priorities for a queue) unusable for such users.
> This limitation comes from [1]. From [1], only word characters (A word
> character: [a-zA-Z_0-9]) (see [2]) are permissible at the moment which is no
> good for AD names that contain a "." (dot).
> Similar discussion has been had in a few HADOOP jiras e.g. HADOOP-7050 and
> HADOOP-15395 and the outcome was to use non-whitespace characters i.e.
> instead of {{\w+}}, use {{\S+}}.
> We could go down similar path and unblock this feature for the AD usernames
> with a "." (dot) in them.
> [1]
> https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java#L1953
> [2]
> https://docs.oracle.com/javase/tutorial/essential/regex/pre_char_classes.html
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
