[ https://issues.apache.org/jira/browse/YARN-10652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17295964#comment-17295964 ]
Peter Bacsko commented on YARN-10652: ------------------------------------- Hi guys, I think we can reach compromise: let's think about scenarios where dotted usernames can be problematic and address them in a follow-up JIRA. For example, we already know that placement rules involving username (%user placeholder) will definitely exhibit unexpected behavior (interestingly enough this has always been a problem, but just hasn't been reported). So in this case, we can go FS-way and just replace "." with "_dot_". Also, FS does this to primary groups as well, that's another thing that we need to fix. Maybe the cleanName() approach is just fine? When it comes to configuration, {{getValByRegex()}} is only used for this property, so it's likely that we're already good and in other cases, property names are concatenated and dot isn't an issue at all. In YARN-9930, I added "yarn.scheduler.capacity.user.<userName>.max-parallel-apps", making it a potential suspect, but I don't use regex, just concat strings. IMO we can handle these on a case-by-case basis. > Capacity Scheduler fails to handle user weights for a user that has a "." > (dot) in it > ------------------------------------------------------------------------------------- > > Key: YARN-10652 > URL: https://issues.apache.org/jira/browse/YARN-10652 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler > Affects Versions: 3.3.0 > Reporter: Siddharth Ahuja > Assignee: Siddharth Ahuja > Priority: Major > Attachments: Correct user weight of 0.76 picked up for the user with > a dot after the patch.png, Incorrect default user weight of 1.0 being picked > for the user with a dot before the patch.png, YARN-10652.001.patch > > > AD usernames can have a "." (dot) in them i.e. they can be of the format -> > {{firstname.lastname}}. However, if you specify a username with this format > against the Capacity Scheduler setting -> > {{yarn.scheduler.capacity.root.default.user-settings.firstname.lastname.weight}}, > it fails to be applied and is instead assigned the default of 1.0f weight. > This renders the user weight feature (being used as a means of setting user > priorities for a queue) unusable for such users. > This limitation comes from [1]. From [1], only word characters (A word > character: [a-zA-Z_0-9]) (see [2]) are permissible at the moment which is no > good for AD names that contain a "." (dot). > Similar discussion has been had in a few HADOOP jiras e.g. HADOOP-7050 and > HADOOP-15395 and the outcome was to use non-whitespace characters i.e. > instead of {{\w+}}, use {{\S+}}. > We could go down similar path and unblock this feature for the AD usernames > with a "." (dot) in them. > [1] > https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java#L1953 > [2] > https://docs.oracle.com/javase/tutorial/essential/regex/pre_char_classes.html -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org