[ https://issues.apache.org/jira/browse/YARN-10958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Szilard Nemeth updated YARN-10958: ---------------------------------- Fix Version/s: 3.4.0 > Use correct configuration for Group service init in CSMappingPlacementRule > -------------------------------------------------------------------------- > > Key: YARN-10958 > URL: https://issues.apache.org/jira/browse/YARN-10958 > Project: Hadoop YARN > Issue Type: Sub-task > Reporter: Peter Bacsko > Assignee: Szilard Nemeth > Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 3h > Remaining Estimate: 0h > > There is a potential problem in {{CSMappingPlacementRule.java}}: > {noformat} > if (groups == null) { > groups = Groups.getUserToGroupsMappingService(conf); > } > {noformat} > The problem is, we're supposed to pass {{scheduler.getConf()}}. The "conf" > object is the config for capacity scheduler, which does not include the > property which selects the group service provider. Therefore, the current > code just works by chance, because Group mapping service is already > initialized at this point. See the original fix in YARN-10053. > Also, need a unit test to verify it. > Idea: > # Create a Configuration object in which the property > "hadoop.security.group.mapping" refers to an existing a test implementation. > # Add a new method to {{Groups}} which nulls out the singleton instance, eg. > {{Groups.reset()}}. > # Create a mock CapacityScheduler where {{getConf()}} and > {{getConfiguration()}} contain different settings for > "hadoop.security.group.mapping". Since {{getConf()}} is the service config, > this should return the config object created in step #1. > # Create an instance of {{CSMappingPlacementRule}} with a single primary > group rule. > # Run the placement evaluation. > # Expected: returned queue matches what is supposed to be coming from the > test group mapping service ("testuser" --> "testqueue"). > # Modify "hadoop.security.group.mapping" in the config object created in > step #1. > # Call {{Groups.refresh()}} which changes the group mapping ("testuser" --> > "testqueue2"). This requires that the test group mapping service implement > {{GroupMappingServiceProvider.cacheGroupsRefresh()}}. > # Create a new instance of {{CSMappingPlacementRule}}. > # Run the placement evaluation again > # Expected: with the same user, the target queue has changed. > This looks convoluted, but these steps make sure that: > # {{CSMappingPlacementRule}} will force the initialization of groups. > # We select the correct configuration for group service init. > # We don't create a new {{Groups}} instance if the singleton is initialized, > so we cover the original problem described in YARN-10597. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org