morningman commented on PR #68220:
URL: https://github.com/apache/doris/pull/68220#issuecomment-5747765958
## Context for reviewers: what this PR does and why
Nothing below is new relative to the description and the code comments; it
lays the pieces out in one place so the diff is easier to read.
### The problem
A Ranger policy item can name **users, groups and roles**. The requests the
`ranger-doris` and `ranger-hive` sources build carry **no groups** - Doris has
no groups of its own to hand over. So a policy item written against a group
never matches:
| item written against a group | what happens today |
|---|---|
| allow | the member gets nothing, down to `SHOW DATABASES` / `SWITCH` |
| row filter / column mask | not applied |
| **deny** | **silently ignored** - a user-level allow on the same table
still admits the user |
As operators see it: "group-based access does not work, and it starts
working the moment the same user is put on the policy item by name."
### Why not just a Ranger property
Ranger's answer for a plugin that cannot compute groups itself is the **user
store** - the users and groups Ranger Admin holds, kept current by usersync -
which a plugin downloads next to its policies. Ranger 2.5+ reads the requesting
user's groups out of it when `ranger.plugin.<type>.use.rangerGroups=true`
(`RangerDefaultRequestProcessor.updateUserGroups`). master builds against
plugin library 2.8.0, so that would work here; branch-3.0 / branch-2.1 are on
2.4.0, which has no such setting, and no Doris document names the property
either. This PR does the same thing in Doris code, so it works regardless of
the Ranger version, and it is **on by default**, keyed on the same property so
that one setting decides both.
### The change is three pieces
1. **`RangerUserStoreGroups`** (ranger-common, new, shared by both sources)
holds all the logic:
- `enabledFor(config)`: `<prefix>.use.rangerGroups`, default **true**
(Ranger reads the same key with default false; reversed on purpose).
- `addUserStoreEnricher(config, policies)`: calls Ranger's
`ServiceDefUtil.addUserStoreEnricher` to put a `userStoreEnricher` on the
downloaded service definition. Retriever class and refresh interval are read
under Ranger's own option names (`userStoreRetrieverClassName`,
`userStoreRefresherPollingInterval`, default 60 s) - the same thing
`RangerBasePlugin.setPolicies` does for `use.rangerGroups`.
- `groupsOf(plugin, user)`:
`plugin.getPluginContext().getAuthContext().getUserStoreUtil().getUserGroups(user)`,
null-safe at every step. Returns an **empty set, never null** when switched
off, when no store has arrived yet, or when the store does not know the user (a
Doris-only account): a request with an empty group set matches user and role
items exactly as before. Returns a copy, because the set belongs to the store
shared by every request until the next download.
- `describe(config)`: one start-up log line saying whether groups are
attached and how to switch it off.
2. **`RangerDorisPlugin` / `RangerHivePlugin` override `setPolicies`**: add
the enricher, then `super.setPolicies`. On *every* call, not only the first: a
delta download comes with its own copy of the service definition
(`RangerBasePlugin` re-adds the enricher on deltas for the same reason).
3. **`RangerDorisAccessController.createRequest` /
`RangerHiveAccessController.createRequest`**: one line each,
`request.setUserGroups(groupsOf(subject))`.
### Call chains
Background - getting the store, on every policy refresh:
```
RangerBasePlugin.init()
└─ PolicyRefresher thread → loadPolicy() → plugin.setPolicies(policies)
├─ [this PR] RangerDorisPlugin / RangerHivePlugin.setPolicies
│ └─ RangerUserStoreGroups.addUserStoreEnricher →
serviceDef.contextEnrichers += userStoreEnricher
└─ super.setPolicies → new policy engine →
RangerUserStoreEnricher.init()
├─ RangerAdminUserStoreRetriever downloads the store;
RangerUserStoreRefresher polls every 60 s
└─ setRangerUserStore →
authContext.addOrReplaceRequestContextEnricher → RangerAuthContext.userStoreUtil
```
Foreground - every access check, row filter and mask evaluation:
```
checkPrivilege / getRowFilters / getDataMasks
├─ createRequest(subject, context)
│ ├─ request.setUser(user)
│ └─ request.setUserGroups(groupsOf(subject)) ← the new line
│ └─ RangerUserStoreGroups.groupsOf(plugin, user)
│ └─
pluginContext.getAuthContext().getUserStoreUtil().getUserGroups(user)
└─ plugin.isAccessAllowed / evalRowFilterPolicies / evalDataMaskPolicies
└─ the policy engine matches items against user + groups + roles
```
### Roles are untouched
`ranger-doris` still sends no roles (deliberate; see the comment in
`createRequest` - sending them would start matching role items this source has
never matched, and belongs with a release note of its own). With none sent,
Ranger resolves its own roles from user **+ groups**
(`RangerDefaultRequestProcessor.preProcess`), so a Ranger role granted to a
group now resolves as well.
### Groups vs roles, briefly
Groups are an identity attribute the organization owns: they come from
LDAP/AD or the OS, usersync mirrors them into Ranger Admin, and Ranger's own
plugins learn a user's groups from the service's environment (Hadoop group
mapping) and put them on the request - Ranger only matches against what the
request carries, which is exactly what Doris cannot supply. Roles (Ranger 2.0+)
are Ranger's own authorization construct: defined in Ranger Admin, they can
contain users, groups and other roles, are downloaded with the policies, and
are resolved by the plugin from the request's user and groups. A role with
users as direct members already worked for Doris before this PR, and
functionally a role can stand in for a group; but existing services - Hive
above all - keep their policies by directory group, deny items included, and
role resolution itself starts from the request's groups. So "use roles instead"
does not avoid the problem: it is the same request that needs the groups.
### Tests
- `RangerUserStoreGroupsTest`: on by default; adds the same enricher
definition Ranger would; honours Ranger's retriever / interval options; adds it
once; leaves the definition alone when switched off; tolerates null policies.
- `RangerTest`: a request carries the groups of a user store handed to the
plugin's auth context; no groups before a store arrives; roles stay empty.
- `ranger_p2/test_ranger_group_policy`: an access policy, a row filter, a
mask and a deny written against a group only, checked as a member of the group
and then after leaving it.
- `httpTest` gains `op "put"`: `setRangerUserGroups` goes through the
user-update API rather than the group-user mapping API, because in Ranger Admin
2.4 only the former bumps the user store version - and the version is what
tells a plugin there is a new store to download.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
