[
https://issues.apache.org/jira/browse/KNOX-3553?focusedWorklogId=1052959&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-1052959
]
ASF GitHub Bot logged work on KNOX-3553:
----------------------------------------
Author: ASF GitHub Bot
Created on: 06/Nov/26 09:05
Start Date: 06/Nov/26 09:05
Worklog Time Spent: 10m
Work Description: smolnar82 opened a new pull request, #1493:
URL: https://github.com/apache/knox/pull/1493
[KNOX-3553](https://issues.apache.org/jira/browse/KNOX-3553) - Set
ROLES_LOOKUP_EXECUTED only when the LDAP roles lookup interceptor processed the
user entry
## What changes were proposed in this pull request?
When `HadoopGroupProvider` used the Knox LDAP service
(`use.ldap.service=true`), it set the `ROLES_LOOKUP_EXECUTED` request attribute
whenever the `rolesLookup` interceptor was configured. However,
`LDAPRolesLookupInterceptor` only looks up roles for each user entry the search
returns. For a principal that isn't in LDAP (e.g. a Kubernetes service account
JWT), the search found 0 entries, so no roles lookup ran. The attribute still
told `KNOX-AUTH-SERVICE` that the lookup was done, so `AbstractAuthResource`
skipped its own lookup and the principal's configured roles never reached the
groups header.
Changes:
- New `UserGroupsLookupResult(groups, rolesLookupApplied)` record.
`KnoxLDAPServerManager` / `KnoxLDAPService#lookupUserGroups` return it.
`rolesLookupApplied` is true only if the user entry was found **and** the roles
lookup interceptor is configured. `getUserGroups` is kept and delegates to the
new method, so existing callers (`KnoxCLI`, `JdbcDelegationPolicyService`) are
unchanged.
- `HadoopGroupProviderFilter` sets `ROLES_LOOKUP_EXECUTED` only when
`rolesLookupApplied` is true. Otherwise it leaves the lookup to downstream
consumers such as `KNOX-AUTH-SERVICE`, which already handles the empty-groups
case.
- More DEBUG logging around where the roles lookup is (or isn't) called:
- `HadoopGroupProviderFilter`: whether the Knox LDAP service applied the
roles lookup for the principal.
- `AbstractAuthResource`: the roles lookup service is not enabled, the
lookup was already executed and is skipped, and looking up the roles / roles
found.
## How was this patch tested?
Unit tests:
- `KnoxLDAPServerManagerTest`: new tests for interceptor on + user not found
(`rolesLookupApplied=false`, the bug scenario), interceptor on + user found
(`true`), and interceptor off + user found (`false`).
- `HadoopGroupProviderFilterTest`: switched to `lookupUserGroups`, made the
request mock strict, and added a case where the user is not found while the
interceptor is configured. It verifies that `ROLES_LOOKUP_EXECUTED` is not set.
- `PreAuthResourceTest` and `ExtAuthzResourceTest` pass.
Manual test on a local Knox install:
- Setup:
- the embedded LDAP with the `rolesLookup` interceptor and the REST
strategy, with the lookup, list and get endpoints pointing to the mock roles
API from `.github/workflows/compose/role-lookup-rest/mock_roles_api.py`,
running in Docker;
- a topology with `HeaderPreAuth`, `HadoopGroupProvider`
(`use.ldap.service=true`) and `KNOX-AUTH-SERVICE`;
- roles configured for `guest` (in LDAP) and `k8s-sa-test` (not in LDAP).
- Before the fix:
```
$ curl -sik -H 'SM_USER: k8s-sa-test'
https://localhost:8443/gateway/preauth-roles/auth/api/v1/pre | grep -i x-awc
x-awc-username: k8s-sa-test
$ curl -sik -H 'SM_USER: guest'
https://localhost:8443/gateway/preauth-roles/auth/api/v1/pre | grep -i x-awc
x-awc-username: guest
x-awc-roles: platform:guest-role
```
- After the fix: the mock API is called only once (no double lookup). The
new DEBUG logs show which component ran the lookup in each case:
```
$ curl -sik -H 'SM_USER: guest'
https://localhost:8443/gateway/preauth-roles/auth/api/v1/pre | grep -i x-awc
x-awc-username: guest
x-awc-roles: platform:guest-role
$ curl -sik -H 'SM_USER: k8s-sa-test'
https://localhost:8443/gateway/preauth-roles/auth/api/v1/pre | grep -i x-awc
x-awc-username: k8s-sa-test
x-awc-roles: platform:sa-role
2026-11-06 09:55:38,365 8ccc6ac0-9f1b-4a98-980c-2086d957a31b DEBUG
knox.gateway (GatewayFilter.java:doFilter(130)) - Received request: GET
/auth/api/v1/pre
2026-11-06 09:55:38,375 8ccc6ac0-9f1b-4a98-980c-2086d957a31b INFO
knox.gateway (HadoopGroupProviderFilter.java:hadoopGroups(136)) - Using Knox
LDAP service to fetch groups...
2026-11-06 09:55:38,378 8ccc6ac0-9f1b-4a98-980c-2086d957a31b DEBUG
services.ldap (UserSearchInterceptor.java:search(112)) - LDAP Search:
dc=hadoop,dc=apache,dc=org | (uid=guest) | [*]
2026-11-06 09:55:38,401 8ccc6ac0-9f1b-4a98-980c-2086d957a31b DEBUG
services.ldap (LdapProxyBackend.java:performPagedSearch(1055)) - LDAP Paged
Search: dc=hadoop,dc=apache,dc=org | (uid=guest), page size 1,000, page 1
2026-11-06 09:55:38,403 8ccc6ac0-9f1b-4a98-980c-2086d957a31b DEBUG
services.ldap (LdapProxyBackend.java:performPagedSearch(1095)) - LDAP Paged
Search Completed: dc=hadoop,dc=apache,dc=org | (uid=guest). Found 1 entries
2026-11-06 09:55:38,404 8ccc6ac0-9f1b-4a98-980c-2086d957a31b DEBUG
services.ldap (LdapProxyBackend.java:performPagedSearch(1055)) - LDAP Paged
Search: ou=groups,dc=hadoop,dc=apache,dc=org |
(|(member=uid=guest,ou=people,dc=hadoop,dc=apache,dc=org)(uniqueMember=uid=guest,ou=people,dc=hadoop,dc=apache,dc=org)),
page size 1,000, page 1
2026-11-06 09:55:38,405 8ccc6ac0-9f1b-4a98-980c-2086d957a31b DEBUG
services.ldap (LdapProxyBackend.java:performPagedSearch(1095)) - LDAP Paged
Search Completed: ou=groups,dc=hadoop,dc=apache,dc=org |
(|(member=uid=guest,ou=people,dc=hadoop,dc=apache,dc=org)(uniqueMember=uid=guest,ou=people,dc=hadoop,dc=apache,dc=org)).
Found 0 entries
2026-11-06 09:55:38,409 8ccc6ac0-9f1b-4a98-980c-2086d957a31b DEBUG
services.ldap (RestApiLdapRolesLookup.java:lookupRoles(59)) - Sending role
lookup request to http://localhost:8090/api/v1/roles/lookup:
{"user_id":"guest","groups":[]}
2026-11-06 09:55:38,424 8ccc6ac0-9f1b-4a98-980c-2086d957a31b DEBUG
services.ldap (RestApiLdapRolesLookup.java:lookupRoles(68)) - Received role
lookup response from http://localhost:8090/api/v1/roles/lookup: HTTP 200, body:
{"user_id": "guest", "roles": [{"scope": "platform", "name": "guest-role"}]}
2026-11-06 09:55:38,426 8ccc6ac0-9f1b-4a98-980c-2086d957a31b DEBUG
knox.gateway (HadoopGroupProviderFilter.java:hadoopGroups(141)) - The Knox LDAP
service looked up the roles of principal guest; marking the roles lookup as
executed for this request
2026-11-06 09:55:38,426 8ccc6ac0-9f1b-4a98-980c-2086d957a31b DEBUG
knox.gateway (HadoopGroupProviderFilter.java:mapGroupPrincipals(116)) - Found
groups for principal guest : [platform:guest-role]
2026-11-06 09:55:38,426 8ccc6ac0-9f1b-4a98-980c-2086d957a31b DEBUG
knox.gateway (VirtualGroupMapper.java:mapGroups(59)) - User guest (with
group(s) [platform:guest-role]) added to group(s) []
2026-11-06 09:55:38,603 8ccc6ac0-9f1b-4a98-980c-2086d957a31b DEBUG
service.auth (AbstractAuthResource.java:lookupRoles(338)) - The roles of user
guest were already looked up by the identity assertion provider; skipping the
roles lookup
2026-11-06 09:55:45,431 3da245e9-5b7b-4ddb-99d4-1ddaec81df16 DEBUG
knox.gateway (GatewayFilter.java:doFilter(130)) - Received request: GET
/auth/api/v1/pre
2026-11-06 09:55:45,432 3da245e9-5b7b-4ddb-99d4-1ddaec81df16 INFO
knox.gateway (HadoopGroupProviderFilter.java:hadoopGroups(136)) - Using Knox
LDAP service to fetch groups...
2026-11-06 09:55:45,432 3da245e9-5b7b-4ddb-99d4-1ddaec81df16 DEBUG
services.ldap (UserSearchInterceptor.java:search(112)) - LDAP Search:
dc=hadoop,dc=apache,dc=org | (uid=k8s-sa-test) | [*]
2026-11-06 09:55:45,434 3da245e9-5b7b-4ddb-99d4-1ddaec81df16 DEBUG
services.ldap (LdapProxyBackend.java:performPagedSearch(1055)) - LDAP Paged
Search: dc=hadoop,dc=apache,dc=org | (uid=k8s-sa-test), page size 1,000, page 1
2026-11-06 09:55:45,435 3da245e9-5b7b-4ddb-99d4-1ddaec81df16 DEBUG
services.ldap (LdapProxyBackend.java:performPagedSearch(1095)) - LDAP Paged
Search Completed: dc=hadoop,dc=apache,dc=org | (uid=k8s-sa-test). Found 0
entries
2026-11-06 09:55:45,435 3da245e9-5b7b-4ddb-99d4-1ddaec81df16 DEBUG
knox.gateway (HadoopGroupProviderFilter.java:hadoopGroups(146)) - The Knox LDAP
service did not look up the roles of principal k8s-sa-test (no roles lookup
interceptor is configured, or the principal was not found in LDAP); leaving the
roles lookup to downstream consumers
2026-11-06 09:55:45,435 3da245e9-5b7b-4ddb-99d4-1ddaec81df16 DEBUG
knox.gateway (HadoopGroupProviderFilter.java:mapGroupPrincipals(116)) - Found
groups for principal k8s-sa-test : []
2026-11-06 09:55:45,435 3da245e9-5b7b-4ddb-99d4-1ddaec81df16 DEBUG
knox.gateway (VirtualGroupMapper.java:mapGroups(59)) - User k8s-sa-test (with
group(s) []) added to group(s) []
2026-11-06 09:55:45,437 3da245e9-5b7b-4ddb-99d4-1ddaec81df16 DEBUG
service.auth (AbstractAuthResource.java:lookupRoles(340)) - Looking up the
roles of user k8s-sa-test with groups []
2026-11-06 09:55:45,438 3da245e9-5b7b-4ddb-99d4-1ddaec81df16 DEBUG
services.ldap (RestApiLdapRolesLookup.java:lookupRoles(59)) - Sending role
lookup request to http://localhost:8090/api/v1/roles/lookup:
{"user_id":"k8s-sa-test","groups":[]}
2026-11-06 09:55:45,440 3da245e9-5b7b-4ddb-99d4-1ddaec81df16 DEBUG
services.ldap (RestApiLdapRolesLookup.java:lookupRoles(68)) - Received role
lookup response from http://localhost:8090/api/v1/roles/lookup: HTTP 200, body:
{"user_id": "k8s-sa-test", "roles": [{"scope": "platform", "name": "sa-role"}]}
2026-11-06 09:55:45,441 3da245e9-5b7b-4ddb-99d4-1ddaec81df16 DEBUG
service.auth (AbstractAuthResource.java:lookupRoles(342)) - Found roles for
user k8s-sa-test: [platform:sa-role]
```
## Integration Tests
No integration test added. The unit tests above cover the fix, and it was
also reproduced and verified manually against the mock roles REST API.
Issue Time Tracking
-------------------
Worklog Id: (was: 1052959)
Remaining Estimate: 0h
Time Spent: 10m
> HadoopGroupProvider marks the roles lookup as executed even when the embedded
> LDAP search finds no user entry
> -------------------------------------------------------------------------------------------------------------
>
> Key: KNOX-3553
> URL: https://issues.apache.org/jira/browse/KNOX-3553
> Project: Apache Knox
> Issue Type: Bug
> Components: Authorization, LDAP, Server
> Affects Versions: 3.0.0
> Reporter: Sandor Molnar
> Assignee: Sandor Molnar
> Priority: Blocker
> Fix For: 3.1.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When {{HadoopGroupProvider}} uses the embedded LDAP service
> ({{use.ldap.service=true}}) and the {{LDAPRolesLookupInterceptor}} is
> configured, {{HadoopGroupProviderFilter#hadoopGroups}} always sets the
> {{ROLES_LOOKUP_EXECUTED}} request attribute:
> {code:java}
> if (request != null && ldapService.hasRolesLookupInterceptor()) {
> request.setAttribute(ROLES_LOOKUP_EXECUTED, Boolean.TRUE.toString());
> }
> {code}
> The interceptor only does a roles lookup for each user entry the search
> returns. A principal that isn't in LDAP (e.g. a Kubernetes service account
> authenticated by {{JWTProvider}}) gets 0 entries back, so no lookup runs. The
> flag still says it did, so {{AbstractAuthResource#lookupRoles}} skips its own
> lookup. The roles configured for that principal are then missing from
> {{X-Knox-Actor-Groups}}.
> *Steps to reproduce:*
> # Set up a topology with {{JWTProvider}} (federation),
> {{HadoopGroupProvider}} ({{LdapGroupsMapping}}, {{use.ldap.service=true}})
> and {{KNOX-AUTH-SERVICE}}, with the roles lookup interceptor enabled on the
> embedded LDAP.
> # Assign a role to a k8s service account in the roles service.
> # Call {{/auth/api/v1/pre}} with that service account's JWT.
> *Actual:* The LDAP log shows "Found 0 entries" and {{X-Knox-Actor-Groups}}
> has no role.
> *Expected:* {{X-Knox-Actor-Groups}} contains the configured role.
> *Proposed fix:* Set {{ROLES_LOOKUP_EXECUTED}} only when the user's entry was
> actually found and processed by the interceptor. Otherwise the auth service
> falls back to its own roles lookup.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
