smolnar82 opened a new pull request, #1493: URL: https://github.com/apache/knox/pull/1493
[KNOX-3553](https://issues.apache.org/jira/browse/KNOX-3553) - Set ROLES_LOOKUP_EXECUTED only when the LDAP roles lookup interceptor processed the user entry ## What changes were proposed in this pull request? When `HadoopGroupProvider` used the Knox LDAP service (`use.ldap.service=true`), it set the `ROLES_LOOKUP_EXECUTED` request attribute whenever the `rolesLookup` interceptor was configured. However, `LDAPRolesLookupInterceptor` only looks up roles for each user entry the search returns. For a principal that isn't in LDAP (e.g. a Kubernetes service account JWT), the search found 0 entries, so no roles lookup ran. The attribute still told `KNOX-AUTH-SERVICE` that the lookup was done, so `AbstractAuthResource` skipped its own lookup and the principal's configured roles never reached the groups header. Changes: - New `UserGroupsLookupResult(groups, rolesLookupApplied)` record. `KnoxLDAPServerManager` / `KnoxLDAPService#lookupUserGroups` return it. `rolesLookupApplied` is true only if the user entry was found **and** the roles lookup interceptor is configured. `getUserGroups` is kept and delegates to the new method, so existing callers (`KnoxCLI`, `JdbcDelegationPolicyService`) are unchanged. - `HadoopGroupProviderFilter` sets `ROLES_LOOKUP_EXECUTED` only when `rolesLookupApplied` is true. Otherwise it leaves the lookup to downstream consumers such as `KNOX-AUTH-SERVICE`, which already handles the empty-groups case. - More DEBUG logging around where the roles lookup is (or isn't) called: - `HadoopGroupProviderFilter`: whether the Knox LDAP service applied the roles lookup for the principal. - `AbstractAuthResource`: the roles lookup service is not enabled, the lookup was already executed and is skipped, and looking up the roles / roles found. ## How was this patch tested? Unit tests: - `KnoxLDAPServerManagerTest`: new tests for interceptor on + user not found (`rolesLookupApplied=false`, the bug scenario), interceptor on + user found (`true`), and interceptor off + user found (`false`). - `HadoopGroupProviderFilterTest`: switched to `lookupUserGroups`, made the request mock strict, and added a case where the user is not found while the interceptor is configured. It verifies that `ROLES_LOOKUP_EXECUTED` is not set. - `PreAuthResourceTest` and `ExtAuthzResourceTest` pass. Manual test on a local Knox install: - Setup: - the embedded LDAP with the `rolesLookup` interceptor and the REST strategy, with the lookup, list and get endpoints pointing to the mock roles API from `.github/workflows/compose/role-lookup-rest/mock_roles_api.py`, running in Docker; - a topology with `HeaderPreAuth`, `HadoopGroupProvider` (`use.ldap.service=true`) and `KNOX-AUTH-SERVICE`; - roles configured for `guest` (in LDAP) and `k8s-sa-test` (not in LDAP). - Before the fix: ``` $ curl -sik -H 'SM_USER: k8s-sa-test' https://localhost:8443/gateway/preauth-roles/auth/api/v1/pre | grep -i x-awc x-awc-username: k8s-sa-test $ curl -sik -H 'SM_USER: guest' https://localhost:8443/gateway/preauth-roles/auth/api/v1/pre | grep -i x-awc x-awc-username: guest x-awc-roles: platform:guest-role ``` - After the fix: the mock API is called only once (no double lookup). The new DEBUG logs show which component ran the lookup in each case: ``` $ curl -sik -H 'SM_USER: guest' https://localhost:8443/gateway/preauth-roles/auth/api/v1/pre | grep -i x-awc x-awc-username: guest x-awc-roles: platform:guest-role $ curl -sik -H 'SM_USER: k8s-sa-test' https://localhost:8443/gateway/preauth-roles/auth/api/v1/pre | grep -i x-awc x-awc-username: k8s-sa-test x-awc-roles: platform:sa-role 2026-11-06 09:55:38,365 8ccc6ac0-9f1b-4a98-980c-2086d957a31b DEBUG knox.gateway (GatewayFilter.java:doFilter(130)) - Received request: GET /auth/api/v1/pre 2026-11-06 09:55:38,375 8ccc6ac0-9f1b-4a98-980c-2086d957a31b INFO knox.gateway (HadoopGroupProviderFilter.java:hadoopGroups(136)) - Using Knox LDAP service to fetch groups... 2026-11-06 09:55:38,378 8ccc6ac0-9f1b-4a98-980c-2086d957a31b DEBUG services.ldap (UserSearchInterceptor.java:search(112)) - LDAP Search: dc=hadoop,dc=apache,dc=org | (uid=guest) | [*] 2026-11-06 09:55:38,401 8ccc6ac0-9f1b-4a98-980c-2086d957a31b DEBUG services.ldap (LdapProxyBackend.java:performPagedSearch(1055)) - LDAP Paged Search: dc=hadoop,dc=apache,dc=org | (uid=guest), page size 1,000, page 1 2026-11-06 09:55:38,403 8ccc6ac0-9f1b-4a98-980c-2086d957a31b DEBUG services.ldap (LdapProxyBackend.java:performPagedSearch(1095)) - LDAP Paged Search Completed: dc=hadoop,dc=apache,dc=org | (uid=guest). Found 1 entries 2026-11-06 09:55:38,404 8ccc6ac0-9f1b-4a98-980c-2086d957a31b DEBUG services.ldap (LdapProxyBackend.java:performPagedSearch(1055)) - LDAP Paged Search: ou=groups,dc=hadoop,dc=apache,dc=org | (|(member=uid=guest,ou=people,dc=hadoop,dc=apache,dc=org)(uniqueMember=uid=guest,ou=people,dc=hadoop,dc=apache,dc=org)), page size 1,000, page 1 2026-11-06 09:55:38,405 8ccc6ac0-9f1b-4a98-980c-2086d957a31b DEBUG services.ldap (LdapProxyBackend.java:performPagedSearch(1095)) - LDAP Paged Search Completed: ou=groups,dc=hadoop,dc=apache,dc=org | (|(member=uid=guest,ou=people,dc=hadoop,dc=apache,dc=org)(uniqueMember=uid=guest,ou=people,dc=hadoop,dc=apache,dc=org)). Found 0 entries 2026-11-06 09:55:38,409 8ccc6ac0-9f1b-4a98-980c-2086d957a31b DEBUG services.ldap (RestApiLdapRolesLookup.java:lookupRoles(59)) - Sending role lookup request to http://localhost:8090/api/v1/roles/lookup: {"user_id":"guest","groups":[]} 2026-11-06 09:55:38,424 8ccc6ac0-9f1b-4a98-980c-2086d957a31b DEBUG services.ldap (RestApiLdapRolesLookup.java:lookupRoles(68)) - Received role lookup response from http://localhost:8090/api/v1/roles/lookup: HTTP 200, body: {"user_id": "guest", "roles": [{"scope": "platform", "name": "guest-role"}]} 2026-11-06 09:55:38,426 8ccc6ac0-9f1b-4a98-980c-2086d957a31b DEBUG knox.gateway (HadoopGroupProviderFilter.java:hadoopGroups(141)) - The Knox LDAP service looked up the roles of principal guest; marking the roles lookup as executed for this request 2026-11-06 09:55:38,426 8ccc6ac0-9f1b-4a98-980c-2086d957a31b DEBUG knox.gateway (HadoopGroupProviderFilter.java:mapGroupPrincipals(116)) - Found groups for principal guest : [platform:guest-role] 2026-11-06 09:55:38,426 8ccc6ac0-9f1b-4a98-980c-2086d957a31b DEBUG knox.gateway (VirtualGroupMapper.java:mapGroups(59)) - User guest (with group(s) [platform:guest-role]) added to group(s) [] 2026-11-06 09:55:38,603 8ccc6ac0-9f1b-4a98-980c-2086d957a31b DEBUG service.auth (AbstractAuthResource.java:lookupRoles(338)) - The roles of user guest were already looked up by the identity assertion provider; skipping the roles lookup 2026-11-06 09:55:45,431 3da245e9-5b7b-4ddb-99d4-1ddaec81df16 DEBUG knox.gateway (GatewayFilter.java:doFilter(130)) - Received request: GET /auth/api/v1/pre 2026-11-06 09:55:45,432 3da245e9-5b7b-4ddb-99d4-1ddaec81df16 INFO knox.gateway (HadoopGroupProviderFilter.java:hadoopGroups(136)) - Using Knox LDAP service to fetch groups... 2026-11-06 09:55:45,432 3da245e9-5b7b-4ddb-99d4-1ddaec81df16 DEBUG services.ldap (UserSearchInterceptor.java:search(112)) - LDAP Search: dc=hadoop,dc=apache,dc=org | (uid=k8s-sa-test) | [*] 2026-11-06 09:55:45,434 3da245e9-5b7b-4ddb-99d4-1ddaec81df16 DEBUG services.ldap (LdapProxyBackend.java:performPagedSearch(1055)) - LDAP Paged Search: dc=hadoop,dc=apache,dc=org | (uid=k8s-sa-test), page size 1,000, page 1 2026-11-06 09:55:45,435 3da245e9-5b7b-4ddb-99d4-1ddaec81df16 DEBUG services.ldap (LdapProxyBackend.java:performPagedSearch(1095)) - LDAP Paged Search Completed: dc=hadoop,dc=apache,dc=org | (uid=k8s-sa-test). Found 0 entries 2026-11-06 09:55:45,435 3da245e9-5b7b-4ddb-99d4-1ddaec81df16 DEBUG knox.gateway (HadoopGroupProviderFilter.java:hadoopGroups(146)) - The Knox LDAP service did not look up the roles of principal k8s-sa-test (no roles lookup interceptor is configured, or the principal was not found in LDAP); leaving the roles lookup to downstream consumers 2026-11-06 09:55:45,435 3da245e9-5b7b-4ddb-99d4-1ddaec81df16 DEBUG knox.gateway (HadoopGroupProviderFilter.java:mapGroupPrincipals(116)) - Found groups for principal k8s-sa-test : [] 2026-11-06 09:55:45,435 3da245e9-5b7b-4ddb-99d4-1ddaec81df16 DEBUG knox.gateway (VirtualGroupMapper.java:mapGroups(59)) - User k8s-sa-test (with group(s) []) added to group(s) [] 2026-11-06 09:55:45,437 3da245e9-5b7b-4ddb-99d4-1ddaec81df16 DEBUG service.auth (AbstractAuthResource.java:lookupRoles(340)) - Looking up the roles of user k8s-sa-test with groups [] 2026-11-06 09:55:45,438 3da245e9-5b7b-4ddb-99d4-1ddaec81df16 DEBUG services.ldap (RestApiLdapRolesLookup.java:lookupRoles(59)) - Sending role lookup request to http://localhost:8090/api/v1/roles/lookup: {"user_id":"k8s-sa-test","groups":[]} 2026-11-06 09:55:45,440 3da245e9-5b7b-4ddb-99d4-1ddaec81df16 DEBUG services.ldap (RestApiLdapRolesLookup.java:lookupRoles(68)) - Received role lookup response from http://localhost:8090/api/v1/roles/lookup: HTTP 200, body: {"user_id": "k8s-sa-test", "roles": [{"scope": "platform", "name": "sa-role"}]} 2026-11-06 09:55:45,441 3da245e9-5b7b-4ddb-99d4-1ddaec81df16 DEBUG service.auth (AbstractAuthResource.java:lookupRoles(342)) - Found roles for user k8s-sa-test: [platform:sa-role] ``` ## Integration Tests No integration test added. The unit tests above cover the fix, and it was also reproduced and verified manually against the mock roles REST API. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
