smolnar82 opened a new pull request, #1494: URL: https://github.com/apache/knox/pull/1494
[KNOX-3554](https://issues.apache.org/jira/browse/KNOX-3554) - HadoopGroupProvider: skip the group lookup for principals known not to exist in LDAP ## What changes were proposed in this pull request? Some authenticated principals are known in advance not to exist in LDAP (e.g. Kubernetes service accounts or other workload identities arriving as JWTs). `HadoopGroupProvider` still ran a group lookup for them. With `use.ldap.service=true`, that meant an embedded KnoxLDAP search plus the backend LDAP round trips, all of which always returned nothing. This PR adds an optional `HadoopGroupProvider` parameter, `group.lookup.skip.principal.pattern`: - It is a regex that must match the whole mapped principal name, i.e. the name after the principal and lowercase mapping. Principals matching it get no groups and no LDAP call. This applies to both the Knox LDAP service path and the Hadoop `LdapGroupsMapping` path. - `ROLES_LOOKUP_EXECUTED` is not set for skipped principals, so downstream consumers (e.g. `KNOX-AUTH-SERVICE`) still do their own roles lookup. - A blank value is ignored. An invalid regex makes the filter's `init()` fail with a `ServletException`. - It can be set in the topology or centrally in gateway-site via `CENTRAL_GROUP_CONFIG_PREFIX`, e.g. `gateway.group.config.group.lookup.skip.principal.pattern`. - Logging: an INFO line with the configured pattern when the filter is initialized, and a DEBUG line for every skipped principal. Example: ```xml <param> <name>group.lookup.skip.principal.pattern</name> <value>system:serviceaccount:.*</value> </param> ``` ## How was this patch tested? Unit tests (`HadoopGroupProviderFilterTest`): - A matching principal skips the lookup on the Knox LDAP service path (a strict `KnoxLDAPService` mock fails on any call) and on the Hadoop groups path. In both cases, no `ROLES_LOOKUP_EXECUTED` is set. - A pattern that only partially matches the principal does not skip the lookup. - A blank pattern is ignored. - An invalid regex fails `init()`. Manual test on a local Knox install (same as in #1493): - Setup: - the embedded LDAP with the `rolesLookup` interceptor and the REST strategy, pointing to the mock roles API from `.github/workflows/compose/role-lookup-rest/mock_roles_api.py`; - a topology with `HeaderPreAuth`, `HadoopGroupProvider` (`use.ldap.service=true`, `group.lookup.skip.principal.pattern=k8s-sa-.*`) and `KNOX-AUTH-SERVICE`. - `k8s-sa-test`: no KnoxLDAP or backend LDAP search was made. `KNOX-AUTH-SERVICE` did the roles lookup (`"groups":[]`) and returned `platform:sa-role`. - `guest` (no match): the LDAP lookup and the interceptor roles lookup ran as before, and `KNOX-AUTH-SERVICE` skipped its own lookup. ``` $ curl -sik -H 'SM_USER: k8s-sa-test' https://localhost:8443/gateway/preauth-roles/auth/api/v1/pre | grep -i x-awc x-awc-username: k8s-sa-test x-awc-roles: platform:sa-role $ curl -sik -H 'SM_USER: guest' https://localhost:8443/gateway/preauth-roles/auth/api/v1/pre | grep -i x-awc x-awc-username: guest x-awc-roles: platform:guest-role 2026-11-06 11:33:46,174 a05cc992-5b1b-4504-b374-29a3a090481d DEBUG knox.gateway (GatewayFilter.java:doFilter(130)) - Received request: GET /auth/api/v1/pre 2026-11-06 11:33:46,176 a05cc992-5b1b-4504-b374-29a3a090481d INFO knox.gateway (HadoopGroupProviderFilter.java:init(92)) - The group lookup is skipped for principals matching k8s-sa-.* 2026-11-06 11:33:46,176 a05cc992-5b1b-4504-b374-29a3a090481d DEBUG knox.gateway (HadoopGroupProviderFilter.java:mapGroupPrincipals(133)) - Skipping the group lookup of principal k8s-sa-test: it matches k8s-sa-.* 2026-11-06 11:33:46,176 a05cc992-5b1b-4504-b374-29a3a090481d DEBUG knox.gateway (VirtualGroupMapper.java:mapGroups(59)) - User k8s-sa-test (with group(s) []) added to group(s) [] 2026-11-06 11:33:46,214 a05cc992-5b1b-4504-b374-29a3a090481d DEBUG service.auth (AbstractAuthResource.java:lookupRoles(340)) - Looking up the roles of user k8s-sa-test with groups [] 2026-11-06 11:33:46,215 a05cc992-5b1b-4504-b374-29a3a090481d DEBUG services.ldap (RestApiLdapRolesLookup.java:lookupRoles(59)) - Sending role lookup request to http://localhost:8090/api/v1/roles/lookup: {"user_id":"k8s-sa-test","groups":[]} 2026-11-06 11:33:46,218 a05cc992-5b1b-4504-b374-29a3a090481d DEBUG services.ldap (RestApiLdapRolesLookup.java:lookupRoles(68)) - Received role lookup response from http://localhost:8090/api/v1/roles/lookup: HTTP 200, body: {"user_id": "k8s-sa-test", "roles": [{"scope": "platform", "name": "sa-role"}]} 2026-11-06 11:33:46,219 a05cc992-5b1b-4504-b374-29a3a090481d DEBUG service.auth (AbstractAuthResource.java:lookupRoles(342)) - Found roles for user k8s-sa-test: [platform:sa-role] 2026-11-06 11:34:20,573 607fd52e-d2bc-4824-82e5-f919e9d03855 DEBUG knox.gateway (GatewayFilter.java:doFilter(130)) - Received request: GET /auth/api/v1/pre 2026-11-06 11:34:20,573 607fd52e-d2bc-4824-82e5-f919e9d03855 INFO knox.gateway (HadoopGroupProviderFilter.java:hadoopGroups(161)) - Using Knox LDAP service to fetch groups... 2026-11-06 11:34:20,574 607fd52e-d2bc-4824-82e5-f919e9d03855 DEBUG services.ldap (UserSearchInterceptor.java:search(112)) - LDAP Search: dc=hadoop,dc=apache,dc=org | (uid=guest) | [*] 2026-11-06 11:34:20,576 607fd52e-d2bc-4824-82e5-f919e9d03855 DEBUG services.ldap (LdapProxyBackend.java:performPagedSearch(1055)) - LDAP Paged Search: dc=hadoop,dc=apache,dc=org | (uid=guest), page size 1,000, page 1 2026-11-06 11:34:20,578 607fd52e-d2bc-4824-82e5-f919e9d03855 DEBUG services.ldap (LdapProxyBackend.java:performPagedSearch(1095)) - LDAP Paged Search Completed: dc=hadoop,dc=apache,dc=org | (uid=guest). Found 1 entries 2026-11-06 11:34:20,578 607fd52e-d2bc-4824-82e5-f919e9d03855 DEBUG services.ldap (LdapProxyBackend.java:performPagedSearch(1055)) - LDAP Paged Search: ou=groups,dc=hadoop,dc=apache,dc=org | (|(member=uid=guest,ou=people,dc=hadoop,dc=apache,dc=org)(uniqueMember=uid=guest,ou=people,dc=hadoop,dc=apache,dc=org)), page size 1,000, page 1 2026-11-06 11:34:20,585 607fd52e-d2bc-4824-82e5-f919e9d03855 DEBUG services.ldap (LdapProxyBackend.java:performPagedSearch(1095)) - LDAP Paged Search Completed: ou=groups,dc=hadoop,dc=apache,dc=org | (|(member=uid=guest,ou=people,dc=hadoop,dc=apache,dc=org)(uniqueMember=uid=guest,ou=people,dc=hadoop,dc=apache,dc=org)). Found 0 entries 2026-11-06 11:34:20,586 607fd52e-d2bc-4824-82e5-f919e9d03855 DEBUG services.ldap (RestApiLdapRolesLookup.java:lookupRoles(59)) - Sending role lookup request to http://localhost:8090/api/v1/roles/lookup: {"user_id":"guest","groups":[]} 2026-11-06 11:34:20,589 607fd52e-d2bc-4824-82e5-f919e9d03855 DEBUG services.ldap (RestApiLdapRolesLookup.java:lookupRoles(68)) - Received role lookup response from http://localhost:8090/api/v1/roles/lookup: HTTP 200, body: {"user_id": "guest", "roles": [{"scope": "platform", "name": "guest-role"}]} 2026-11-06 11:34:20,589 607fd52e-d2bc-4824-82e5-f919e9d03855 DEBUG knox.gateway (HadoopGroupProviderFilter.java:hadoopGroups(166)) - The Knox LDAP service looked up the roles of principal guest; marking the roles lookup as executed for this request 2026-11-06 11:34:20,589 607fd52e-d2bc-4824-82e5-f919e9d03855 DEBUG knox.gateway (HadoopGroupProviderFilter.java:mapGroupPrincipals(141)) - Found groups for principal guest : [platform:guest-role] 2026-11-06 11:34:20,589 607fd52e-d2bc-4824-82e5-f919e9d03855 DEBUG knox.gateway (VirtualGroupMapper.java:mapGroups(59)) - User guest (with group(s) [platform:guest-role]) added to group(s) [] 2026-11-06 11:34:20,590 607fd52e-d2bc-4824-82e5-f919e9d03855 DEBUG service.auth (AbstractAuthResource.java:lookupRoles(338)) - The roles of user guest were already looked up by the identity assertion provider; skipping the roles lookup ``` ## Integration Tests No integration test added. The unit tests cover the new parameter, and it was also verified manually against the mock roles REST API. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
