Tamás Marcinkovics created KNOX-3406:
----------------------------------------
Summary: Fix intermittent LDAP proxy search truncation
Key: KNOX-3406
URL: https://issues.apache.org/jira/browse/KNOX-3406
Project: Apache Knox
Issue Type: Task
Components: Server
Affects Versions: 3.0.0
Reporter: Tamás Marcinkovics
Assignee: David Han
Problem: CI intermittently fails test_search_all_groups_by_objectclass — an LDAP
subtree search for (objectClass=groupOfNames) returns only ['level1']
instead of all 6 groups.
[https://github.com/apache/knox/actions/runs/30453311904/attempts/1]
test_knox_ldap_proxy_search.py::TestKnoxLdapProxySearch::test_search_all_groups_by_objectclass
ests-1 | =================================== FAILURES
===================================
tests-1 | ________
TestKnoxLdapProxySearch.test_search_all_groups_by_objectclass _________
tests-1 |
tests-1 | self = <test_knox_ldap_proxy_search.TestKnoxLdapProxySearch
testMethod=test_search_all_groups_by_objectclass>
tests-1 |
tests-1 | def test_search_all_groups_by_objectclass(self) -> None:
tests-1 | """All groupOfNames entries under ou=groups are returned."""
tests-1 | groups = self.rdn_values(GROUPS_BASE, "(objectClass=groupOfNames)")
tests-1 | for expected in ("analyst", "scientist", "admin", "level1", "level2",
"level3"):
tests-1 | > self.assertIn(expected, groups)
tests-1 | E AssertionError: 'analyst' not found in ['level1']
tests-1 |
tests-1 | test_knox_ldap_proxy_search.py:97: AssertionError
I could reproduce this on an ubuntu vm.
Root cause:
LdapProxyBackend.search() opens an EntryCursor
over the outer search, then inside the while (cursor.next()) loop calls
addGroupMemberships(entry, connection, ...) which, when
useMemberOf=false, calls getUserGroupsInternal(connection, ...)
, opening a second EntryCursor on the same LdapConnection.
The Apache Directory LDAP client is not designed for two concurrent
SearchRequests on one connection: the SearchResultDone PDU from the inner
search is consumed
by the outer cursor, which interprets it as "no more entries" and
terminates early after only 1 result.
This is a nested/concurrent cursor problem. It is intermittent because the
inner search's response must race ahead of the outer cursor's next() call —
which is CPU/network-scheduling dependent and more likely in CI's
faster-burst containers.
Fix:
Drain the outer cursor first, then enrich.
In search(), collect all raw entries into a list before calling
addGroupMemberships on any of them. This ensures the outer EntryCursor is
fully consumed and closed before any secondary LDAP operation is issued on
the connection.
Three methods need fixing:
1. searchUsers — outer while (cursor.next()) loop calls addGroupMemberships on
each iteration. Fix: collect all sourceEntry objects into a list inside the
cursor block, then iterate that list after the cursor closes.
2. search — same pattern as searchUsers. Same fix.
3. getUser — outer cursor does cursor.next() then calls addGroupMemberships
before the cursor is closed. Since it only reads one entry though, this is
lower risk: the inner search fires after cursor.next() returns, while the outer
cursor is still open. The fix: copy the entry out, close the cursor (let the
try-with-resources do it naturally by moving addGroupMemberships after the try
block), then call addGroupMemberships.
getUserGroupsEntries also open cursors, but those are called from
addGroupMemberships — they are the inner cursors, not the outer ones. This does
not need to be changed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
