[
https://issues.apache.org/jira/browse/KNOX-3406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18102230#comment-18102230
]
David Han commented on KNOX-3406:
---------------------------------
The root cause presented does not match the current behavior of the
LdapProxyBackend since the addition of paging. The LdapProxyBackend pages
through all the entries and closes the cursor prior to retrieving the groups so
the cursor should be closed. However, I have reproduced some intermittent test
failures and am analyzing the problem.
The linked failure above has the following error:
```
'analyst' not found in ['level1']
```
In this case, only one group `level1` was retrieved.
The errors in my repro show:
```
tests-1 | self = <test_knox_ldap_proxy_search.TestKnoxLdapProxySearch
testMethod=test_search_all_groups_by_objectclass>
tests-1 |
tests-1 | def test_search_all_groups_by_objectclass(self) -> None:
tests-1 | """All groupOfNames entries under ou=groups are returned."""
tests-1 | groups = self.rdn_values(GROUPS_BASE,
"(objectClass=groupOfNames)")
tests-1 | for expected in ("analyst", "scientist", "admin", "level1",
"level2", "level3"):
tests-1 | > self.assertIn(expected, groups)
tests-1 | E AssertionError: 'scientist' not found in ['analyst',
'level3', 'level1']
```
This shows that 3 groups were retrieved, which matches the page size. I'm
taking a closer look at the paging behavior to see if we could be missing the
cookie.
> Fix intermittent LDAP proxy search truncation
> ---------------------------------------------
>
> Key: KNOX-3406
> URL: https://issues.apache.org/jira/browse/KNOX-3406
> Project: Apache Knox
> Issue Type: Bug
> Components: Server
> Affects Versions: 3.0.0
> Reporter: Tamás Marcinkovics
> Assignee: David Han
> Priority: Minor
>
> Problem: CI intermittently fails test_search_all_groups_by_objectclass — an
> LDAP
> subtree search for (objectClass=groupOfNames) returns only ['level1']
> instead of all 6 groups.
> [https://github.com/apache/knox/actions/runs/30453311904/attempts/1]
> test_knox_ldap_proxy_search.py::TestKnoxLdapProxySearch::test_search_all_groups_by_objectclass
> ests-1 | =================================== FAILURES
> ===================================
> tests-1 | ________
> TestKnoxLdapProxySearch.test_search_all_groups_by_objectclass _________
> tests-1 |
> tests-1 | self = <test_knox_ldap_proxy_search.TestKnoxLdapProxySearch
> testMethod=test_search_all_groups_by_objectclass>
> tests-1 |
> tests-1 | def test_search_all_groups_by_objectclass(self) -> None:
> tests-1 | """All groupOfNames entries under ou=groups are returned."""
> tests-1 | groups = self.rdn_values(GROUPS_BASE, "(objectClass=groupOfNames)")
> tests-1 | for expected in ("analyst", "scientist", "admin", "level1",
> "level2", "level3"):
> tests-1 | > self.assertIn(expected, groups)
> tests-1 | E AssertionError: 'analyst' not found in ['level1']
> tests-1 |
> tests-1 | test_knox_ldap_proxy_search.py:97: AssertionError
>
> I could reproduce this on an ubuntu vm.
>
> Root cause:
> LdapProxyBackend.search() opens an EntryCursor
> over the outer search, then inside the while (cursor.next()) loop calls
> addGroupMemberships(entry, connection, ...) which, when
> useMemberOf=false, calls getUserGroupsInternal(connection, ...)
> , opening a second EntryCursor on the same LdapConnection.
> The Apache Directory LDAP client is not designed for two concurrent
> SearchRequests on one connection: the SearchResultDone PDU from the inner
> search is consumed
> by the outer cursor, which interprets it as "no more entries" and
> terminates early after only 1 result.
> This is a nested/concurrent cursor problem. It is intermittent because the
> inner search's response must race ahead of the outer cursor's next() call —
> which is CPU/network-scheduling dependent and more likely in CI's
> faster-burst containers.
> Fix:
> Drain the outer cursor first, then enrich.
> In search(), collect all raw entries into a list before calling
> addGroupMemberships on any of them. This ensures the outer EntryCursor is
> fully consumed and closed before any secondary LDAP operation is issued on
> the connection.
> Three methods need fixing:
> 1. searchUsers — outer while (cursor.next()) loop calls addGroupMemberships
> on each iteration. Fix: collect all sourceEntry objects into a list inside
> the cursor block, then iterate that list after the cursor closes.
> 2. search — same pattern as searchUsers. Same fix.
> 3. getUser — outer cursor does cursor.next() then calls addGroupMemberships
> before the cursor is closed. Since it only reads one entry though, this is
> lower risk: the inner search fires after cursor.next() returns, while the
> outer cursor is still open. The fix: copy the entry out, close the cursor
> (let the try-with-resources do it naturally by moving addGroupMemberships
> after the try block), then call addGroupMemberships.
> getUserGroupsEntries also open cursors, but those are called from
> addGroupMemberships — they are the inner cursors, not the outer ones. This
> does not need to be changed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
