sreejasahithi opened a new pull request, #10300:
URL: https://github.com/apache/ozone/pull/10300
## What changes were proposed in this pull request?
The ozone admin container list --all command produces duplicate container
entries in its output. The duplicates are non-deterministic across runs, which
containers are duplicated depends on the order in which the SCM returns them,
which can vary.
Root Cause:
The bug is a mismatch between the server-side sort key and the client-side
pagination key.
Server side (SCMClientProtocolServer.listContainerInternal):The method
filters containers by `containerID >= startContainerID`, then calls `.sorted()`
which invokes `ContainerInfo.compareTo()`.
That comparator is defined as:
> private static final Comparator<ContainerInfo> COMPARATOR =
Comparator.comparingLong(info -> info.getLastUsed().toEpochMilli());
So the batch is returned sorted by lastUsed timestamp, not by containerID.
Client side (ListSubcommand.listAllContainers): The pagination loop advances
the cursor using the last returned element's container ID.
This assumes the last element of the batch has the highest container ID.
That is only true if the batch is sorted by container ID. Since it is actually
sorted by lastUsed, the last element can have a lower container ID than other
elements already in the same batch.
Example: With batch size 40, a batch may return containers with IDs [...,
41, 42, 44, 40] (sorted by lastUsed). The cursor is set to 40 + 1 = 41. The
next batch fetches containers with containerID >= 41, returning [41, 42, 43,
44, 45, ...] — re-fetching 41, 42, and 44.
Fix :
In SCMClientProtocolServer.listContainerInternal, replace the lastUsed-based
sort with a containerID-based sort, aligning the sort key with the pagination
key:
> .sorted(Comparator.comparing(ContainerInfo::containerID))
## What is the link to the Apache JIRA
[HDDS-15305](https://issues.apache.org/jira/browse/HDDS-15305)
## How was this patch tested?
Manually tested in docket cluster.
Green CI : https://github.com/sreejasahithi/ozone/actions/runs/26017534135
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
