Huang Kuan Hao created HDDS-17028:
-------------------------------------
Summary: ListObjectVersions repeats CommonPrefixes across pages
Key: HDDS-17028
URL: https://issues.apache.org/jira/browse/HDDS-17028
Project: Apache Ozone
Issue Type: Bug
Components: s3gateway
Reporter: Huang Kuan Hao
ListObjectVersions can return the same CommonPrefixes entry on consecutive
pages when a page ends inside a prefix.
Reproduced on master at 10b5015f64d7 after HDDS-16739 / PR #11426 was merged.
h3. Reproduction
Create keys {{dir:a}}, {{dir:b}} and {{file}}. With the default S3 Gateway
configuration:
{noformat}
GET /bucket?versions&delimiter=:&max-keys=1
CommonPrefixes: dir:
NextKeyMarker: dir:a
NextVersionIdMarker: null
GET
/bucket?versions&delimiter=:&max-keys=1&key-marker=dir:a&version-id-marker=null
Actual: CommonPrefixes: dir:
Expected: Version: file, with no CommonPrefixes
{noformat}
The marker advances, so listing eventually completes, but paginators receive
duplicate prefixes and make unnecessary requests. The same filtering gap is
reproducible with keys {{dir/a}}, {{dir/b}}, {{file}}, {{delimiter=/}}, shallow
listing disabled, and {{key-marker=dir/a}}.
h3. Expected behavior
[The ListObjectVersions
API|https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectVersions.html]
excludes CommonPrefixes that are not lexicographically greater than
key-marker. Filter these prefixes before counting them against max-keys, and
cover a paginated listing that ends on a CommonPrefixes entry.
h3. Context
* Follow-up to [HDDS-16739 / PR
#11426|https://github.com/apache/ozone/pull/11426].
* The shared ListObjects V1 loop also has this behavior; this report focuses on
ListObjectVersions.
* [HDDS-898|https://issues.apache.org/jira/browse/HDDS-898] fixed a related
continuation-token problem. ListObjectVersions resumes with key-marker and
version-id-marker instead.
* Fresh local JDK 21 run of TestBucketList: 41 tests, 2 failures, 0 errors. The
39 existing tests passed; both added probes failed because the response still
contained the earlier CommonPrefixes entry. These were endpoint unit tests, not
requests against AWS S3.
Generated-by: Codex (GPT-6)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
