[
https://issues.apache.org/jira/browse/HDDS-17028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Huang Kuan Hao reassigned HDDS-17028:
-------------------------------------
Assignee: Huang Kuan Hao
> ListObjectVersions repeats CommonPrefixes across pages
> ------------------------------------------------------
>
> Key: HDDS-17028
> URL: https://issues.apache.org/jira/browse/HDDS-17028
> Project: Apache Ozone
> Issue Type: Bug
> Components: s3gateway
> Reporter: Huang Kuan Hao
> Assignee: Huang Kuan Hao
> Priority: Major
>
> ListObjectVersions can return the same CommonPrefixes entry on consecutive
> pages when a page ends inside a prefix.
> Reproduced on master at 10b5015f64d7 after HDDS-16739 / PR #11426 was merged.
> h3. Reproduction
> Create keys {{dir:a}}, {{dir:b}} and {{file}}. With the default S3 Gateway
> configuration:
> {noformat}
> GET /bucket?versions&delimiter=:&max-keys=1
> CommonPrefixes: dir:
> NextKeyMarker: dir:a
> NextVersionIdMarker: null
> GET
> /bucket?versions&delimiter=:&max-keys=1&key-marker=dir:a&version-id-marker=null
> Actual: CommonPrefixes: dir:
> Expected: Version: file, with no CommonPrefixes
> {noformat}
> The marker advances, so listing eventually completes, but paginators receive
> duplicate prefixes and make unnecessary requests. The same filtering gap is
> reproducible with keys {{dir/a}}, {{dir/b}}, {{file}}, {{delimiter=/}},
> shallow listing disabled, and {{key-marker=dir/a}}.
> h3. Expected behavior
> [The ListObjectVersions
> API|https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectVersions.html]
> excludes CommonPrefixes that are not lexicographically greater than
> key-marker. Filter these prefixes before counting them against max-keys, and
> cover a paginated listing that ends on a CommonPrefixes entry.
> h3. Context
> * Follow-up to [HDDS-16739 / PR
> #11426|https://github.com/apache/ozone/pull/11426].
> * The shared ListObjects V1 loop also has this behavior; this report focuses
> on ListObjectVersions.
> * [HDDS-898|https://issues.apache.org/jira/browse/HDDS-898] fixed a related
> continuation-token problem. ListObjectVersions resumes with key-marker and
> version-id-marker instead.
> * Fresh local JDK 21 run of TestBucketList: 41 tests, 2 failures, 0 errors.
> The 39 existing tests passed; both added probes failed because the response
> still contained the earlier CommonPrefixes entry. These were endpoint unit
> tests, not requests against AWS S3.
> Generated-by: Codex (GPT-6)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
