[
https://issues.apache.org/jira/browse/HDDS-15067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chu Cheng Li updated HDDS-15067:
--------------------------------
Description:
Conditional MPU completion should reuse the same conditional write fields
already present in {{{}KeyArgs{}}}, but unlike normal conditional {{{}PUT{}}},
it does not need an open-key plus later commit bridge. The final key is
assembled and committed inside one OM transaction in
{{S3MultipartUploadCompleteRequest.validateAndUpdateCache(...)}} while holding
the normal bucket write lock
h3. Gateway Flow
# Parse {{If-None-Match}} / {{If-Match}} using the same shared helper used by
conditional {{{}PUT{}}}.
# Extend the existing {{completeMultipartUpload}} client API, or add a
conditional overload, so the gateway can pass the parsed condition to OM
through {{{}OmKeyArgs{}}}.
# Populate the request as follows:
** {{If-None-Match: *}} -> {{expectedDataGeneration =
OzoneConsts.EXPECTED_GEN_CREATE_IF_NOT_EXISTS}}
** {{If-Match: "<etag>"}} -> {{expectedETag = <etag>}}
# Ensure the OM protocol translator copies these optional fields from
{{OmKeyArgs}} into the {{CompleteMultiPartUploadRequest}} {{{}KeyArgs{}}}.
# Map conditional validation failures to the same S3 {{PreconditionFailed}}
response used by the other conditional write paths.
h3. OM Validation
Validation should occur in
{{S3MultipartUploadCompleteRequest.validateAndUpdateCache(...)}} after the
request acquires the bucket write lock and before it assembles the final key
from the uploaded parts.
The proposed flow is:
# Load the current committed destination key from {{{}keyTable{}}}.
# Reuse the same {{If-Match}} validation helper already used by conditional
{{PUT}} to convert {{expectedETag}} into {{{}expectedDataGeneration{}}}.
# Reuse the same atomic rewrite validation helper to evaluate:
** create-if-absent semantics for {{If-None-Match: *}}
** generation/ETag match semantics for {{If-Match}}
# If validation passes, continue with the existing MPU complete logic:
validate part order, validate part identity, compute the MPU ETag, write the
final key, and delete the multipart metadata/open-key state.
# Clear conditional-only fields before persisting the committed key so they
remain request-scoped metadata rather than part of the final key state.
Because the final destination validation and the key-table write happen under
the same OM bucket lock in one request, this path does not need a separate
second-phase commit revalidation step like conditional {{{}PutObject{}}}.
was:
Conditional MPU completion should reuse the same conditional write fields
already present in {{{}KeyArgs{}}}, but unlike normal conditional {{{}PUT{}}},
it does not need an open-key plus later commit bridge. The final key is
assembled and committed inside one OM transaction in
{{S3MultipartUploadCompleteRequest.validateAndUpdateCache(...)}} while holding
the normal bucket write l
h3. Gateway Flow
# Parse {{If-None-Match}} / {{If-Match}} using the same shared helper used by
conditional {{{}PUT{}}}.
# Extend the existing {{completeMultipartUpload}} client API, or add a
conditional overload, so the gateway can pass the parsed condition to OM
through {{{}OmKeyArgs{}}}.
# Populate the request as follows:
** {{If-None-Match: *}} -> {{expectedDataGeneration =
OzoneConsts.EXPECTED_GEN_CREATE_IF_NOT_EXISTS}}
** {{If-Match: "<etag>"}} -> {{expectedETag = <etag>}}
# Ensure the OM protocol translator copies these optional fields from
{{OmKeyArgs}} into the {{CompleteMultiPartUploadRequest}} {{{}KeyArgs{}}}.
# Map conditional validation failures to the same S3 {{PreconditionFailed}}
response used by the other conditional write paths.
h3. OM Validation
Validation should occur in
{{S3MultipartUploadCompleteRequest.validateAndUpdateCache(...)}} after the
request acquires the bucket write lock and before it assembles the final key
from the uploaded parts.
The proposed flow is:
# Load the current committed destination key from {{{}keyTable{}}}.
# Reuse the same {{If-Match}} validation helper already used by conditional
{{PUT}} to convert {{expectedETag}} into {{{}expectedDataGeneration{}}}.
# Reuse the same atomic rewrite validation helper to evaluate:
** create-if-absent semantics for {{If-None-Match: *}}
** generation/ETag match semantics for {{If-Match}}
# If validation passes, continue with the existing MPU complete logic:
validate part order, validate part identity, compute the MPU ETag, write the
final key, and delete the multipart metadata/open-key state.
# Clear conditional-only fields before persisting the committed key so they
remain request-scoped metadata rather than part of the final key state.
Because the final destination validation and the key-table write happen under
the same OM bucket lock in one request, this path does not need a separate
second-phase commit revalidation step like conditional {{{}PutObject{}}}.
> Conditional CompleteMultipartUpload
> -----------------------------------
>
> Key: HDDS-15067
> URL: https://issues.apache.org/jira/browse/HDDS-15067
> Project: Apache Ozone
> Issue Type: Sub-task
> Components: OM, s3gateway
> Reporter: Chu Cheng Li
> Priority: Major
> Attachments: image-2026-04-21-15-25-36-673.png
>
>
> Conditional MPU completion should reuse the same conditional write fields
> already present in {{{}KeyArgs{}}}, but unlike normal conditional
> {{{}PUT{}}}, it does not need an open-key plus later commit bridge. The final
> key is assembled and committed inside one OM transaction in
> {{S3MultipartUploadCompleteRequest.validateAndUpdateCache(...)}} while
> holding the normal bucket write lock
> h3. Gateway Flow
> # Parse {{If-None-Match}} / {{If-Match}} using the same shared helper used
> by conditional {{{}PUT{}}}.
> # Extend the existing {{completeMultipartUpload}} client API, or add a
> conditional overload, so the gateway can pass the parsed condition to OM
> through {{{}OmKeyArgs{}}}.
> # Populate the request as follows:
> ** {{If-None-Match: *}} -> {{expectedDataGeneration =
> OzoneConsts.EXPECTED_GEN_CREATE_IF_NOT_EXISTS}}
> ** {{If-Match: "<etag>"}} -> {{expectedETag = <etag>}}
> # Ensure the OM protocol translator copies these optional fields from
> {{OmKeyArgs}} into the {{CompleteMultiPartUploadRequest}} {{{}KeyArgs{}}}.
> # Map conditional validation failures to the same S3 {{PreconditionFailed}}
> response used by the other conditional write paths.
>
> h3. OM Validation
> Validation should occur in
> {{S3MultipartUploadCompleteRequest.validateAndUpdateCache(...)}} after the
> request acquires the bucket write lock and before it assembles the final key
> from the uploaded parts.
> The proposed flow is:
> # Load the current committed destination key from {{{}keyTable{}}}.
> # Reuse the same {{If-Match}} validation helper already used by conditional
> {{PUT}} to convert {{expectedETag}} into {{{}expectedDataGeneration{}}}.
> # Reuse the same atomic rewrite validation helper to evaluate:
> ** create-if-absent semantics for {{If-None-Match: *}}
> ** generation/ETag match semantics for {{If-Match}}
> # If validation passes, continue with the existing MPU complete logic:
> validate part order, validate part identity, compute the MPU ETag, write the
> final key, and delete the multipart metadata/open-key state.
> # Clear conditional-only fields before persisting the committed key so they
> remain request-scoped metadata rather than part of the final key state.
>
> Because the final destination validation and the key-table write happen under
> the same OM bucket lock in one request, this path does not need a separate
> second-phase commit revalidation step like conditional {{{}PutObject{}}}.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
