[
https://issues.apache.org/jira/browse/HDDS-17696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chi-Hsuan Huang updated HDDS-17696:
-----------------------------------
Status: Patch Available (was: Open)
> S3 CompleteMultipartUpload returns 500 when the parent directory was replaced
> by a file after initiation
> --------------------------------------------------------------------------------------------------------
>
> Key: HDDS-17696
> URL: https://issues.apache.org/jira/browse/HDDS-17696
> Project: Apache Ozone
> Issue Type: Bug
> Components: S3
> Reporter: Chi-Hsuan Huang
> Assignee: Chi-Hsuan Huang
> Priority: Minor
> Labels: pull-request-available
>
> In an FSO bucket, if the parent directory of an in\-progress multipart upload
> is removed and a regular file is created with the same name,
> {{CompleteMultipartUpload}} fails with HTTP 500 {{InternalError}} instead of
> a 4xx error.
> _Steps to Reproduce_
> Tested on master \(28c85143e5\) with the {{compose/ozone}} cluster and the
> AWS CLI:
> {code}
> ozone sh bucket create /s3v/fso \-\-layout FILE\_SYSTEM\_OPTIMIZED
> aws s3api create\-multipart\-upload \-\-bucket fso \-\-key a/b #
> creates directory "a"
> aws s3api upload\-part \-\-bucket fso \-\-key a/b \-\-part\-number 1
> \-\-upload\-id $UPLOAD\_ID \-\-body part1
> ozone fs \-rmdir ofs://om/s3v/fso/a #
> succeeds, "a" has no committed children
> aws s3api put\-object \-\-bucket fso \-\-key a \-\-body part1 #
> "a" is now a file
> aws s3api complete\-multipart\-upload \-\-bucket fso \-\-key a/b
> \-\-upload\-id $UPLOAD\_ID
> \-\-multipart\-upload 'Parts=\[{ETag=...,PartNumber=1}\]'
> {code}
> _Actual Result_
> * {{CompleteMultipartUpload}} returns HTTP 500 {{InternalError}} \("We
> encountered an internal error. Please try again."\). The AWS CLI retries it
> several times.
> * OM log:
> {noformat}
> ERROR multipart.S3MultipartUploadCompleteRequest: MultipartUpload Complete
> request failed for Key: a/b in Volume/Bucket s3v/fso
> DIRECTORY\_NOT\_FOUND org.apache.hadoop.ozone.om.exceptions.OMException:
> Failed to find parent directory of a/b
> at
> ...S3MultipartUploadCompleteRequestWithFSO.getDBOzoneKey\(S3MultipartUploadCompleteRequestWithFSO.java:132\)
> at
> ...S3MultipartUploadCompleteRequest.validateAndUpdateCache\(S3MultipartUploadCompleteRequest.java:276\)
> {noformat}
> * The upload can still be aborted, and the file {{a}} is not affected.
> _Expected Result_
> {{CompleteMultipartUpload}} should be rejected with a 4xx error \(for example
> 400 {{InvalidRequest}}, consistent with {{PutObject}} under an existing
> file\), not 500.
> _Cause_
> * {{S3MultipartUploadCompleteRequestWithFSO.checkDirectoryAlreadyExists}}
> only rejects {{DIRECTORY\_EXISTS}}. It does not reject
> {{FILE\_EXISTS\_IN\_GIVENPATH}}, so the request continues and fails later in
> {{getDBOzoneKey}} with {{DIRECTORY\_NOT\_FOUND}}.
> * S3 Gateway's {{S3ErrorTable.translateResultCode}} has no mapping for
> {{DIRECTORY\_NOT\_FOUND}}, so it becomes {{INTERNAL\_ERROR}}.
> _Notes_
> * HDDS\-17644 rejects the same parent\-is\-a\-file case at
> {{CreateMultipartUpload}}, and maps {{NOT\_A\_FILE}} to 400. It does not
> cover this case, because the file is created after the upload was initiated.
> * The "can this path be written as a file" check on {{OMDirectoryResult}} is
> currently duplicated across {{OMFileCreateRequest}}, {{OMKeyCreateRequest}},
> {{OMKeyCreateRequestWithFSO}}, {{S3InitiateMultipartUploadRequestWithFSO}}
> and {{S3MultipartUploadCompleteRequestWithFSO}}, with different coverage.
> Sharing one check would fix this case and keep the requests consistent.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
