Chi-Hsuan Huang created HDDS-17696:
--------------------------------------
Summary: S3 CompleteMultipartUpload returns 500 when the parent
directory was replaced by a file after initiation
Key: HDDS-17696
URL: https://issues.apache.org/jira/browse/HDDS-17696
Project: Apache Ozone
Issue Type: Bug
Components: S3
Reporter: Chi-Hsuan Huang
Assignee: Chi-Hsuan Huang
In an FSO bucket, if the parent directory of an in\-progress multipart upload
is removed and a regular file is created with the same name,
{{CompleteMultipartUpload}} fails with HTTP 500 {{InternalError}} instead of a
4xx error.
_Steps to Reproduce_
Tested on master \(28c85143e5\) with the {{compose/ozone}} cluster and the AWS
CLI:
{code}
ozone sh bucket create /s3v/fso \-\-layout FILE\_SYSTEM\_OPTIMIZED
aws s3api create\-multipart\-upload \-\-bucket fso \-\-key a/b #
creates directory "a"
aws s3api upload\-part \-\-bucket fso \-\-key a/b \-\-part\-number 1
\-\-upload\-id $UPLOAD\_ID \-\-body part1
ozone fs \-rmdir ofs://om/s3v/fso/a # succeeds,
"a" has no committed children
aws s3api put\-object \-\-bucket fso \-\-key a \-\-body part1 # "a"
is now a file
aws s3api complete\-multipart\-upload \-\-bucket fso \-\-key a/b \-\-upload\-id
$UPLOAD\_ID
\-\-multipart\-upload 'Parts=\[{ETag=...,PartNumber=1}\]'
{code}
_Actual Result_
* {{CompleteMultipartUpload}} returns HTTP 500 {{InternalError}} \("We
encountered an internal error. Please try again."\). The AWS CLI retries it
several times.
* OM log:
{noformat}
ERROR multipart.S3MultipartUploadCompleteRequest: MultipartUpload Complete
request failed for Key: a/b in Volume/Bucket s3v/fso
DIRECTORY\_NOT\_FOUND org.apache.hadoop.ozone.om.exceptions.OMException: Failed
to find parent directory of a/b
at
...S3MultipartUploadCompleteRequestWithFSO.getDBOzoneKey\(S3MultipartUploadCompleteRequestWithFSO.java:132\)
at
...S3MultipartUploadCompleteRequest.validateAndUpdateCache\(S3MultipartUploadCompleteRequest.java:276\)
{noformat}
* The upload can still be aborted, and the file {{a}} is not affected.
_Expected Result_
{{CompleteMultipartUpload}} should be rejected with a 4xx error \(for example
400 {{InvalidRequest}}, consistent with {{PutObject}} under an existing file\),
not 500.
_Cause_
* {{S3MultipartUploadCompleteRequestWithFSO.checkDirectoryAlreadyExists}} only
rejects {{DIRECTORY\_EXISTS}}. It does not reject
{{FILE\_EXISTS\_IN\_GIVENPATH}}, so the request continues and fails later in
{{getDBOzoneKey}} with {{DIRECTORY\_NOT\_FOUND}}.
* S3 Gateway's {{S3ErrorTable.translateResultCode}} has no mapping for
{{DIRECTORY\_NOT\_FOUND}}, so it becomes {{INTERNAL\_ERROR}}.
_Notes_
* HDDS\-17644 rejects the same parent\-is\-a\-file case at
{{CreateMultipartUpload}}, and maps {{NOT\_A\_FILE}} to 400. It does not cover
this case, because the file is created after the upload was initiated.
* The "can this path be written as a file" check on {{OMDirectoryResult}} is
currently duplicated across {{OMFileCreateRequest}}, {{OMKeyCreateRequest}},
{{OMKeyCreateRequestWithFSO}}, {{S3InitiateMultipartUploadRequestWithFSO}} and
{{S3MultipartUploadCompleteRequestWithFSO}}, with different coverage. Sharing
one check would fix this case and keep the requests consistent.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]