Chi-Hsuan Huang created HDDS-17696:
--------------------------------------

             Summary: S3 CompleteMultipartUpload returns 500 when the parent 
directory was replaced by a file after initiation
                 Key: HDDS-17696
                 URL: https://issues.apache.org/jira/browse/HDDS-17696
             Project: Apache Ozone
          Issue Type: Bug
          Components: S3
            Reporter: Chi-Hsuan Huang
            Assignee: Chi-Hsuan Huang


In an FSO bucket, if the parent directory of an in\-progress multipart upload 
is removed and a regular file is created with the same name, 
{{CompleteMultipartUpload}} fails with HTTP 500 {{InternalError}} instead of a 
4xx error.

_Steps to Reproduce_

Tested on master \(28c85143e5\) with the {{compose/ozone}} cluster and the AWS 
CLI:

{code}
ozone sh bucket create /s3v/fso \-\-layout FILE\_SYSTEM\_OPTIMIZED
aws s3api create\-multipart\-upload \-\-bucket fso \-\-key a/b          # 
creates directory "a"
aws s3api upload\-part \-\-bucket fso \-\-key a/b \-\-part\-number 1 
\-\-upload\-id $UPLOAD\_ID \-\-body part1
ozone fs \-rmdir ofs://om/s3v/fso/a                                # succeeds, 
"a" has no committed children
aws s3api put\-object \-\-bucket fso \-\-key a \-\-body part1            # "a" 
is now a file
aws s3api complete\-multipart\-upload \-\-bucket fso \-\-key a/b \-\-upload\-id 
$UPLOAD\_ID 
  \-\-multipart\-upload 'Parts=\[{ETag=...,PartNumber=1}\]'
{code}

_Actual Result_

* {{CompleteMultipartUpload}} returns HTTP 500 {{InternalError}} \("We 
encountered an internal error. Please try again."\). The AWS CLI retries it 
several times.
* OM log:
{noformat}
ERROR multipart.S3MultipartUploadCompleteRequest: MultipartUpload Complete 
request failed for Key: a/b in Volume/Bucket s3v/fso
DIRECTORY\_NOT\_FOUND org.apache.hadoop.ozone.om.exceptions.OMException: Failed 
to find parent directory of a/b
at 
...S3MultipartUploadCompleteRequestWithFSO.getDBOzoneKey\(S3MultipartUploadCompleteRequestWithFSO.java:132\)
at 
...S3MultipartUploadCompleteRequest.validateAndUpdateCache\(S3MultipartUploadCompleteRequest.java:276\)
{noformat}
* The upload can still be aborted, and the file {{a}} is not affected.

_Expected Result_

{{CompleteMultipartUpload}} should be rejected with a 4xx error \(for example 
400 {{InvalidRequest}}, consistent with {{PutObject}} under an existing file\), 
not 500.

_Cause_

* {{S3MultipartUploadCompleteRequestWithFSO.checkDirectoryAlreadyExists}} only 
rejects {{DIRECTORY\_EXISTS}}. It does not reject 
{{FILE\_EXISTS\_IN\_GIVENPATH}}, so the request continues and fails later in 
{{getDBOzoneKey}} with {{DIRECTORY\_NOT\_FOUND}}.
* S3 Gateway's {{S3ErrorTable.translateResultCode}} has no mapping for 
{{DIRECTORY\_NOT\_FOUND}}, so it becomes {{INTERNAL\_ERROR}}.

_Notes_

* HDDS\-17644 rejects the same parent\-is\-a\-file case at 
{{CreateMultipartUpload}}, and maps {{NOT\_A\_FILE}} to 400. It does not cover 
this case, because the file is created after the upload was initiated.
* The "can this path be written as a file" check on {{OMDirectoryResult}} is 
currently duplicated across {{OMFileCreateRequest}}, {{OMKeyCreateRequest}}, 
{{OMKeyCreateRequestWithFSO}}, {{S3InitiateMultipartUploadRequestWithFSO}} and 
{{S3MultipartUploadCompleteRequestWithFSO}}, with different coverage. Sharing 
one check would fix this case and keep the requests consistent.





--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to