[ https://issues.apache.org/jira/browse/HDDS-3946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17158785#comment-17158785 ]
Bharat Viswanadham commented on HDDS-3946: ------------------------------------------ When a part is uploaded twice, in multipartKeyInfo, we override the part, and delete the old part. TestOzoneRpcClientAbstract.java testMultipartUploadOverride() tests this behavior. *Aws S3 behavior:* If you upload a new part using the same part number as a previously uploaded part, the previously uploaded part is overwritten. https://docs.aws.amazon.com/AmazonS3/latest/dev/mpuoverview.html So, during complete MultipartUpload, if you specify latest part information that should work. > s3g multi-upload failed with partName check > ------------------------------------------- > > Key: HDDS-3946 > URL: https://issues.apache.org/jira/browse/HDDS-3946 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Reporter: Sammi Chen > Priority: Major > > LOGs in S3g, > INVALID_PART org.apache.hadoop.ozone.om.exceptions.OMException: Complete > Multipart Upload Failed: volume: s325d55ad283aa400af464c76d713c07adbucket: > konajdk-profilerkey: > root@10.121.81.124/dfbd3e3905c34d73ac76ebae5650b7d3.hprof. Provided Part info > is { > /s325d55ad283aa400af464c76d713c07ad/konajdk-profiler/root@10.121.81.124/dfbd3e3905c34d73ac76ebae5650b7d3.hprof104478123544835723, > 200}, where as OM has partName > /s325d55ad283aa400af464c76d713c07ad/konajdk-profiler/root@10.121.81.124/dfbd3e3905c34d73ac76ebae5650b7d3.hprof104478114690135525 > at > org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.handleError(OzoneManagerProtocolClientSideTranslatorPB.java:589) > at > org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.completeMultipartUpload(OzoneManagerProtocolClientSideTranslatorPB.java:884) > at > org.apache.hadoop.ozone.client.rpc.RpcClient.completeMultipartUpload(RpcClient.java:900) > at > org.apache.hadoop.ozone.client.OzoneBucket.completeMultipartUpload(OzoneBucket.java:446) > at > org.apache.hadoop.ozone.s3.endpoint.ObjectEndpoint.completeMultipartUpload(ObjectEndpoint.java:476) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > The investigation shows that partNumber 200 block is committed twice with > different partName, because the ClientID for two commit is different. > 2020-07-08 20:00:50,087 | INFO | OMAudit | user=root | ip=100.76.18.99 | > op=COMMIT_MULTIPART_UPLOAD_PARTKEY > {volume=s325d55ad283aa400af464c76d713c07ad, bucket=konajdk-profiler, > key=root@10.121.81.124/dfbd3e3905c34d73ac76ebae5650b7d3.hprof, > dataSize=104857600, replicationType=RATIS, replicationFactor=ONE, > partNumber=200, > partName=/s325d55ad283aa400af464c76d713c07ad/konajdk-profiler/root@10.121.81.124/dfbd3e3905c34d73ac76ebae5650b7d3.hprof104478123544835723} > | ret=SUCCESS | > 2020-07-08 20:00:50,087 | INFO | OMAudit | user=root | ip=100.76.18.99 | > op=COMMIT_MULTIPART_UPLOAD_PARTKEY > {volume=s325d55ad283aa400af464c76d713c07ad, bucket=konajdk-profiler, > key=root@10.121.81.124/dfbd3e3905c34d73ac76ebae5650b7d3.hprof, > dataSize=104857600, replicationType=RATIS, replicationFactor=ONE, > partNumber=200, > partName=/s325d55ad283aa400af464c76d713c07ad/konajdk-profiler/root@10.121.81.124/dfbd3e3905c34d73ac76ebae5650b7d3.hprof104478123544835723} > | ret=SUCCESS | > So the question is, can we loose the partName check for this case? -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org