[
https://issues.apache.org/jira/browse/HDDS-3946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sammi Chen updated HDDS-3946:
-----------------------------
Description:
LOGs in S3g,
INVALID_PART org.apache.hadoop.ozone.om.exceptions.OMException: Complete
Multipart Upload Failed: volume: s325d55ad283aa400af464c76d713c07adbucket:
konajdk-profilerkey: root@10.121.81.124/dfbd3e3905c34d73ac76ebae5650b7d3.hprof.
Provided Part info is {
/s325d55ad283aa400af464c76d713c07ad/konajdk-profiler/root@10.121.81.124/dfbd3e3905c34d73ac76ebae5650b7d3.hprof104478123544835723,
200}, where as OM has partName
/s325d55ad283aa400af464c76d713c07ad/konajdk-profiler/root@10.121.81.124/dfbd3e3905c34d73ac76ebae5650b7d3.hprof104478114690135525
at
org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.handleError(OzoneManagerProtocolClientSideTranslatorPB.java:589)
at
org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.completeMultipartUpload(OzoneManagerProtocolClientSideTranslatorPB.java:884)
at
org.apache.hadoop.ozone.client.rpc.RpcClient.completeMultipartUpload(RpcClient.java:900)
at
org.apache.hadoop.ozone.client.OzoneBucket.completeMultipartUpload(OzoneBucket.java:446)
at
org.apache.hadoop.ozone.s3.endpoint.ObjectEndpoint.completeMultipartUpload(ObjectEndpoint.java:476)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
The investigation shows that partNumber 200 block is committed twice with
different partName, because the ClientID for two commit is different.
2020-07-08 20:00:50,087 | INFO | OMAudit | user=root | ip=100.76.18.99 |
op=COMMIT_MULTIPART_UPLOAD_PARTKEY {volume=s325d55ad283aa400af464c76d713c07ad,
bucket=konajdk-profiler,
key=root@10.121.81.124/dfbd3e3905c34d73ac76ebae5650b7d3.hprof,
dataSize=104857600, replicationType=RATIS, replicationFactor=ONE,
partNumber=200,
partName=/s325d55ad283aa400af464c76d713c07ad/konajdk-profiler/root@10.121.81.124/dfbd3e3905c34d73ac76ebae5650b7d3.hprof104478123544835723}
| ret=SUCCESS |
2020-07-08 20:00:50,087 | INFO | OMAudit | user=root | ip=100.76.18.99 |
op=COMMIT_MULTIPART_UPLOAD_PARTKEY {volume=s325d55ad283aa400af464c76d713c07ad,
bucket=konajdk-profiler,
key=root@10.121.81.124/dfbd3e3905c34d73ac76ebae5650b7d3.hprof,
dataSize=104857600, replicationType=RATIS, replicationFactor=ONE,
partNumber=200,
partName=/s325d55ad283aa400af464c76d713c07ad/konajdk-profiler/root@10.121.81.124/dfbd3e3905c34d73ac76ebae5650b7d3.hprof104478123544835723}
| ret=SUCCESS |
So the question is, can we loose the partName check for this case？
was:
LOGs in S3g,
INVALID_PART org.apache.hadoop.ozone.om.exceptions.OMException: Complete
Multipart Upload Failed: volume: s325d55ad283aa400af464c76d713c07adbucket:
konajdk-profilerkey: root@10.121.81.124/dfbd3e3905c34d73ac76ebae5650b7d3.hprof.
Provided Part info is {
/s325d55ad283aa400af464c76d713c07ad/konajdk-profiler/root@10.121.81.124/dfbd3e3905c34d73ac76ebae5650b7d3.hprof104478123544835723,
200}, where as OM has partName
/s325d55ad283aa400af464c76d713c07ad/konajdk-profiler/root@10.121.81.124/dfbd3e3905c34d73ac76ebae5650b7d3.hprof104478114690135525
at
org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.handleError(OzoneManagerProtocolClientSideTranslatorPB.java:589)
at
org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.completeMultipartUpload(OzoneManagerProtocolClientSideTranslatorPB.java:884)
at
org.apache.hadoop.ozone.client.rpc.RpcClient.completeMultipartUpload(RpcClient.java:900)
at
org.apache.hadoop.ozone.client.OzoneBucket.completeMultipartUpload(OzoneBucket.java:446)
at
org.apache.hadoop.ozone.s3.endpoint.ObjectEndpoint.completeMultipartUpload(ObjectEndpoint.java:476)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
The investigation shows that partNumber 200 block is committed twice with
different partName, because the ClientID for two commit is different.
2020-07-08 20:00:50,087 | INFO | OMAudit | user=root | ip=100.76.18.99 |
op=COMMIT_MULTIPART_UPLOAD_PARTKEY {volume=s325d55ad283aa400af464c76d713c07ad,
bucket=konajdk-profiler,
key=root@10.121.81.124/dfbd3e3905c34d73ac76ebae5650b7d3.hprof,
dataSize=104857600, replicationType=RATIS, replicationFactor=ONE,
partNumber=200,
partName=/s325d55ad283aa400af464c76d713c07ad/konajdk-profiler/root@10.121.81.124/dfbd3e3905c34d73ac76ebae5650b7d3.hprof104478123544835723}
| ret=SUCCESS |
2020-07-08 20:00:50,087 | INFO | OMAudit | user=root | ip=100.76.18.99 |
op=COMMIT_MULTIPART_UPLOAD_PARTKEY {volume=s325d55ad283aa400af464c76d713c07ad,
bucket=konajdk-profiler,
key=root@10.121.81.124/dfbd3e3905c34d73ac76ebae5650b7d3.hprof,
dataSize=104857600, replicationType=RATIS, replicationFactor=ONE,
partNumber=200,
partName=/s325d55ad283aa400af464c76d713c07ad/konajdk-profiler/root@10.121.81.124/dfbd3e3905c34d73ac76ebae5650b7d3.hprof104478123544835723}
| ret=SUCCESS |
So the question is, can we loose the partName change for this case？
> s3g multi-upload failed with partName check
> -------------------------------------------
>
> Key: HDDS-3946
> URL: https://issues.apache.org/jira/browse/HDDS-3946
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Reporter: Sammi Chen
> Priority: Major
>
> LOGs in S3g,
> INVALID_PART org.apache.hadoop.ozone.om.exceptions.OMException: Complete
> Multipart Upload Failed: volume: s325d55ad283aa400af464c76d713c07adbucket:
> konajdk-profilerkey:
> root@10.121.81.124/dfbd3e3905c34d73ac76ebae5650b7d3.hprof. Provided Part info
> is {
> /s325d55ad283aa400af464c76d713c07ad/konajdk-profiler/root@10.121.81.124/dfbd3e3905c34d73ac76ebae5650b7d3.hprof104478123544835723,
> 200}, where as OM has partName
> /s325d55ad283aa400af464c76d713c07ad/konajdk-profiler/root@10.121.81.124/dfbd3e3905c34d73ac76ebae5650b7d3.hprof104478114690135525
> at
> org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.handleError(OzoneManagerProtocolClientSideTranslatorPB.java:589)
> at
> org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.completeMultipartUpload(OzoneManagerProtocolClientSideTranslatorPB.java:884)
> at
> org.apache.hadoop.ozone.client.rpc.RpcClient.completeMultipartUpload(RpcClient.java:900)
> at
> org.apache.hadoop.ozone.client.OzoneBucket.completeMultipartUpload(OzoneBucket.java:446)
> at
> org.apache.hadoop.ozone.s3.endpoint.ObjectEndpoint.completeMultipartUpload(ObjectEndpoint.java:476)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> The investigation shows that partNumber 200 block is committed twice with
> different partName, because the ClientID for two commit is different.
> 2020-07-08 20:00:50,087 | INFO | OMAudit | user=root | ip=100.76.18.99 |
> op=COMMIT_MULTIPART_UPLOAD_PARTKEY
> {volume=s325d55ad283aa400af464c76d713c07ad, bucket=konajdk-profiler,
> key=root@10.121.81.124/dfbd3e3905c34d73ac76ebae5650b7d3.hprof,
> dataSize=104857600, replicationType=RATIS, replicationFactor=ONE,
> partNumber=200,
> partName=/s325d55ad283aa400af464c76d713c07ad/konajdk-profiler/root@10.121.81.124/dfbd3e3905c34d73ac76ebae5650b7d3.hprof104478123544835723}
> | ret=SUCCESS |
> 2020-07-08 20:00:50,087 | INFO | OMAudit | user=root | ip=100.76.18.99 |
> op=COMMIT_MULTIPART_UPLOAD_PARTKEY
> {volume=s325d55ad283aa400af464c76d713c07ad, bucket=konajdk-profiler,
> key=root@10.121.81.124/dfbd3e3905c34d73ac76ebae5650b7d3.hprof,
> dataSize=104857600, replicationType=RATIS, replicationFactor=ONE,
> partNumber=200,
> partName=/s325d55ad283aa400af464c76d713c07ad/konajdk-profiler/root@10.121.81.124/dfbd3e3905c34d73ac76ebae5650b7d3.hprof104478123544835723}
> | ret=SUCCESS |
> So the question is, can we loose the partName check for this case？
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
