[jira] [Commented] (HDDS-1948) S3 MPU can't be created with octet-stream content-type

2019-08-23 Thread Nanda kumar (Jira) 


[ 
https://issues.apache.org/jira/browse/HDDS-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16914382#comment-16914382
 ] 

Nanda kumar commented on HDDS-1948:
---

Cherry-picked this to ozone-0.4.1.

> S3 MPU can't be created with octet-stream content-type 
> ---
>
> Key: HDDS-1948
> URL: https://issues.apache.org/jira/browse/HDDS-1948
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>  Components: S3
>Reporter: Elek, Marton
>Assignee: Elek, Marton
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.4.1, 0.5.0
>
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> This problem is reported offline by [~shaneku...@gmail.com].
> When aws-sdk-go is used to access to s3 gateway of Ozone it sends the Multi 
> Part Upload initialize message with "application/octet-stream" Content-Type. 
> This Content-Type is missing from the aws-cli which is used to reimplement s3 
> endpoint.
> The problem is that we use the same rest endpoint for initialize and complete 
> Multipart Upload request. For the completion we need the 
> CompleteMultipartUploadRequest parameter which is parsed from the body.
> For initialize we have an empty body which can't be serialized to 
> CompleteMultipartUploadRequest.
> The workaround is to set a specific content type from a filter which help up 
> to create two different REST method for initialize and completion message.
> Here is an example to test (using bogus AWS credentials).
> {code}
> curl -H 'Host:yourhost' -H 'User-Agent:aws-sdk-go/1.15.11 (go1.11.2; linux; 
> amd64)' -H 'Content-Length:0' -H 'Authorization:AWS4-HMAC-SHA256 
> Credential=qwe/20190809/ozone/s3/aws4_request, 
> SignedHeaders=content-type;host;x-amz-acl;x-amz-content-sha256;x-amz-date;x-amz-storage-class,
>  Signature=7726ed63990ba3f4f1f796d4ab263f5d9c3374528840f5e49d106dbef491f22c' 
> -H 'Content-Type:application/octet-stream' -H 'X-Amz-Acl:private' -H 
> 'X-Amz-Content-Sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'
>  -H 'X-Amz-Date:20190809T070142Z' -H 'X-Amz-Storage-Class:STANDARD' -H 
> 'Accept-Encoding:gzip' -X POST 
> 'http://localhost:/docker/docker/registry/v2/repositories/apache/ozone-runner/_uploads/2173f019-09c3-466b-bb7d-c31ce749d826/data?uploads
> {code}
> Without the patch it returns with HTTP 405 (Not supported Media Type).



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDDS-1948) S3 MPU can't be created with octet-stream content-type

2019-08-23 Thread Hudson (Jira) 


[ 
https://issues.apache.org/jira/browse/HDDS-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913983#comment-16913983
 ] 

Hudson commented on HDDS-1948:
--

FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #17173 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/17173/])
HDDS-1948. S3 MPU can't be created with octet-stream content-type  (bharat: rev 
edd708527d34d0bf3b09dc35a7f645f49e7becb3)
* (edit) 
hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestInitiateMultipartUpload.java
* (edit) 
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/VirtualHostStyleFilter.java
* (edit) 
hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestAbortMultipartUpload.java
* (edit) 
hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestMultipartUploadComplete.java
* (edit) 
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/ObjectEndpoint.java
* (edit) 
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/HeaderPreprocessor.java
* (edit) 
hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestPartUpload.java
* (edit) 
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/S3GatewayHttpServer.java
* (edit) 
hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestListParts.java


> S3 MPU can't be created with octet-stream content-type 
> ---
>
> Key: HDDS-1948
> URL: https://issues.apache.org/jira/browse/HDDS-1948
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>  Components: S3
>Reporter: Elek, Marton
>Assignee: Elek, Marton
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.5.0
>
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> This problem is reported offline by [~shaneku...@gmail.com].
> When aws-sdk-go is used to access to s3 gateway of Ozone it sends the Multi 
> Part Upload initialize message with "application/octet-stream" Content-Type. 
> This Content-Type is missing from the aws-cli which is used to reimplement s3 
> endpoint.
> The problem is that we use the same rest endpoint for initialize and complete 
> Multipart Upload request. For the completion we need the 
> CompleteMultipartUploadRequest parameter which is parsed from the body.
> For initialize we have an empty body which can't be serialized to 
> CompleteMultipartUploadRequest.
> The workaround is to set a specific content type from a filter which help up 
> to create two different REST method for initialize and completion message.
> Here is an example to test (using bogus AWS credentials).
> {code}
> curl -H 'Host:yourhost' -H 'User-Agent:aws-sdk-go/1.15.11 (go1.11.2; linux; 
> amd64)' -H 'Content-Length:0' -H 'Authorization:AWS4-HMAC-SHA256 
> Credential=qwe/20190809/ozone/s3/aws4_request, 
> SignedHeaders=content-type;host;x-amz-acl;x-amz-content-sha256;x-amz-date;x-amz-storage-class,
>  Signature=7726ed63990ba3f4f1f796d4ab263f5d9c3374528840f5e49d106dbef491f22c' 
> -H 'Content-Type:application/octet-stream' -H 'X-Amz-Acl:private' -H 
> 'X-Amz-Content-Sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'
>  -H 'X-Amz-Date:20190809T070142Z' -H 'X-Amz-Storage-Class:STANDARD' -H 
> 'Accept-Encoding:gzip' -X POST 
> 'http://localhost:/docker/docker/registry/v2/repositories/apache/ozone-runner/_uploads/2173f019-09c3-466b-bb7d-c31ce749d826/data?uploads
> {code}
> Without the patch it returns with HTTP 405 (Not supported Media Type).



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org