dupeiran001 opened a new issue, #4119:
URL: https://github.com/apache/opendal/issues/4119
I'm using s3 writer as an AsyncWriter, like this:
```rust
let mut writer = Operator::new(s3::default()).writer().await?;
tokio::io::copy(&mut stream_reader, &mut writer).await?;
writer.close().await?;
```
where the `stream_reader :impl AsyncRead`
but I got the following error: `code: "EntityTooSmall", message: "Your
proposed upload is smaller than the minimum allowed object size."`
this is more detailed error info:
```
2024-02-01 09:46:34.306 ERROR opendal::services: service=s3
operation=Writer::close path=ego/rfq/tbq/202402/1706751992585.zip
written=138017B -> data close failed: Unexpected (permanent) at Writer::close,
context: { uri: http://xxxxxx, response: Parts { status: 400, version:
HTTP/1.1, headers: {"server": "openresty", "date": "Thu, 01 Feb 2024 01:46:33
GMT", "content-type": "application/xml", "content-length": "260", "connection":
"keep-alive", "x-reqid": "xxx", "x-amz-request-id": "xxxx", "x-log": "X-Log"}
}, service: s3, path: ego/rfq/tbq/202402/1706751992585.zip } => S3Error { code:
"EntityTooSmall", message: "Your proposed upload is smaller than the minimum
allowed object size.", resource:
"/tobson0001/ego/rfq/tbq/202402/1706751992585.zip", request_id: "xxxx" }
2024-02-01 09:46:34.306 ERROR file_operation_rs::controller::zip: zip error:
Unexpected (permanent) at Writer::close, context: { uri:
http://xxxx.zip?uploadId=xx, response: Parts { status: 400, version: HTTP/1.1,
headers: {"server": "openresty", "date": "Thu, 01 Feb 2024 01:46:33 GMT",
"content-type": "application/xml", "content-length": "260", "connection":
"keep-alive", "x-reqid": "xxx", "x-amz-request-id": "xxx", "x-log": "X-Log"} },
service: s3, path: ego/rfq/tbq/202402/1706751992585.zip } => S3Error { code:
"EntityTooSmall", message: "Your proposed upload is smaller than the minimum
allowed object size.", resource:
"/tobson0001/ego/rfq/tbq/202402/1706751992585.zip", request_id: "xxx" }
```
And I found in the [aws
doc](https://docs.aws.amazon.com/AmazonS3/latest/API/API_CompleteMultipartUpload.html)
that when using multipart upload, each part should be more than 5 MiB and less
than 2GB, unless the last part.
I suppose opendal use the s3 multipart upload as the s3 writer backend, and
it should have some limit in the AsyncWrite implement.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
