Tran Hong Quan created JAMES-4137:
-------------------------------------
Summary: Upgrade S3 SDK to 2.30.x onward version
Key: JAMES-4137
URL: https://issues.apache.org/jira/browse/JAMES-4137
Project: James Server
Issue Type: Improvement
Components: Blob
Reporter: Tran Hong Quan
In S3 SDK version 2.30.0, AWS team introduced a breaking change in the
integrity check in the driver:
- Checksum is always required by default by AWS SDK >= 2.30.x
- AWS SDK >= 2.30.x does not support MD5 checksum anymore (which 3rd party S3
storages rely on), but relies on CRC32 checksum.
cf: [https://github.com/aws/aws-sdk-java-v2/discussions/5802]
This breaks the compatibility with 3rd party object storage(s) that still rely
on MD5 checksum. We failed to upgrade the SDK in the past:
[https://github.com/apache/james-project/pull/2638.]
It seems after receiving a lot of complaints from the community (Apache Hadoop,
Apache Iceberg, Apache Spark...), AWS has agreed to have an option to maintain
backward compatibility with S3-compatible storages. cf
[https://github.com/aws/aws-sdk-java-v2/pull/6055.|https://github.com/aws/aws-sdk-java-v2/pull/6055]
For example, Apache Iceberg is starting to adopt the backward compatibility
option: [https://github.com/apache/iceberg/pull/12264.]
Let's try to adopt and test this on our side, with care.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
