Steve Loughran created HADOOP-19484:
---------------------------------------
Summary: S3A: disable strong integrity checksum feature from
recent AWS SDKs
Key: HADOOP-19484
URL: https://issues.apache.org/jira/browse/HADOOP-19484
Project: Hadoop Common
Issue Type: Bug
Components: fs/s3
Affects Versions: 3.4.1, 3.5.0
Reporter: Steve Loughran
AWS SDK 2.30.0+ added checksum headers to all requests
https://github.com/aws/aws-sdk-java-v2/discussions/5802
This apparently causes a lot of regressions, especially with third party
stores, as reported in icebeg
https://github.com/apache/iceberg/pull/12264
note the disclaimer that the SDK is for AWS S3 storage only
bq. the AWS SDKs and CLI are designed for usage with official AWS services. We
may introduce and enable new features by default, such as these new default
integrity protections, prior to them being supported or otherwise handled by
third-party service implementations.
I understand why the team doesn't bother testing with other people's stores,
but we have to support them and need to consider all new SDK features are
potentially trouble, so disable them by default.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
