h-vetinari opened a new issue, #45304: URL: https://github.com/apache/arrow/issues/45304
### Describe the enhancement requested Not sure whether to classify this as a bug or as an enhancement. It feels like a regression, but likely in AWS itself. In conda-forge, we don't build all patch versions of the SDK, because it gets released way too often to migrate ever time. Recently we went from 1.1.458 to 1.1.488, which ran into consistent errors on unix. Failures look roughly like this ``` [----------] 28 tests from TestS3FS [ RUN ] TestS3FS.GetFileInfoRoot $SRC_DIR/cpp/src/arrow/filesystem/s3fs_test.cc:519: Failure Failed 'PopulateTestBucket()' failed with IOError: AWS Error UNKNOWN (HTTP status 400) during PutObject operation: Unable to parse ExceptionName: BadRequest Message: malformed chunked encoding [ FAILED ] TestS3FS.GetFileInfoRoot (43 ms) [ RUN ] TestS3FS.GetFileInfoBucket $SRC_DIR/cpp/src/arrow/filesystem/s3fs_test.cc:519: Failure Failed 'PopulateTestBucket()' failed with IOError: AWS Error UNKNOWN (HTTP status 400) during PutObject operation: Unable to parse ExceptionName: BadRequest Message: malformed chunked encoding [ FAILED ] TestS3FS.GetFileInfoBucket (52 ms) ``` all due to `malformed chunked encoding` This also shows up in the python tests (these are from arrow v15, which is the last version where we built the python bindings on the same feedstock; since the errors are uniform across versions though, I strongly suspect this to be the case also for main, as well as v19 and down). ``` =========================== short test summary info ============================ FAILED pyarrow/tests/test_fs.py::test_filesystem_is_functional_after_pickling[builtin_pickle-S3FileSystem] - OSError: When creating key 'a/' in bucket 'pyarrow-filesystem': AWS Error UNKNOWN (HTTP status 400) during PutObject operation: Unable to parse ExceptionName: BadRequest Message: malformed chunked encoding FAILED pyarrow/tests/test_fs.py::test_filesystem_is_functional_after_pickling[cloudpickle-S3FileSystem] - OSError: When creating key 'a/' in bucket 'pyarrow-filesystem': AWS Error UNKNOWN (HTTP status 400) during PutObject operation: Unable to parse ExceptionName: BadRequest Message: malformed chunked encoding FAILED pyarrow/tests/test_dataset.py::test_write_dataset_s3 - OSError: When creating key 'dataset/' in bucket 'mybucket': AWS Error UNKNOWN (HTTP status 400) during PutObject operation: Unable to parse ExceptionName: BadRequest Message: malformed chunked encoding FAILED pyarrow/tests/test_fs.py::test_create_dir[S3FileSystem] - OSError: When creating key 'test-directory/' in bucket 'pyarrow-filesystem': AWS Error UNKNOWN (HTTP status 400) during PutObject operation: Unable to parse ExceptionName: BadRequest Message: malformed chunked encoding FAILED pyarrow/tests/test_fs.py::test_copy_file[S3FileSystem] - OSError: When uploading part for key 'test-copy-source-file' in bucket 'pyarrow-filesystem': AWS Error UNKNOWN (HTTP status 400) during UploadPart operation: Unable to parse ExceptionName: BadRequest Message: malformed chunked encoding FAILED pyarrow/tests/test_fs.py::test_move_file[S3FileSystem] - OSError: When uploading part for key 'test-move-source-file' in bucket 'pyarrow-filesystem': AWS Error UNKNOWN (HTTP status 400) during UploadPart operation: Unable to parse ExceptionName: BadRequest Message: malformed chunked encoding FAILED pyarrow/tests/test_fs.py::test_delete_file[S3FileSystem] - OSError: When uploading part for key 'test-delete-target-file' in bucket 'pyarrow-filesystem': AWS Error UNKNOWN (HTTP status 400) during UploadPart operation: Unable to parse ExceptionName: BadRequest Message: malformed chunked encoding FAILED pyarrow/tests/test_fs.py::test_filesystem_from_uri_s3 - OSError: When creating key 'foo/' in bucket 'mybucket': AWS Error UNKNOWN (HTTP status 400) during PutObject operation: Unable to parse ExceptionName: BadRequest Message: malformed chunked encoding ``` Xref also #45195 ### Component(s) C++, Python, Packaging -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
