XiaoHongbo-Hope opened a new pull request, #10456: URL: https://github.com/apache/paimon/pull/10456
### Purpose Split the deletion compatibility fix from #9821. PyArrow 22/23 `DeleteObjects` fails on real OSS even with `WHEN_REQUIRED`, because this operation requires a checksum. Use Boto3 batches of up to 1,000 keys with Content-MD5 for PyArrow-backed OSS/custom-S3 directory deletion. Delete non-printable keys individually to avoid XML corruption. Share S3 connection options with Arrow and recreate the deletion client after deserialization. Collect keys before deletion, validate bucket/prefix and batch results, and retain schema files until the remaining-content check passes. This supports failure retries but does not make concurrent deletion atomic. PyArrow dependency limits and Catalog logic are unchanged. ### Validation - PyArrow 19: **101 passed, 7 skipped**; 22 and 23: **108 passed each**. Includes cross-bucket/list-status paths, partial failures, pagination, timeouts and schema-only retries. - Real OSS: native 22/23 deletion failed with HTTP 400; the fallback passed with Boto3 1.36/1.43, including `\r`, `\x01` and XML-special keys. Adjacent objects survived; test data was cleaned up. - Added Arrow 22/23 CI. Flake8, Python 3.6 syntax and diff checks pass. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
