tustvold commented on PR #6794: URL: https://github.com/apache/arrow-rs/pull/6794#issuecomment-2508919453
So I think there is an less intrusive way to support this. UploadPart is not an issue as the per-part meta is never exposed. The issue solely concerns MultipartStore, in particular how to carry the checksum through https://docs.rs/object_store/latest/object_store/multipart/struct.PartId.html. Now we could add a checksum field to this, but this would be a breaking change. However, the content id is intentionally opaque. We could therefore just use the XML encoded https://docs.aws.amazon.com/AmazonS3/latest/API/API_CompletedPart.html instead of an ETag when necessary. There would need to still be logic to handle an ETag based PartId, as these may have been stored in a database, but when checksums, but detecting this should be trivial e.g. by attempting to decode the XML. This would avoid needing to make any breaking changes, whilst also being future proof for any of the other checksum types -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
