alamb commented on issue #5458: URL: https://github.com/apache/arrow-rs/issues/5458#issuecomment-1984683654
> Had a good sync with @alamb and I think we've devised a way to support the original vision of exposing a MultiPart API for stores, including LocalFilesystem. Apologies for the noise To avoid leaving anyone in suspense, as I recall the baisic idea is to require, for file backed object stores, that each multi-part upload is the same size (except for the last one). In this way, when writing multiple "parts" to a file, we can calculate apriori what offsets in the file each part will go. If a user tries to upload a part that is not the required size, the API will error with a clear message https://docs.rs/object_store/latest/object_store/multipart/trait.MultiPartStore.html#tymethod.put_part I may be misremembering this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
