abhiaagarwal opened a new issue, #5899: URL: https://github.com/apache/arrow-rs/issues/5899
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.** Per the documentation for `MultipartUpload`, to maximize compatibility with all object stores, the uploaded payload should have a fixed length until the last payload. This is achievable with a `PayloadMut`. However, it's a bit wordy to handle the logic for making sure each payload is a fixed size. `WriteMultipart` already has an implementation of this logic, but it would be nice to pull that logic out and make it into own struct. For an API design, I imagine something like `FixedPayloadMut`, where you can constantly extend data inside of it, and you can pull a chunk of it out calling some method that returns `Some(Payload)` with the specified fixed length if it has that much data or `None` otherwise. Or, maybe the `push` method potentially returns with a `Some(Payload)` if the internal buffer gets filled (or a vec???). To pull out the final chunk of the data that is <= fixed length, there should be a `finish` method of some sort. All of this should be zero-copy as well. A ringbuffer might be a good idea. **Describe the solution you'd like** <!-- A clear and concise description of what you want to happen. --> Pull out the logic used in `WriteMultipart` to its own struct. `WriteMultipart` is a nice struct, but it isn't sufficient you need finer control over when to send data. **Describe alternatives you've considered** <!-- A clear and concise description of any alternative solutions or features you've considered. --> This doesn't necessarily need to be part of the object_store crate, but it would be nice to have one canonical representation. **Additional context** I can submit a PR with the design if y'all are interested in the idea? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
