LiaCastaneda opened a new issue, #8938: URL: https://github.com/apache/arrow-rs/issues/8938
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.** When tracking memory usage with memory pools, users must manually call `claim()` on buffers after array creation. It could be easy to forget to call `claim()` on newly created arrays, leading to potentially under-reporting memory usage, also it requires having direct access to `ArrayData` just to access buffers for claiming, which in some cases is complicated. It would be much more ergonomic and reliable to automatically claim buffers at array creation time. **Describe the solution you'd like** Add optional `MemoryPool` parameters to array constructors and builders, so buffers are automatically claimed at creation time. For example we could have: `PrimitiveArray:: try_new_with_pool()` in array costructors, `PrimitiveBuilder::new_with_pool` that claim buffers once we call `finish()` etc. **Additional context** This feature is needed to improve memory accounting and tracking in downstream projects like DataFusion. Related discussions: - https://github.com/apache/datafusion/issues/16841 Related epic in arrow: - https://github.com/apache/arrow-rs/issues/8137 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
