LiaCastaneda commented on issue #8938: URL: https://github.com/apache/arrow-rs/issues/8938#issuecomment-3599000335
You're right, using this approach we would need to create a new kernel function to use the memory pool which doesn't feel super ergonomic. The ultimately goal is to be able keep track of the size of each Array created. fwiw, I looked into how Arrow Go and Arrow cpp handle this, and it seems to pass a context object everywhere ([cpp example](https://github.com/apache/arrow/blob/618090843fdb199ea86a1febb1e6e4f36cf35038/cpp/src/arrow/compute/kernel.h#L93)), and inside the context is the memory pool/allocator. Maybe we could do something similar in arrow? although I imagine that requires a considerable refactor and still wouldn’t resolve the “don’t rely on passing an extra parameter everywhere” concern -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
