waynexia commented on PR #6336: URL: https://github.com/apache/arrow-rs/pull/6336#issuecomment-2373962336
>It seems like the new Array will use the default system allocator rather than the custom allocator which will limit the effectiveness of this approach Yes, it will be `Global` without additional code changes. This is the downside of this `Allocator` trait 😢 >Would it be possible with this API to support changing the global allocator for all new Arrays that are created? Still take `unkai` as an example, it also provides one global allocator wrapper for similar things [here](https://docs.rs/unkai/0.2.0/unkai/#use-with-globalalloc), which I think would be a common pattern if one wants global memory usage tracking. But on the other hand, the global style can't tell the usage per object (or per session, of course). I don't wish for the full support of `Allocator` type param in even one or two releases. But with this patch as the beginning, we can expect more cases to use it like in https://github.com/apache/arrow-rs/issues/6439. Or tracking & limiting memory consumption of one write buffer (a group of mutable buffers in GreptimeDB). >Would it be possible to open that followup PR already in draft mode? Here it is https://github.com/apache/arrow-rs/pull/6455, I only changed the `into_buffer()` which could demonstrate the conversion between mutable buffer to immutable buffer (with some new miri test). As for the type param, since we have our own `Bytes` definition with trait object, I think we don't need to add new param to them like in `MutableBuffer`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
