Yes, byte_size.h is what I was looking for, thanks.
Yaron. ________________________________ From: David Li <lidav...@apache.org> Sent: Friday, October 28, 2022 9:52 AM To: dev@arrow.apache.org <dev@arrow.apache.org> Subject: Re: measuring memory usage of Arrow structures This would also be a good cookbook example, so I filed [1]. (Sorry for the double-email.) [1]: https://github.com/apache/arrow-cookbook/issues/273 On Fri, Oct 28, 2022, at 09:51, David Li wrote: > I think Yaron is looking more for the APIs in arrow/util/byte_size.h > [1] which has utilities for estimating the size of the buffers in an > array or record batch. (I'm assuming you're not concerned with the size > of the C++ structures themselves.) > > [1]: > https://github.com/apache/arrow/blob/master/cpp/src/arrow/util/byte_size.h > > On Fri, Oct 28, 2022, at 09:40, Rok Mihevc wrote: >> Hey Yaron, >> >> If you're using jemalloc you can use jemalloc_get_stat [1] to monitor total >> memory allocation. Another option would be LoggingMemoryPool, see tests on >> possible usage [2]. >> >> Rok >> >> [1] >> https://github.com/apache/arrow/blob/master/cpp/src/arrow/memory_pool_jemalloc.cc#L157 >> [2] >> https://github.com/apache/arrow/blob/master/cpp/src/arrow/memory_pool_test.cc >> >> On Fri, Oct 28, 2022 at 3:10 PM Yaron Gvili <rt...@hotmail.com> wrote: >> >>> Hi, >>> >>> Is there a supported/convenient way for measuring the memory usage of >>> Arrow structures? For my specific use case, measuring memory usage of >>> either a record batch or an array would be sufficiently convenient. >>> >>> >>> Cheers, >>> Yaron. >>>
