I can't find it anymore, but there is a quite old issue that made the same observation: RecordBatch's heavy use of shared_ptr in C++ can lead to a lot of overhead just calling destructors. That may be something to explore more (e.g. I think someone had tried to "unbox" some of the fields in RecordBatch).
On Fri, May 12, 2023, at 13:04, Will Jones wrote: > Hello, > > I'm not sure if there are easy ways to avoid calling the destructors. > However, I would point out memory space reuse is handled through memory > pools; if you have one enabled it shouldn't be handing memory back to the > OS between each iteration. > > Best, > > Will Jones > > On Fri, May 12, 2023 at 9:59 AM SHI BEI <shibei...@foxmail.com> wrote: > >> Hi community, >> >> >> I'm using the RecordBatchReader::ReadNext interface to read Parquet >> data in my project, and I've noticed that there are a lot of temporary >> object destructors being generated during usage. Has the community >> considered providing an interface to reuse RecordBatch objects >> and their memory space for storing data? >> >> >> >> >> SHI BEI >> shibei...@foxmail.com
