[
https://issues.apache.org/jira/browse/ARROW-464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17657498#comment-17657498
]
Rok Mihevc commented on ARROW-464:
----------------------------------
This issue has been migrated to [issue
#16108|https://github.com/apache/arrow/issues/16108] on GitHub. Please see the
[migration documentation|https://github.com/apache/arrow/issues/14542] for
further details.
> C++: More intelligent array growing
> -----------------------------------
>
> Key: ARROW-464
> URL: https://issues.apache.org/jira/browse/ARROW-464
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
> Reporter: Uwe Korn
> Assignee: Uwe Korn
> Priority: Major
>
> Three things to consider:
> * Instead of always growing the memory to twice the size in the Builders, we
> should consider 1.5 as a growth factor as this still leads to less memory
> wastage. As the allocation costly is mostly linear in the number of newly
> requested pages, this shouldn't be a noticeable impact. Due to memory below
> the size of a single page (i.e. 4KiB) not being able to be expanded in place,
> we should keep the factor of there.
> * In the case of jemalloc, we can ask the allocator with the function
> {{nallocx}} for the size it would actually allocate for the requested size
> and then allocate this as the new capacity for the Builder/Buffer/... and not
> trigger an reallocation as long as the output of {{nallocx}} doesn't change.
> * If not using {{jemalloc}}, we should be careful about the cost of minor
> allocation size changes. It would be preferable in these cases to have also
> an implementation along the lines of {{nallocx}} that does some rounding to
> avoid unnecessary reallocations.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
