[
https://issues.apache.org/jira/browse/ARROW-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17658155#comment-17658155
]
Rok Mihevc commented on ARROW-1123:
-----------------------------------
This issue has been migrated to [issue
#16734|https://github.com/apache/arrow/issues/16734] on GitHub. Please see the
[migration documentation|https://github.com/apache/arrow/issues/14542] for
further details.
> C++: Make jemalloc the default allocator
> ----------------------------------------
>
> Key: ARROW-1123
> URL: https://issues.apache.org/jira/browse/ARROW-1123
> Project: Apache Arrow
> Issue Type: New Feature
> Components: C++
> Reporter: Uwe Korn
> Assignee: Uwe Korn
> Priority: Major
> Fix For: 0.5.0
>
>
> While one of the advertised point of Arrow Java is that it comes with an
> efficient allocator that is based on jemalloc, we only provide jemalloc as an
> alternative allocator in C++. I would propose to switch to it as the default
> allocator on all platforms jemalloc is available (i.e. MacOS and Linux) as it
> provides the following benefits:
> * in-place resizing while ensuring 64byte alignment, for allocations >4kb
> this won't touch the data but rather only changes some pointers in kernel
> internal structs pointing to the respecitive memory pages.
> * better allocation algorithm, see
> https://www.facebook.com/notes/facebook-engineering/scalable-memory-allocation-using-jemalloc/480222803919/
> that should lead to less fragmentation and support fast multithreaded
> allocations
> Especially for the builders, simply changing the allocator shows speedups
> between 2x and 10x. In the optimal case, this puts builders nearly on the
> same level as if one would have pre-allocated the final size.
>
> This change should also include a blog post that outlines the benefits.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
