[jira] [Commented] (ARROW-12316) [C++] Switch default memory allocator from jemalloc to mimalloc
[ https://issues.apache.org/jira/browse/ARROW-12316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320307#comment-17320307 ] Antoine Pitrou commented on ARROW-12316: It would be nice if the benchmark results could be posted publicly somewhere, so that they can be referenced in the PR and commit description. > [C++] Switch default memory allocator from jemalloc to mimalloc > --- > > Key: ARROW-12316 > URL: https://issues.apache.org/jira/browse/ARROW-12316 > Project: Apache Arrow > Issue Type: New Feature > Components: C++ >Reporter: Neal Richardson >Priority: Major > Fix For: 4.0.0 > > > Benchmarking shows that mimalloc seems to be faster on real workflows (at > least on macOS, still collecting data on Ubuntu). We could switch the default > memory pool cases so that mimalloc is preferred. > cc [~jonkeane] [~apitrou] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-12316) [C++] Switch default memory allocator from jemalloc to mimalloc
[ https://issues.apache.org/jira/browse/ARROW-12316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320299#comment-17320299 ] Neal Richardson commented on ARROW-12316: - The idea we landed on was to make the default behavior conditional on platform. On macOS, we should prefer mimalloc if available; elsewhere we prefer jemalloc. This makes sense given that we had to do some special handling of jemalloc settings on macOS, and it is reasonable to believe that those settings have negative effects on performance. > [C++] Switch default memory allocator from jemalloc to mimalloc > --- > > Key: ARROW-12316 > URL: https://issues.apache.org/jira/browse/ARROW-12316 > Project: Apache Arrow > Issue Type: New Feature > Components: C++ >Reporter: Neal Richardson >Priority: Major > Fix For: 4.0.0 > > > Benchmarking shows that mimalloc seems to be faster on real workflows (at > least on macOS, still collecting data on Ubuntu). We could switch the default > memory pool cases so that mimalloc is preferred. > cc [~jonkeane] [~apitrou] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-12316) [C++] Switch default memory allocator from jemalloc to mimalloc
[ https://issues.apache.org/jira/browse/ARROW-12316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320247#comment-17320247 ] Antoine Pitrou commented on ARROW-12316: Also cc [~uwe] > [C++] Switch default memory allocator from jemalloc to mimalloc > --- > > Key: ARROW-12316 > URL: https://issues.apache.org/jira/browse/ARROW-12316 > Project: Apache Arrow > Issue Type: New Feature > Components: C++ >Reporter: Neal Richardson >Priority: Major > Fix For: 4.0.0 > > > Benchmarking shows that mimalloc seems to be faster on real workflows (at > least on macOS, still collecting data on Ubuntu). We could switch the default > memory pool cases so that mimalloc is preferred. > cc [~jonkeane] [~apitrou] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-12316) [C++] Switch default memory allocator from jemalloc to mimalloc
[ https://issues.apache.org/jira/browse/ARROW-12316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320246#comment-17320246 ] Antoine Pitrou commented on ARROW-12316: Feel free to post your final decision here. > [C++] Switch default memory allocator from jemalloc to mimalloc > --- > > Key: ARROW-12316 > URL: https://issues.apache.org/jira/browse/ARROW-12316 > Project: Apache Arrow > Issue Type: New Feature > Components: C++ >Reporter: Neal Richardson >Priority: Major > Fix For: 4.0.0 > > > Benchmarking shows that mimalloc seems to be faster on real workflows (at > least on macOS, still collecting data on Ubuntu). We could switch the default > memory pool cases so that mimalloc is preferred. > cc [~jonkeane] [~apitrou] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-12316) [C++] Switch default memory allocator from jemalloc to mimalloc
[ https://issues.apache.org/jira/browse/ARROW-12316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17319520#comment-17319520 ] Jonathan Keane commented on ARROW-12316: I've attached the two reports I've compiled. The first (unmarked, memory_allocators.html) is on macOS, the second (marked ubuntu) is on ubuntu. > [C++] Switch default memory allocator from jemalloc to mimalloc > --- > > Key: ARROW-12316 > URL: https://issues.apache.org/jira/browse/ARROW-12316 > Project: Apache Arrow > Issue Type: New Feature > Components: C++ >Reporter: Neal Richardson >Priority: Major > Fix For: 4.0.0 > > > Benchmarking shows that mimalloc seems to be faster on real workflows (at > least on macOS, still collecting data on Ubuntu). We could switch the default > memory pool cases so that mimalloc is preferred. > cc [~jonkeane] [~apitrou] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-12316) [C++] Switch default memory allocator from jemalloc to mimalloc
[ https://issues.apache.org/jira/browse/ARROW-12316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17318918#comment-17318918 ] Neal Richardson commented on ARROW-12316: - [~jonkeane] can you attach your reports? > [C++] Switch default memory allocator from jemalloc to mimalloc > --- > > Key: ARROW-12316 > URL: https://issues.apache.org/jira/browse/ARROW-12316 > Project: Apache Arrow > Issue Type: New Feature > Components: C++ >Reporter: Neal Richardson >Priority: Major > Fix For: 4.0.0 > > > Benchmarking shows that mimalloc seems to be faster on real workflows (at > least on macOS, still collecting data on Ubuntu). We could switch the default > memory pool cases so that mimalloc is preferred. > cc [~jonkeane] [~apitrou] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-12316) [C++] Switch default memory allocator from jemalloc to mimalloc
[ https://issues.apache.org/jira/browse/ARROW-12316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17318907#comment-17318907 ] Uwe Korn commented on ARROW-12316: -- [~npr] Where can I find these benchmarks? > [C++] Switch default memory allocator from jemalloc to mimalloc > --- > > Key: ARROW-12316 > URL: https://issues.apache.org/jira/browse/ARROW-12316 > Project: Apache Arrow > Issue Type: New Feature > Components: C++ >Reporter: Neal Richardson >Priority: Major > Fix For: 4.0.0 > > > Benchmarking shows that mimalloc seems to be faster on real workflows (at > least on macOS, still collecting data on Ubuntu). We could switch the default > memory pool cases so that mimalloc is preferred. > cc [~jonkeane] [~apitrou] -- This message was sent by Atlassian Jira (v8.3.4#803005)
