Weston Pace created ARROW-11228:
-----------------------------------
Summary: [C++] Allow find tuning of memory pool from environment
variable(s)
Key: ARROW-11228
URL: https://issues.apache.org/jira/browse/ARROW-11228
Project: Apache Arrow
Issue Type: Improvement
Reporter: Weston Pace
Both jemalloc and mimalloc allow for some amount of fine tuning from
environment variables. However, in some cases, the settings entered into these
environment variables are overridden by arrow.
For example, jemalloc docs describe MALLOC_CONF, an environment variable that
can be used to tune any jemalloc parameter. But, the global variable
`malloc_conf` (which arrow uses) takes precedence.
Mimalloc also exposes some tuning parameters
([https://microsoft.github.io/mimalloc/environment.html)]. I have not tested
which of these work and which do not work so it may be that all of them are
accessible already.
Tuning may be useful in certain environments (for example, if someone wanted to
use jemalloc over mimalloc in an environment where overcommit is disabled and
so they wanted to set the retain:true option) and so it would be good to allow
for these tuning parameters to be set at runtime. One potential solution may
be to expose our own environment variable (e.g. ARROW_MALLOC_CONF).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
