comphead commented on code in PR #7356: URL: https://github.com/apache/datafusion-comet/pull/7356#discussion_r4570104711
########## docs/source/user-guide/latest/source.md: ########## @@ -78,3 +78,32 @@ To build Comet from the source distribution on an isolated environment without a ```console make release-nogit PROFILES="-Pspark-4.1" ``` + +## Building with the jemalloc allocator + +The native library allocates its memory with the system allocator, which is glibc malloc on Linux. +In a long-running executor, glibc can keep several GiB of freed native memory resident (see +[#7108](https://github.com/apache/datafusion-comet/issues/7108)). The published jars use the system +allocator. To build Comet with [jemalloc](https://jemalloc.net/) instead, add the `jemalloc` profile: + +```console +make release PROFILES="-Pspark-4.1 -Pjemalloc" +``` + +`make release-nogit` accepts it too. The profile builds the native library with its `jemalloc` Cargo Review Comment: Done in 0846d92248. The paragraph now says that the profile has no settings of its own and that `make` adds the `jemalloc` Cargo feature when `PROFILES` names it. It also says that a library built with a plain `cargo build` and then packaged with `./mvnw install -Pjemalloc` uses the system allocator without any warning, and that `--features jemalloc` belongs on the `cargo build` in that case. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
