comphead commented on PR #7356:
URL:
https://github.com/apache/datafusion-comet/pull/7356#issuecomment-6619746050
@andygrove @sunchao, following up on the jar size point: I dropped the
second library from the jar in 1c23fb5989. A `jemalloc` Maven profile now
builds Comet with jemalloc instead: `make release PROFILES="-Pspark-4.1
-Pjemalloc"`. The published jars are unchanged and keep the system allocator,
so users who never opt in download nothing extra. The runtime config, its
environment variable and the fallback between two libraries are gone, and with
them the config read timing question.
- `-Pjemalloc` passes `comet.jemalloc=true` to the tests, and
`CometSparkSessionExtensionsSuite` then asserts
`NativeBase.isFeatureEnabled("jemalloc")`. The `run-jemalloc-tests` label
builds the CI library with `--features jemalloc` and runs the Linux suites with
`-Pjemalloc`, so a run that did not load jemalloc fails.
- 64 KiB pages: users now build their own jemalloc jar, so the
build-from-source guide documents `JEMALLOC_SYS_WITH_LG_PAGE=16` for executors
on 64 KiB-page kernels. I didn't make the Makefile turn it on for every arm64
build, because that would also change jemalloc's behavior on 4 KiB kernels.
- The release and snapshot builds are back to one native build per
architecture, so they take no extra time.
- `make release-nogit` now passes `COMET_FEATURES`, which it ignored before.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
