luomh0910 opened a new pull request, #13618:
URL: https://github.com/apache/gluten/pull/13618
## What changes are proposed in this pull request?
Fixes #13617
Wrap global off-heap reservations with the existing
`DynamicOffHeapSizingMemoryTarget` so they participate in shared off-heap
accounting, memory-budget checks, and JVM heap shrinking. This includes
global reservations used by Velox broadcast hash-table construction and
the Arrow global allocator.
Add `GlutenCoreConfig.isDynamicOffHeapSizingEnabled` to resolve the flag
from `SparkEnv` when available, falling back to the existing config getter
otherwise. Use this lookup consistently for dynamic-target wrapping,
task/global memory-pool selection, and available-memory reporting.
Determine the global target's memory mode at construction and keep it
stable across borrowing and releasing threads.
Expose the existing shared-counter testing hook and add focused tests
for global accounting and configuration lookup.
This patch reuses the existing dynamic-sizing and JVM heap shrink logic;
it does not change the shrink algorithm or its thresholds.
## How was this patch tested?
Focused JVM tests were compiled and run directly with JDK 17 and local
Spark 3.4.4 dependencies:
- `GlobalOffHeapDynamicSizingSuite`: all 4 tests passed, covering enabled
global accounting and release, disabled-mode storage accounting,
rejected reservations, and SparkEnv configuration precedence.
- `DynamicOffHeapSizingMemoryTargetTest`: all 10 existing tests passed.
## Was this patch authored or co-authored using generative AI tooling?
Generated-by: Codex GPT-6-Astra
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]