| Issue |
243406
|
| Summary |
[OpenMP][libomp] The dependency hash of an implicit task leaks every time a hot team grows
|
| Labels |
|
| Assignees |
|
| Reporter |
ACSimon33
|
## Summary
Tasks with `depend` clauses created directly from an implicit task, e.g. in a `master` or `single` region, attach a dependency hash to that implicit task ([kmp_taskdeps.cpp:702-703](https://github.com/llvm/llvm-project/blob/319bd32d5276bb9
936db4d8f80b66b507a2aa681/openmp/runtime/src/kmp_taskdeps.cpp#L702-L703) on `main` at 319bd32d). Its entries are freed at the end of each region ([kmp_tasking.cpp:1108-1122](https://github.com/llvm/llvm-project/blob/319bd32d5276bb9936db4d8f80b66b507a2aa681/openmp/runtime/src/kmp_tasking.cpp#L1108-L1122), or the walk-up at [lines 720-734](https://github.com/llvm/llvm-project/blob/319bd32d5276bb9936db4d8f80b66b507a2aa681/openmp/runtime/src/kmp_tasking.cpp#L720-L734)). The table itself is kept for the next region and is freed only in `__kmp_free_implicit_task()` ([lines 1134-1136](https://github.com/llvm/llvm-project/blob/319bd32d5276bb9936db4d8f80b66b507a2aa681/openmp/runtime/src/kmp_tasking.cpp#L1134-L1136)).
When a hot team grows, `__kmp_allocate_team()` calls `__kmp_initialize_info()` again for every thread of the team, the primary thread included ([kmp_runtime.cpp:5409-5412](https://github.com/llvm/llvm-project/blob/319bd32d5276bb9936db4d8f80b66b507a2aa681/openmp/runtime/src/kmp_runtime.cpp#L5409-L5412)). That calls `__kmp_init_implicit_task(..., TRUE)` ([line 4303](https://github.com/llvm/llvm-project/blob/319bd32d5276bb9936db4d8f80b66b507a2aa681/openmp/runtime/src/kmp_runtime.cpp#L4303)), which sets `td_dephash = NULL` without freeing the table ([kmp_tasking.cpp:1078](https://github.com/llvm/llvm-project/blob/319bd32d5276bb9936db4d8f80b66b507a2aa681/openmp/runtime/src/kmp_tasking.cpp#L1078)). Each growth leaks one table, about 8 KB. A program that alternates between two team sizes and creates dependent tasks in a `master` region therefore loses about 8 KB every second region. The application I found this in grew by about 3 MB/s.
## Reproducer
[`libomp_dephash_leak_on_grow.c`](https://github.com/user-attachments/files/34923231/libomp_dephash_leak_on_grow.c)
The team size alternates between 2 and 4. In each region the primary thread creates 64 independent tasks with `depend(inout)` on distinct addresses. The program prints the maximum RSS every 20000 regions.
```
clang -O1 -g -fopenmp libomp_dephash_leak_on_grow.c -o repro
./repro 200000
```
Observed (Apple M1 Pro, macOS 27.2), maximum RSS after 20000 / 100000 / 200000 regions:
| runtime | 20000 | 100000 | 200000 |
|----------------------------------|----------|----------|----------|
| `main` at 319bd32d (Release) | 82.8 MB | 404.1 MB | 805.7 MB |
| Homebrew libomp 23.1.4 | 82.8 MB | 404.0 MB | 805.6 MB |
| `main` at 319bd32d + patch below | 2.5 MB | 2.6 MB | 2.6 MB |
That is (805.7 - 82.8) MB / 90000 growths, i.e. 8.0 KB per growth. With a constant team size, or with the tasks inside an explicit task (`task if(0)` + `taskwait`), memory does not grow.
## Proposed fix
Free a table that is still attached before the pointer is cleared:
```diff
--- a/openmp/runtime/src/kmp_tasking.cpp
+++ b/openmp/runtime/src/kmp_tasking.cpp
@@ -1075,6 +1075,13 @@
// Not used: don't need to deallocate implicit task
KMP_ATOMIC_ST_REL(&task->td_allocated_child_tasks, 0);
task->td_taskgroup = NULL; // An implicit task does not have taskgroup
+ // A hot team that grows reinitializes the implicit tasks of all its
+ // threads, so a dependency hash of an earlier region may still be attached.
+ // This runs on the thread that forks the team, so free with its own
+ // kmp_info_t.
+ if (task->td_dephash) {
+ __kmp_dephash_free(__kmp_threads[__kmp_get_gtid()], task->td_dephash);
+ }
task->td_dephash = NULL;
__kmp_push_current_task_to_thread(this_thr, team, tid);
} else {
```
`__kmp_init_implicit_task()` is called by the forking thread for the other threads' implicit tasks as well. So the free uses the calling thread's `kmp_info_t` rather than `this_thr`, and `__kmp_fast_free()` returns blocks owned by another thread through its cross-thread list. The table's entries have already been freed by the previous region, whose join barrier the forking thread has passed. Fresh implicit task arrays come from `__kmp_allocate()` and are zeroed, so the branch is taken only for a table that is really attached.
An alternative is to keep the table across the reinitialization instead of freeing it, which saves a free and an allocation per growth. Until a fix is available, creating the dependent tasks inside an explicit task (`task if(0)` + `taskwait`) avoids the leak, because the hash then belongs to that task.
## Related issues
I found no existing report. The reproducer also runs into #226812, since it creates dependent tasks from the implicit task, and into the stale-worker crash (#XXXXX), since the team shrinks. Its tasks release no successors, which keeps the latter from crashing it.
## Environment
LLVM `main` at 319bd32d built from source (Release) and Homebrew libomp 23.1.4; Apple clang 21.0.0; macOS 27.2, Apple M1 Pro (arm64).
_______________________________________________
llvm-bugs mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs