Issue 243406
Summary [OpenMP][libomp] The dependency hash of an implicit task leaks every time a hot team grows
Labels
Assignees
Reporter ACSimon33
    ## Summary

Tasks with `depend` clauses created directly from an implicit task, e.g. in a `master` or `single` region, attach a dependency hash to that implicit task ([kmp_taskdeps.cpp:702-703](https://github.com/llvm/llvm-project/blob/319bd32d5276bb9

936db4d8f80b66b507a2aa681/openmp/runtime/src/kmp_taskdeps.cpp#L702-L703) on `main` at 319bd32d). Its entries are freed at the end of each region ([kmp_tasking.cpp:1108-1122](https://github.com/llvm/llvm-project/blob/319bd32d5276bb9936db4d8f80b66b507a2aa681/openmp/runtime/src/kmp_tasking.cpp#L1108-L1122), or the walk-up at [lines 720-734](https://github.com/llvm/llvm-project/blob/319bd32d5276bb9936db4d8f80b66b507a2aa681/openmp/runtime/src/kmp_tasking.cpp#L720-L734)). The table itself is kept for the next region and is freed only in `__kmp_free_implicit_task()` ([lines 1134-1136](https://github.com/llvm/llvm-project/blob/319bd32d5276bb9936db4d8f80b66b507a2aa681/openmp/runtime/src/kmp_tasking.cpp#L1134-L1136)).

When a hot team grows, `__kmp_allocate_team()` calls `__kmp_initialize_info()` again for every thread of the team, the primary thread included ([kmp_runtime.cpp:5409-5412](https://github.com/llvm/llvm-project/blob/319bd32d5276bb9936db4d8f80b66b507a2aa681/openmp/runtime/src/kmp_runtime.cpp#L5409-L5412)). That calls `__kmp_init_implicit_task(..., TRUE)` ([line 4303](https://github.com/llvm/llvm-project/blob/319bd32d5276bb9936db4d8f80b66b507a2aa681/openmp/runtime/src/kmp_runtime.cpp#L4303)), which sets `td_dephash = NULL` without freeing the table ([kmp_tasking.cpp:1078](https://github.com/llvm/llvm-project/blob/319bd32d5276bb9936db4d8f80b66b507a2aa681/openmp/runtime/src/kmp_tasking.cpp#L1078)). Each growth leaks one table, about 8 KB. A program that alternates between two team sizes and creates dependent tasks in a `master` region therefore loses about 8 KB every second region. The application I found this in grew by about 3 MB/s.

## Reproducer

[`libomp_dephash_leak_on_grow.c`](https://github.com/user-attachments/files/34923231/libomp_dephash_leak_on_grow.c)

The team size alternates between 2 and 4. In each region the primary thread creates 64 independent tasks with `depend(inout)` on distinct addresses. The program prints the maximum RSS every 20000 regions.

```
clang -O1 -g -fopenmp libomp_dephash_leak_on_grow.c -o repro
./repro 200000
```

Observed (Apple M1 Pro, macOS 27.2), maximum RSS after 20000 / 100000 / 200000 regions:

| runtime                          | 20000    | 100000   | 200000 |
|----------------------------------|----------|----------|----------|
| `main` at 319bd32d (Release)     | 82.8 MB  | 404.1 MB | 805.7 MB |
| Homebrew libomp 23.1.4           | 82.8 MB  | 404.0 MB | 805.6 MB |
| `main` at 319bd32d + patch below | 2.5 MB   | 2.6 MB   | 2.6 MB   |

That is (805.7 - 82.8) MB / 90000 growths, i.e. 8.0 KB per growth. With a constant team size, or with the tasks inside an explicit task (`task if(0)` + `taskwait`), memory does not grow.

## Proposed fix

Free a table that is still attached before the pointer is cleared:

```diff
--- a/openmp/runtime/src/kmp_tasking.cpp
+++ b/openmp/runtime/src/kmp_tasking.cpp
@@ -1075,6 +1075,13 @@
     // Not used: don't need to deallocate implicit task
 KMP_ATOMIC_ST_REL(&task->td_allocated_child_tasks, 0);
 task->td_taskgroup = NULL; // An implicit task does not have taskgroup
+ // A hot team that grows reinitializes the implicit tasks of all its
+    // threads, so a dependency hash of an earlier region may still be attached.
+ // This runs on the thread that forks the team, so free with its own
+ // kmp_info_t.
+    if (task->td_dephash) {
+ __kmp_dephash_free(__kmp_threads[__kmp_get_gtid()], task->td_dephash);
+ }
     task->td_dephash = NULL;
 __kmp_push_current_task_to_thread(this_thr, team, tid);
   } else {
```

`__kmp_init_implicit_task()` is called by the forking thread for the other threads' implicit tasks as well. So the free uses the calling thread's `kmp_info_t` rather than `this_thr`, and `__kmp_fast_free()` returns blocks owned by another thread through its cross-thread list. The table's entries have already been freed by the previous region, whose join barrier the forking thread has passed. Fresh implicit task arrays come from `__kmp_allocate()` and are zeroed, so the branch is taken only for a table that is really attached.

An alternative is to keep the table across the reinitialization instead of freeing it, which saves a free and an allocation per growth. Until a fix is available, creating the dependent tasks inside an explicit task (`task if(0)` + `taskwait`) avoids the leak, because the hash then belongs to that task.

## Related issues

I found no existing report. The reproducer also runs into #226812, since it creates dependent tasks from the implicit task, and into the stale-worker crash (#XXXXX), since the team shrinks. Its tasks release no successors, which keeps the latter from crashing it.

## Environment

LLVM `main` at 319bd32d built from source (Release) and Homebrew libomp 23.1.4; Apple clang 21.0.0; macOS 27.2, Apple M1 Pro (arm64).

_______________________________________________
llvm-bugs mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs

Reply via email to