casaroli opened a new pull request, #21016: URL: https://github.com/apache/nuttx/pull/21016
## Summary `pthread_atfork()` allocates its entries with `lib_malloc()` in the heap of the calling process: in a protected or kernel build that is the user heap. When the process exits, `task_uninit_info()` freed them with `lib_free()`, which is `kmm_free()` in the kernel. So the kernel gave user pointers to the kernel heap: with `CONFIG_DEBUG_ASSERTIONS` an assertion in `kmm_free()` for each entry, without it a corrupt kernel heap. This PR frees the entries with `group_free()`, which selects the heap of the group. In a kernel build it does not free the entries of an unprivileged group: they lie in its user heap, which goes away with its address environment. `task_uninit_stream()` does the same for the streams. ## Impact Only builds with `CONFIG_PTHREAD_ATFORK`. In a flat build without `CONFIG_MM_KERNEL_HEAP`, `group_free()` and `lib_free()` free in the same heap: no change. Protected and kernel builds no longer free user memory in the kernel heap. ## Testing Test program `atforkexit.c` on qemu-armv8a:knsh (QEMU with HVF; the defconfig has `CONFIG_DEBUG_ASSERTIONS`) with `CONFIG_PTHREAD_ATFORK`: it starts 5 child processes one after the other; each child registers 4 handlers with `pthread_atfork()` and exits. | | master | this PR | |---|---|---| | console | 20 times `Assertion failed : at file: kmm_heap/kmm_free.c:57 task: atforkexit` (one for each entry) | no assertion | | children exit with status 0 | 5 of 5 | 5 of 5 | ostest exits with status 0 on qemu-armv8a:knsh and qemu-armv8a:nsh with `CONFIG_PTHREAD_ATFORK`. Build with `CONFIG_PTHREAD_ATFORK`, each OK: qemu-armv8a:nsh (flat, arm64), qemu-armv8a:knsh (kernel, arm64), rv-virt:nsh64 (flat, rv64), rv-virt:pnsh64 (protected, rv64), rv-virt:knsh64 (kernel, rv64). lm3s6965-ek:qemu-protected (protected, armv7-m) does not link on master today: its user flash overflows (772 bytes with Arm GNU 13.2, as in CI). This PR does not change the user image: with `CONFIG_PTHREAD_ATFORK` and Arm GNU 15.3, master and this PR overflow by the same 1172 bytes. `tools/checkpatch.sh -c -u -m -g` passes. No documentation change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
