casaroli opened a new pull request, #21898: URL: https://github.com/apache/nuttx/pull/21898
## Summary This PR fixes two error paths of `exec_module()`. An error after `nxtask_init()` (`addrenv_attach()`, `addrenv_restore()` or `spawn_execattrs()`) goes to `errout_with_tcbinit`. #21711 fixed the second free of the TCB on this path. Three problems stay: - After `nxtask_uninit()`, the path goes on into the error path of the steps before `nxtask_init()`. It releases the argument list, the environment list and the file actions a second time. In a kernel build they are copies in the kernel heap. - It calls `addrenv_restore()` a second time after an error of `addrenv_restore()` or `spawn_execattrs()`, and so gives back a reference to the address environment of the caller. - In a kernel build, `nxtask_uninit()` frees the user stack of the new task with `kumm_free()` while the address environment of the caller is selected. The stack lies in the user heap of the new process, so the free goes into the heap of the caller. Now an error of `addrenv_attach()` goes to a new label, `errout_with_attach`, that restores the address environment of the caller. The errors of `addrenv_restore()` and `spawn_execattrs()` go to `errout_with_tcbinit`, where the address environment of the caller is already current. In a kernel build, `errout_with_tcbinit` keeps the user stack for the address environment of the new process. `group_leave()` also clears `TCB_FLAG_FREE_STACK` for an ending process, but `nxsched_release_tcb()` calls `up_release_stack()` before `group_leave()`, so that is too late on this path. Then the path calls `nxtask_uninit()` and returns. An error before `addrenv_attach()` (the allocation of the TCB, the copies of the lists, `nxtask_init()`, for example a file action of `posix_spawn()` that fails) did not release the address environment that `load_module()` made. `unload_module()` does not release it. So each such error lost the page tables, the segments and the heap of the program. Now the early error path releases it. A user program gets both paths in a kernel build: `posix_spawn()` with `POSIX_SPAWN_SETSCHEDPARAM` and priority 0 (late error), or with a file action that opens a file in a directory that does not exist (early error). ## Impact Kernel builds (`CONFIG_BUILD_KERNEL`): no double free in the kernel heap, no free into the heap of the caller, no second `addrenv_restore()`, and no lost address environment after a failed `exec()` or `posix_spawn()`. Builds without `CONFIG_BUILD_KERNEL` do not copy the lists and do not select the address environment of the new task: there the late error path now returns after `nxtask_uninit()` instead of calling functions that do nothing. A flat or protected build with `CONFIG_ARCH_ADDRENV` (for example sama5d4-ek:elf and fvp-armv8r:pnsh) now also releases the address environment of the program after an early error. The result and the errno of every call stay the same. No API change, no documentation change. ## Testing Test program `spawnlate.c` (7 cases: SETSCHEDPARAM priority 0, SCHED_FIFO below its range, late errors in 3 child processes that then check their memory, a failing file action, the page pool after 30 late and 30 early errors, a normal `posix_spawn()` at the end), qemu-armv8a:knsh (kernel build, HVF): - master fab614ba83d (no change in these files up to 46066440966): `Assertion failed : at file: mm_heap/mm_malloc_size.c:73` at the first late error, then a loop of `Assertion failed : at file: semaphore/sem_wait.c:138`. ostest does not run. - this PR (on 46066440966): `SPAWNLATE pass=7 fail=0`, page pool +0 KiB after 30 late and 30 early errors. ostest: `Exiting with status 0`. - first commit only: the page pool fills up after the early errors (posix_spawn() gives ENOMEM), ostest exits with status 256. ostest on qemu-armv8a:nsh (flat): `Exiting with status 0`. Compiled with 0 warnings: rv-virt:knsh64 (kernel, RISC-V), lm3s6965-ek:qemu-protected (protected) and sama5d4-ek:elf (flat with `CONFIG_ARCH_ADDRENV`). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
