This is an automated email from the ASF dual-hosted git repository.
xiaoxiang781216 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/nuttx.git
The following commit(s) were added to refs/heads/master by this push:
new 69bae44b10e sched/group: Do not free the stacks of an ending process
into its heap.
69bae44b10e is described below
commit 69bae44b10e75a810d4194a00e934d57b55c7992
Author: Marco Casaroli <[email protected]>
AuthorDate: Thu Oct 29 19:17:50 2026 +0100
sched/group: Do not free the stacks of an ending process into its heap.
In the kernel build the stack of a thread of an unprivileged group is a
block of the user heap of the group. When a thread ends,
nxsched_release_tcb() frees it with up_release_stack(), which calls
kumm_free(): the kernel then works on the heap structures of the
program. When the whole process ends, that heap goes away with the
address environment anyway, and the program can have overwritten it: a
stack overflow runs down through the heap, over the heap structure at
its start. Seen with a thread that overflows its stack: the fatal
signal ends the process, and nxsig_terminate() -> nx_pthread_exit() ->
nxtask_exit() -> nxsched_release_tcb() -> up_release_stack() ->
mm_free() fails DEBUGASSERT(mm_heapmember()) on the broken heap.
When a user thread leaves its group, and the group ends (it is the
last member, or group_kill_children() set GROUP_FLAG_EXITING), clear
TCB_FLAG_FREE_STACK: every architecture frees the stack only when that
flag is set, and still clears the stack fields of the TCB. This is
done in group_leave(), where the group is known: when the TCB is
released, it can have left the group already. The stack of a thread
that ends alone (pthread_exit()) is freed as before.
A vfork() child (GROUP_FLAG_VFORK) keeps the flag: its stack is in the
heap of its parent, which stays. Without this, each vfork() lost a
block of the size of the stack of the parent in the heap of the parent.
When another process ends the child, group_leave() clears both flags:
then the heap of the parent is not the current heap.
Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Marco Casaroli <[email protected]>
---
sched/group/group_leave.c | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/sched/group/group_leave.c b/sched/group/group_leave.c
index 2138839385d..6d5dd0e83ca 100644
--- a/sched/group/group_leave.c
+++ b/sched/group/group_leave.c
@@ -237,6 +237,21 @@ void group_leave(FAR struct tcb_s *tcb)
/* Have all of the members left the group? */
#endif
+#ifdef CONFIG_BUILD_KERNEL
+ /* The heap of an ending user group goes away with its address
+ * environment, and a stack overflow can break it: do not free the
+ * stack into it. The heap of a vfork() child stays.
+ */
+
+ if ((tcb->flags & TCB_FLAG_TTYPE_MASK) != TCB_FLAG_TTYPE_KERNEL &&
+ (group->tg_flags &
+ (GROUP_FLAG_PRIVILEGED | GROUP_FLAG_VFORK)) == 0 &&
+ (release || (group->tg_flags & GROUP_FLAG_EXITING) != 0))
+ {
+ tcb->flags &= ~TCB_FLAG_FREE_STACK;
+ }
+#endif
+
if (release)
{
/* Yes.. Release all of the resource held by the task group */
@@ -251,6 +266,7 @@ void group_leave(FAR struct tcb_s *tcb)
tcb->addrenv_own != this_task()->addrenv_curr))
{
group->tg_flags &= ~GROUP_FLAG_VFORK;
+ tcb->flags &= ~TCB_FLAG_FREE_STACK;
}
#endif