** Description changed:
- on 5.15 a seccomp filter is freed at reap (release_task()), not at exit.
- Each filter's JIT image is charged to net.core.bpf_jit_limit (half the
- module area: 512 MiB amd64, 64 MiB arm64). Unreaped zombies from
- short‑lived containerised processes keep pinning images until the limit
- is hit; then bpf_jit_charge_modmem() refuses unprivileged allocations
- and, because every Ubuntu kernel has CONFIG_BPF_JIT_ALWAYS_ON=y (no
- interpreter fallback — verified in debian.master/config/annotations),
- bpf_prog_select_runtime() returns -ENOTSUPP = 524. That's the kernel err
- loading seccomp filter errno 524: unknown error 524 on the k8s node:
- pods with a seccomp profile fail to start while everything else looks
- healthy. I verified this chain in the jammy source
- (kernel/bpf/core.c:846, :1980, include/linux/errno.h:27).
+ [ Impact ]
+
+ On Jammy (5.15) seccomp filters attached to a task are only released in
+ release_task(), i.e. when the zombie is *reaped*, not when the task
+ exits. Every seccomp filter is a classic-BPF program that is JIT
+ compiled, and the JIT image of every filter installed by an unprivileged
+ process is charged against the global net.core.bpf_jit_limit
+ (bpf_jit_charge_modmem()). As long as a zombie exists, its filter and
+ JIT image stay charged against that limit even though the task can
+ never execute another syscall.
+
+ On container hosts this turns into a leak: every process that exits
+ with a seccomp filter but is never reaped keeps one JIT image pinned.
+ This is the normal situation in containers whose PID 1 is not an init
+ (e.g. `busybox sleep`, a shell, or the application itself): orphaned
+ children of `docker exec` / kubelet exec probes are reparented to that
+ PID 1, which never calls wait(), so every probe invocation leaves a
+ zombie behind – and with it a pinned seccomp filter until it reaches
bpf_jit_limit (512MB in amd64).
+
+ Once the limit is reached, bpf_jit_charge_modmem() refuses any further
+ unprivileged allocation. Because all Ubuntu kernels are built with
+ CONFIG_BPF_JIT_ALWAYS_ON=y there is no interpreter fallback, so
+ bpf_prog_select_runtime() fails with -ENOTSUPP (524) and seccomp(2)
+ returns it to userspace. On a Kubernetes node this surfaces as new pods
+ that use a seccomp profile failing to
+ start with:
+
+ kernel err loading seccomp filter errno 524: unknown error 524
+ (runc / containerd-shim: "error loading seccomp filter into kernel")
+
+ while the node otherwise looks healthy. Pods that do not use seccomp
+ still start, which makes the failure hard to diagnose. The only
+ recovery is to restart the affected containers, raise
+ net.core.bpf_jit_limit, or reboot the node.
+
+ Upstream fixed the underlying problem in v6.11 with commit bfafe5efa975
+ ("seccomp: release task filters when the task exits"): the filter is
+ now detached and released in do_exit(), right after PF_EXITING is set,
+ so a zombie no longer pins a seccomp filter or its JIT image, and
+ bpf_jit_current is uncharged as soon as the process exits regardless of
+ when (or whether) its parent calls wait(). It also fixes a second
+ symptom of the same root cause: SECCOMP_FILTER_FLAG_TSYNC fails
+ permanently once the thread-group leader installed a filter and exited,
+ because seccomp_can_sync_threads() still walks the zombie leader.
+
+ Noble already carries this commit (6.8.0-50.51, LP: #2085849). Jammy
+ does not, and linux-5.15.y upstream never received it.
+
+ [ Fix ]
+
+ Backport upstream commit bfafe5efa975 ("seccomp: release task filters
+ when the task exits"). It moves seccomp_filter_release() from
+ release_task() into do_exit() (taking siglock to avoid racing
+ seccomp_sync_threads()), and makes seccomp_can_sync_threads() /
+ seccomp_sync_threads() skip PF_EXITING threads.
+
+
+ [ Test Plan ]
+
+ A. Docker reproduction – on a Jammy host
+ with docker.io and the default seccomp profile:
+
+ 1. Start a container whose PID 1 is a plain program that never
+ reaps children (no systemd/tini/init):
+ docker run -d --name z busybox sleep 999999
+ 2. Record the JIT usage baseline :
+ sudo awk '/bpf_jit/{s+=$2} END{printf "%.1f MiB in %d
regions\n",s/1048576,NR}' /proc/vmallocinfo
+ 3. Repeatedly exec a "probe"-style command that leaves an orphaned
+ child behind, e.g. 1000 times:
+ for i in $(seq 1000); do
+ docker exec z sh -c 'sleep 0.1 & exit 0'
+ done
+ Each docker exec makes runc install a new seccomp filter for the
+ exec'd process; the backgrounded child inherits it, is reparented
+ to the container's PID 1 (sleep) when sh exits, and becomes a
+ zombie that is never reaped:
+
+ 4. Re-check the count from step 2 while the zombies exist.
+ - Unpatched: grow as the accumulated docker exec probe invocation.
+ - Patched: stay the same
+
+
+ [ Where problems could occur ]
+
+ The change is on the exit path of every task, with or without seccomp.
+ seccomp_filter_release() now runs in do_exit() under a brief siglock
+ instead of at reap time; a mistake there would show up as a WARN_ON
+ (the patch adds one if the function is reached without PF_EXITING)
** Changed in: linux (Ubuntu)
Status: New => Incomplete
** Changed in: linux (Ubuntu)
Importance: Undecided => Medium
** Changed in: linux (Ubuntu)
Status: Incomplete => In Progress
** Also affects: linux (Ubuntu Jammy)
Importance: Undecided
Status: New
** Changed in: linux (Ubuntu Jammy)
Importance: Undecided => Medium
** Changed in: linux (Ubuntu Jammy)
Assignee: (unassigned) => dongdong tao (taodd)
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2164699
Title:
seccomp filter leak in bpf_jit due to unreaped zombie process
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2164699/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
