[Bug 2040483] Re: AppArmor denies crun sending signals to containers (stop, kill)
PR accepted upstream. I've backported the patch the oracular MP above. What needs to be done now to get this into an SRU for noble? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2040483 Title: AppArmor denies crun sending signals to containers (stop, kill) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/golang-github-containers-common/+bug/2040483/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2040483] Re: AppArmor denies crun sending signals to containers (stop, kill)
PR opened upstream: https://github.com/containers/common/pull/2004 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2040483 Title: AppArmor denies crun sending signals to containers (stop, kill) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/golang-github-containers-common/+bug/2040483/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2040483] Re: AppArmor denies crun sending signals to containers (stop, kill)
Thanks Neil, I'll let you handle the upstream. I think what you have in the MP is fine. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2040483 Title: AppArmor denies crun sending signals to containers (stop, kill) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/golang-github-containers-common/+bug/2040483/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2040483] Re: AppArmor denies crun sending signals to containers (stop, kill)
I've pushed the changes based on your comments to the MP above. I've left the signal set for podman as (int, quit, term, kill). Do you think that signal set should be tighter, or is that a good compromise? If that seems ok with you, I'll happily handle the PR upstream at GitHub. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2040483 Title: AppArmor denies crun sending signals to containers (stop, kill) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/golang-github-containers-common/+bug/2040483/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2040483] Re: AppArmor denies crun sending signals to containers (stop, kill)
Sorry, I missed the conmon-podman denial. Would you mind making a PR to the upstream with your changes with issue you posted linked? I think Lucas will not have time until end of week. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2040483 Title: AppArmor denies crun sending signals to containers (stop, kill) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/golang-github-containers-common/+bug/2040483/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2040483] Re: AppArmor denies crun sending signals to containers (stop, kill)
The debdiff is in the MP above. Podman does try to kill the container itself, as the error trace above testifies. May 14 11:14:41 srv-omzr6 kernel: audit: type=1400 audit(1715685281.392:118): apparmor="DENIED" operation="signal" class="signal" profile="containers-default-0.57.4" pid=7458 comm="conmon" requested_mask="receive" denied_mask="receive" signal=term peer="podman" It's trying to kill conmon in some scenarios, which means your policy changes so far are deficient in that regard. We can tighten the signal set there to term and kill, which is certainly no worse than the pre-4.0.0 situation. I note the point about the signal set on the runtimes, and that should be removed. The stop signals can be set to anything within the container. I would suggest extending the AARE to cover the binaries as well as the policy name. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2040483 Title: AppArmor denies crun sending signals to containers (stop, kill) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/golang-github-containers-common/+bug/2040483/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2040483] Re: AppArmor denies crun sending signals to containers (stop, kill)
Also, thanks for linking the podman issue. I'll try to merge patch upstream similar to moby and containerd. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2040483 Title: AppArmor denies crun sending signals to containers (stop, kill) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/golang-github-containers-common/+bug/2040483/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2040483] Re: AppArmor denies crun sending signals to containers (stop, kill)
@neil-aldur, did you forget to attach the debdiff? By restricting the signal set you also restrict what $SIG you can put to "podman kill --signal $SIG". I did not realize that there's a podman reference profile as well, but since podman doesn't try to kill the container by itself, I wonder if it makes sense to arbitrarily open a policy like this. Also, whether you changes are good or not, they diverge from the policy changes we have already merged to containerd and moby upstream. Not sure if that's a problem. Regarding your changes to the changelog entry in your MP: I based my entry on comment on a code comment from ahasenack (https://code.launchpad.net/~fun2program8/ubuntu/+source/crun/+git/crun/+merge/464233, you have to select b879 commit, it's the first code comment). I don't think we should copy the commit message into changelog entries. It's already in the patch. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2040483 Title: AppArmor denies crun sending signals to containers (stop, kill) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/golang-github-containers-common/+bug/2040483/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2040483] Re: AppArmor denies crun sending signals to containers (stop, kill)
Adding the podman signal line, and building a libpod that overrides the default packages eliminates the errors I was getting. All the tests in this ticket pass with the updated packages. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2040483 Title: AppArmor denies crun sending signals to containers (stop, kill) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/golang-github-containers-common/+bug/2040483/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2040483] Re: AppArmor denies crun sending signals to containers (stop, kill)
I've built a backported 4.9.4 libpod for noble based on an updated golang-github-containers-common including the above patch. It's available from ppa:brightbox/experimental -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2040483 Title: AppArmor denies crun sending signals to containers (stop, kill) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/golang-github-containers-common/+bug/2040483/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2040483] Re: AppArmor denies crun sending signals to containers (stop, kill)
The debdiff I've put together for oracular updates the patch to be a bit
more general and cover all the signals I've seen so far in testing. (As
well as dropping the other patch that has been incorporated upstream).
# Allow certain signals from OCI runtimes (podman, runc and crun)
signal (receive) set=(int, quit, kill, term)
peer={/usr/bin/,/usr/sbin/,}runc,
signal (receive) set=(int, quit, kill, term)
peer={/usr/bin/,/usr/sbin/,}crun,
signal (receive) set=(int, quit, kill, term)
peer={/usr/bin/,/usr/sbin/,}podman,
Upstream have said they have no apparmor experience, so I suspect they
will take a PR. See https://github.com/containers/common/issues/1898
** Bug watch added: github.com/containers/common/issues #1898
https://github.com/containers/common/issues/1898
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2040483
Title:
AppArmor denies crun sending signals to containers (stop, kill)
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/golang-github-containers-common/+bug/2040483/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2040483] Re: AppArmor denies crun sending signals to containers (stop, kill)
@lucaskanashiro: This patch is for golang-github-containers-common source package. This source package produces golang-github-containers- common-dev binary package, which is just source code on filesystem. But podman binary package, which is produced from libpod source package, has golang-github-containers-common-dev in its Build-Depends. It doesn't depend on golang-github-containers-common-dev during runtime. So when golang-github-containers-common is updated, libpod should be rebuilt. Should I keep libpod as confirmed, and make a patch against libpod to make a rebuild? @neil-aldur: I'll try to reproduce. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2040483 Title: AppArmor denies crun sending signals to containers (stop, kill) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/golang-github-containers-common/+bug/2040483/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2040483] Re: AppArmor denies crun sending signals to containers (stop, kill)
The patch above doesn't work as it stands. We are still getting signal filters in the audit log May 14 11:13:06 srv-omzr6 kernel: audit: type=1400 audit(1715685186.296:112): apparmor="DENIED" operation="signal" class="signal" profile="containers-default-0.57.4" pid=8031 comm="3" requested_mask="receive" denied_mask="receive" signal=term peer="crun" May 14 11:13:06 srv-omzr6 kernel: audit: type=1400 audit(1715685186.318:113): apparmor="DENIED" operation="signal" class="signal" profile="containers-default-0.57.4" pid=8033 comm="3" requested_mask="receive" denied_mask="receive" signal=term peer="crun" May 14 11:13:16 srv-omzr6 kernel: audit: type=1400 audit(1715685196.340:114): apparmor="DENIED" operation="signal" class="signal" profile="containers-default-0.57.4" pid=8035 comm="3" requested_mask="receive" denied_mask="receive" signal=kill peer="crun" May 14 11:13:21 srv-omzr6 kernel: audit: type=1400 audit(1715685201.413:115): apparmor="DENIED" operation="signal" class="signal" profile="containers-default-0.57.4" pid=7664 comm="conmon" requested_mask="receive" denied_mask="receive" signal=term peer="podman" May 14 11:14:31 srv-omzr6 kernel: audit: type=1400 audit(1715685271.577:116): apparmor="DENIED" operation="signal" class="signal" profile="containers-default-0.57.4" pid=8049 comm="3" requested_mask="receive" denied_mask="receive" signal=term peer="crun" May 14 11:14:36 srv-omzr6 kernel: audit: type=1400 audit(1715685276.326:117): apparmor="DENIED" operation="signal" class="signal" profile="containers-default-0.57.4" pid=8052 comm="3" requested_mask="receive" denied_mask="receive" signal=kill peer="crun" May 14 11:14:41 srv-omzr6 kernel: audit: type=1400 audit(1715685281.392:118): apparmor="DENIED" operation="signal" class="signal" profile="containers-default-0.57.4" pid=7458 comm="conmon" requested_mask="receive" denied_mask="receive" signal=term peer="podman" May 14 11:14:41 srv-omzr6 kernel: audit: type=1400 audit(1715685281.604:119): apparmor="DENIED" operation="signal" class="signal" profile="containers-default-0.57.4" pid=8055 comm="3" requested_mask="receive" denied_mask="receive" signal=kill peer="crun" -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2040483 Title: AppArmor denies crun sending signals to containers (stop, kill) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/golang-github-containers-common/+bug/2040483/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2040483] Re: AppArmor denies crun sending signals to containers (stop, kill)
To move this on a bit more rapidly as it is a blocking issue for me. It's the same version in Oracular at present. I've pushed the changes as an MP against ubuntu/devel. What needs to happen next? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2040483 Title: AppArmor denies crun sending signals to containers (stop, kill) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/golang-github-containers-common/+bug/2040483/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2040483] Re: AppArmor denies crun sending signals to containers (stop, kill)
** Merge proposal linked: https://code.launchpad.net/~neil-aldur/ubuntu/+source/golang-github-containers-common/+git/golang-github-containers-common/+merge/465970 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2040483 Title: AppArmor denies crun sending signals to containers (stop, kill) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/golang-github-containers-common/+bug/2040483/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2040483] Re: AppArmor denies crun sending signals to containers (stop, kill)
I also see that you are patching golang-github-containers-common. Does that mean that no patch in libpod is needed? If the answer is yes, we need to mark the libpod tasks as Invalid. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2040483 Title: AppArmor denies crun sending signals to containers (stop, kill) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/golang-github-containers-common/+bug/2040483/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2040483] Re: AppArmor denies crun sending signals to containers (stop, kill)
Hi Tomáš, Thanks for investigating this issue and providing the patch (MP) to fix it in Noble. However, before fixing it in Noble, we need to fix it in Oracular (development release). Would you like to provide a patch or MP targeting Oracular? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2040483 Title: AppArmor denies crun sending signals to containers (stop, kill) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/golang-github-containers-common/+bug/2040483/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2040483] Re: AppArmor denies crun sending signals to containers (stop, kill)
** Description changed: [ Impact ] * On mantic and noble, when run as root, podman cannot stop any container running in background because crun is being run with a new profile introduced in AppArmor v4.0.0 that doesn't have corresponding signal receive rule container's profile. * Without the fix, users would have to resort to figuring out container's PID 1 and killing it as root or by other privileged and unconfined process. This is a regression from basic podman functionality. * The fix adds signal receive rules for currently confined OCI runtimes in AppArmor v4.0.0 (runc and crun) to the profile used by podman. [ Test Plan ] All commands must be invoked as root. + + Run tests below with both crun and runc OCI runtimes. For crun, nothing + has to be changed (it's installed and used by default). For runc, first + install the runc pakcage, and then insert "--runtime /usr/sbin/runc" + arguments after "podman run". Start container in background and then stop it: # Run container in background (-d) podman run -d --name foo docker.io/library/nginx:latest # Stop the container podman stop foo On success, the last command should print the container name and the container running in background should be stopped (verify with "podman ps"). Additional tests: Verify that container running in foreground TTY can be stopped. # Terminal 1: # Run container on this TTY podman run -it --name bar --rm docker.io/library/ubuntu:22.04 # Terminal 2: # Stop the container podman stop bar On success, the last command should print the container name, the process running in terminal 1 should stop, and the container should be removed (verify with "podman ps -a"). Verify that container running with dumb init can be killed. # Run container in background (-d) with dumb init podman run -d --name bar --rm --init ubuntu:22.04 sleep infinity # Stop the container podman stop bar On success, the last command should print the container name and the container running in background should be stopped and removed (verify with "podman ps -a"). Verify container processes can signal each other # Run container in foreground with processes sending signals between themselves podman run ubuntu:22.04 sh -c 'sleep inf & sleep 1 ; kill $!' On success, the last command should exit after cca 1 second with exit status 0. [ Where problems could occur ] * The fix requires a rebuild of podman that will pull in any other changes in the archive since the last build, which could potentially break some functionality. [ Original report ] Mantic's system podman containers are completely broken due to bug 2040082. However, after fixing that (rebuilding with the patch, or a *shht don't try this at home* hack [1]), the AppArmor policy still causes bugs: podman run -it --rm docker.io/busybox Then podman stop -l fails with 2023-10-25T11:06:33.873998Z: send signal to pidfd: Permission denied and journal shows audit: type=1400 audit(1698231993.870:92): apparmor="DENIED" operation="signal" class="signal" profile="containers-default-0.50.1" pid=4713 comm="3" requested_mask="receive" denied_mask="receive" signal=term peer="/usr/bin/crun" This leaves the container in a broken state: # podman ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 61749260f9c4 docker.io/library/busybox:latest sh 40 seconds ago Exited (-1) 29 seconds ago confident_bouman # podman rm --all 2023-10-25T11:07:21.428701Z: send signal to pidfd: Permission denied Error: cleaning up container 61749260f9c4c96a51dc27fdd9cb8a86d80e4f2aa14eb7ed5b271791ff8008ae: removing container 61749260f9c4c96a51dc27fdd9cb8a86d80e4f2aa14eb7ed5b271791ff8008ae from runtime: `/usr/bin/crun delete --force 61749260f9c4c96a51dc27fdd9cb8a86d80e4f2aa14eb7ed5b271791ff8008ae` failed: exit status 1 audit: type=1400 audit(1698232041.422:93): apparmor="DENIED" operation="signal" class="signal" profile="containers-default-0.50.1" pid=4839 comm="3" requested_mask="receive" denied_mask="receive" signal=kill peer="/usr/bin/crun" [1] sed -i 's/~alpha2/000/' /usr/sbin/apparmor_parser Ubuntu 23.10 ii apparmor4.0.0~alpha2-0ubuntu5 amd64 user-space parser utility for AppArmor ii golang-github-containers-common 0.50.1+ds1-4 all Common files for github.com/containers repositories ii podman 4.3.1+ds1-8 amd64engine to run OCI-based containers in Pods -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2040483 Title: AppArmor denies
[Bug 2040483] Re: AppArmor denies crun sending signals to containers (stop, kill)
** Description changed: [ Impact ] - * On mantic and noble, when run as root, podman cannot stop any + * On mantic and noble, when run as root, podman cannot stop any container running in background because crun is being run with a new profile introduced in AppArmor v4.0.0 that doesn't have corresponding signal receive rule container's profile. - * Without the fix, users would have to resort to figuring out + * Without the fix, users would have to resort to figuring out container's PID 1 and killing it as root or by other privileged and unconfined process. This is a regression from basic podman functionality. - * The fix adds signal receive rules for currently confined OCI runtimes + * The fix adds signal receive rules for currently confined OCI runtimes in AppArmor v4.0.0 (runc and crun) to the profile used by podman. [ Test Plan ] All commands must be invoked as root. Start container in background and then stop it: - # Run container in background (-d) - podman run -d --name foo docker.io/library/nginx:latest - # Stop the container - podman stop foo + # Run container in background (-d) + podman run -d --name foo docker.io/library/nginx:latest + # Stop the container + podman stop foo On success, the last command should print the container name and the container running in background should be stopped (verify with "podman ps"). Additional tests: Verify that container running in foreground TTY can be stopped. - # Terminal 1: - # Run container on this TTY - podman run -it --name bar --rm docker.io/library/ubuntu:22.04 + # Terminal 1: + # Run container on this TTY + podman run -it --name bar --rm docker.io/library/ubuntu:22.04 - # Terminal 2: - # Stop the container - podman stop foo + # Terminal 2: + # Stop the container + podman stop bar On success, the last command should print the container name, the process running in terminal 1 should stop, and the container should be removed (verify with "podman ps -a"). Verify that container running with dumb init can be killed. - # Run container in background (-d) with dumb init - podman run -d --name bar --rm --init ubuntu:22.04 sleep infinity - # Stop the container - podman stop foo + # Run container in background (-d) with dumb init + podman run -d --name bar --rm --init ubuntu:22.04 sleep infinity + # Stop the container + podman stop bar On success, the last command should print the container name and the container running in background should be stopped and removed (verify with "podman ps -a"). Verify container processes can signal each other - # Run container in foreground with processes sending signals between themselves - podman run ubuntu:22.04 sh -c 'sleep inf & sleep 1 ; kill $!' + # Run container in foreground with processes sending signals between themselves + podman run ubuntu:22.04 sh -c 'sleep inf & sleep 1 ; kill $!' - - On success, the last command should exit after cca 1 second with exit status 0. + On success, the last command should exit after cca 1 second with exit + status 0. [ Where problems could occur ] - * The fix requires a rebuild of podman that will pull in any other + * The fix requires a rebuild of podman that will pull in any other changes in the archive since the last build, which could potentially break some functionality. - [ Original report ] - - Mantic's system podman containers are completely broken due to bug 2040082. However, after fixing that (rebuilding with the patch, or a *shht don't try this at home* hack [1]), the AppArmor policy still causes bugs: + Mantic's system podman containers are completely broken due to bug + 2040082. However, after fixing that (rebuilding with the patch, or a + *shht don't try this at home* hack [1]), the AppArmor policy still + causes bugs: podman run -it --rm docker.io/busybox Then podman stop -l fails with 2023-10-25T11:06:33.873998Z: send signal to pidfd: Permission denied and journal shows audit: type=1400 audit(1698231993.870:92): apparmor="DENIED" operation="signal" class="signal" profile="containers-default-0.50.1" pid=4713 comm="3" requested_mask="receive" denied_mask="receive" signal=term peer="/usr/bin/crun" This leaves the container in a broken state: # podman ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 61749260f9c4 docker.io/library/busybox:latest sh 40 seconds ago Exited (-1) 29 seconds ago confident_bouman # podman rm --all 2023-10-25T11:07:21.428701Z: send signal to pidfd: Permission denied Error: cleaning up container 61749260f9c4c96a51dc27fdd9cb8a86d80e4f2aa14eb7ed5b271791ff8008ae: removing container 61749260f9c4c96a51dc27fdd9cb8a86d80e4f2aa14eb7ed5b271791ff8008ae from runtime:
[Bug 2040483] Re: AppArmor denies crun sending signals to containers (stop, kill)
Status changed to 'Confirmed' because the bug affects multiple users. ** Changed in: golang-github-containers-common (Ubuntu) Status: New => Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2040483 Title: AppArmor denies crun sending signals to containers (stop, kill) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/golang-github-containers-common/+bug/2040483/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2040483] Re: AppArmor denies crun sending signals to containers (stop, kill)
** Description changed: - Mantic's system podman containers are completely broken due to bug - 2040082. However, after fixing that (rebuilding with the patch, or a - *shht don't try this at home* hack [1]), the AppArmor policy still - causes bugs: + [ Impact ] + + * On mantic and noble, when run as root, podman cannot stop any + container running in background because crun is being run with a new + profile introduced in AppArmor v4.0.0 that doesn't have corresponding + signal receive rule container's profile. + + * Without the fix, users would have to resort to figuring out + container's PID 1 and killing it as root or by other privileged and + unconfined process. This is a regression from basic podman + functionality. + + * The fix adds signal receive rules for currently confined OCI runtimes + in AppArmor v4.0.0 (runc and crun) to the profile used by podman. + + [ Test Plan ] + + All commands must be invoked as root. + + Start container in background and then stop it: + + # Run container in background (-d) + podman run -d --name foo docker.io/library/nginx:latest + # Stop the container + podman stop foo + + On success, the last command should print the container name and the + container running in background should be stopped (verify with "podman + ps"). + + Additional tests: + + Verify that container running in foreground TTY can be stopped. + + # Terminal 1: + # Run container on this TTY + podman run -it --name bar --rm docker.io/library/ubuntu:22.04 + + # Terminal 2: + # Stop the container + podman stop foo + + On success, the last command should print the container name, the + process running in terminal 1 should stop, and the container should be + removed (verify with "podman ps -a"). + + Verify that container running with dumb init can be killed. + + # Run container in background (-d) with dumb init + podman run -d --name bar --rm --init ubuntu:22.04 sleep infinity + # Stop the container + podman stop foo + + On success, the last command should print the container name and the + container running in background should be stopped and removed (verify + with "podman ps -a"). + + Verify container processes can signal each other + + # Run container in foreground with processes sending signals between themselves + podman run ubuntu:22.04 sh -c 'sleep inf & sleep 1 ; kill $!' + + + On success, the last command should exit after cca 1 second with exit status 0. + + [ Where problems could occur ] + + * The fix requires a rebuild of podman that will pull in any other + changes in the archive since the last build, which could potentially + break some functionality. + + + [ Original report ] + + + Mantic's system podman containers are completely broken due to bug 2040082. However, after fixing that (rebuilding with the patch, or a *shht don't try this at home* hack [1]), the AppArmor policy still causes bugs: podman run -it --rm docker.io/busybox Then podman stop -l fails with 2023-10-25T11:06:33.873998Z: send signal to pidfd: Permission denied and journal shows audit: type=1400 audit(1698231993.870:92): apparmor="DENIED" operation="signal" class="signal" profile="containers-default-0.50.1" pid=4713 comm="3" requested_mask="receive" denied_mask="receive" signal=term peer="/usr/bin/crun" This leaves the container in a broken state: # podman ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 61749260f9c4 docker.io/library/busybox:latest sh 40 seconds ago Exited (-1) 29 seconds ago confident_bouman # podman rm --all 2023-10-25T11:07:21.428701Z: send signal to pidfd: Permission denied Error: cleaning up container 61749260f9c4c96a51dc27fdd9cb8a86d80e4f2aa14eb7ed5b271791ff8008ae: removing container 61749260f9c4c96a51dc27fdd9cb8a86d80e4f2aa14eb7ed5b271791ff8008ae from runtime: `/usr/bin/crun delete --force 61749260f9c4c96a51dc27fdd9cb8a86d80e4f2aa14eb7ed5b271791ff8008ae` failed: exit status 1 audit: type=1400 audit(1698232041.422:93): apparmor="DENIED" operation="signal" class="signal" profile="containers-default-0.50.1" pid=4839 comm="3" requested_mask="receive" denied_mask="receive" signal=kill peer="/usr/bin/crun" [1] sed -i 's/~alpha2/000/' /usr/sbin/apparmor_parser Ubuntu 23.10 ii apparmor4.0.0~alpha2-0ubuntu5 amd64 user-space parser utility for AppArmor ii golang-github-containers-common 0.50.1+ds1-4 all Common files for github.com/containers repositories ii podman 4.3.1+ds1-8 amd64engine to run OCI-based containers in Pods -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2040483 Title: AppArmor denies crun sending signals to
[Bug 2040483] Re: AppArmor denies crun sending signals to containers (stop, kill)
** Merge proposal linked: https://code.launchpad.net/~virtustom/ubuntu/+source/golang-github-containers-common/+git/golang-github-containers-common/+merge/465117 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2040483 Title: AppArmor denies crun sending signals to containers (stop, kill) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/golang-github-containers-common/+bug/2040483/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2040483] Re: AppArmor denies crun sending signals to containers (stop, kill)
** Also affects: golang-github-containers-common (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2040483 Title: AppArmor denies crun sending signals to containers (stop, kill) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/golang-github-containers-common/+bug/2040483/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2040483] Re: AppArmor denies crun sending signals to containers (stop, kill)
There's a similar issue with runc (and containerd and docker) reported here https://bugs.launchpad.net/ubuntu/+source/apparmor/+bug/2039294 I've opened PRs with a fix upstream: - https://github.com/containerd/containerd/pull/10123 - https://github.com/moby/moby/pull/47749 I think I'll need to work a little bit more on them to dynamically add rules only for profiles that exist on the system, even though it works even if they don't exist. Is this a proper way to fix it? I have gained all my experience with AppArmor in last 2 days. For podman a similar change should be applied to the profile template defined here https://github.com/containers/common/blob/main/pkg/apparmor/apparmor_linux_template.go. I can do that later. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2040483 Title: AppArmor denies crun sending signals to containers (stop, kill) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libpod/+bug/2040483/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 2040483] Re: AppArmor denies crun sending signals to containers (stop, kill)
** Tags added: cockpit-test -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2040483 Title: AppArmor denies crun sending signals to containers (stop, kill) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libpod/+bug/2040483/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
