Apparmor fixes for running Oracular unprivileged containers from `ubuntu-daily:24.10` are being progressively rolled out to the `latest/stable` channel now.
-- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/2046486 Title: units with credentials fail in LXD containers Status in cloud-images: Confirmed Status in lxd: New Status in lxd package in Ubuntu: Fix Committed Status in samba package in Ubuntu: Fix Released Status in systemd package in Ubuntu: Triaged Bug description: Many units shipped by systemd use credentials in some way by default now (in v256). So this issue is now about much more than the original test case failure. For example, root@oracular:~# apt policy systemd systemd: Installed: 256-1ubuntu1 Candidate: 256-1ubuntu1 Version table: *** 256-1ubuntu1 100 100 http://archive.ubuntu.com/ubuntu oracular-proposed/main amd64 Packages 100 /var/lib/dpkg/status 255.4-1ubuntu8 500 500 http://archive.ubuntu.com/ubuntu oracular/main amd64 Packages root@oracular:~# for service in $(find /usr/lib/systemd/system -maxdepth 1 -name "systemd-*.service"); do grep -q "Credential.*=" "$service" && echo "$service"; done /usr/lib/systemd/system/systemd-sysusers.service /usr/lib/systemd/system/systemd-resolved.service /usr/lib/systemd/system/systemd-firstboot.service /usr/lib/systemd/system/systemd-network-generator.service /usr/lib/systemd/system/systemd-journald.service /usr/lib/systemd/system/systemd-sysctl.service /usr/lib/systemd/system/systemd-tmpfiles-setup-dev-early.service /usr/lib/systemd/system/systemd-tmpfiles-setup-dev.service /usr/lib/systemd/system/systemd-tmpfiles-setup.service /usr/lib/systemd/system/systemd-udev-load-credentials.service /usr/lib/systemd/system/systemd-tmpfiles-clean.service /usr/lib/systemd/system/systemd-networkd.service root@oracular:~# systemctl status systemd-sysusers.service systemd-resolved.service systemd-firstboot.service systemd-network-generator.service systemd-journald.service systemd-sysctl.service systemd-tmpfiles-setup-dev-early.service systemd-tmpfiles-setup-dev.service systemd-tmpfiles-setup.service systemd-udev-load-credentials.service systemd-tmpfiles-clean.service systemd-networkd.service ○ systemd-sysusers.service - Create System Users Loaded: loaded (/usr/lib/systemd/system/systemd-sysusers.service; static) Active: inactive (dead) Condition: start condition unmet at Mon 2024-06-24 18:58:48 UTC; 1min 0s ago ├─ ConditionNeedsUpdate=|/etc was not met └─ ConditionCredential=|sysusers.extra was not met Docs: man:sysusers.d(5) man:systemd-sysusers.service(8) × systemd-resolved.service - Network Name Resolution Loaded: loaded (/usr/lib/systemd/system/systemd-resolved.service; enabled; preset: enabled) Active: failed (Result: exit-code) since Mon 2024-06-24 18:58:49 UTC; 59s ago Invocation: b1aaa662750f48868fe3388e4524c462 Docs: man:systemd-resolved.service(8) man:org.freedesktop.resolve1(5) https://systemd.io/WRITING_NETWORK_CONFIGURATION_MANAGERS https://systemd.io/WRITING_RESOLVER_CLIENTS Process: 258 ExecStart=/usr/lib/systemd/systemd-resolved (code=exited, status=243/CREDENTIALS) Main PID: 258 (code=exited, status=243/CREDENTIALS) ○ systemd-firstboot.service - First Boot Wizard Loaded: loaded (/usr/lib/systemd/system/systemd-firstboot.service; static) Active: inactive (dead) Condition: start condition unmet at Mon 2024-06-24 18:58:48 UTC; 59s ago └─ ConditionFirstBoot=yes was not met Docs: man:systemd-firstboot(1) ○ systemd-network-generator.service - Generate network units from Kernel command line Loaded: loaded (/usr/lib/systemd/system/systemd-network-generator.service; disabled; preset: enabled) Active: inactive (dead) Docs: man:systemd-network-generator.service(8) × systemd-journald.service - Journal Service Loaded: loaded (/usr/lib/systemd/system/systemd-journald.service; static) Drop-In: /usr/lib/systemd/system/systemd-journald.service.d └─nice.conf Active: failed (Result: exit-code) since Mon 2024-06-24 18:58:48 UTC; 1min 0s ago Invocation: 7caace7a15c749f3a86fb15fcfb94dff TriggeredBy: × systemd-journald-dev-log.socket × systemd-journald.socket ○ systemd-journald-audit.socket Docs: man:systemd-journald.service(8) man:journald.conf(5) Process: 124 ExecStart=/usr/lib/systemd/systemd-journald (code=exited, status=243/CREDENTIALS) Main PID: 124 (code=exited, status=243/CREDENTIALS) FD Store: 0 (limit: 4224) × systemd-sysctl.service - Apply Kernel Variables Loaded: loaded (/usr/lib/systemd/system/systemd-sysctl.service; static) Active: failed (Result: exit-code) since Mon 2024-06-24 18:58:48 UTC; 1min 0s ago Invocation: 5e90310a27b043ceae80c96e35c41451 Docs: man:systemd-sysctl.service(8) man:sysctl.d(5) Process: 97 ExecStart=/usr/lib/systemd/systemd-sysctl (code=exited, status=243/CREDENTIALS) Main PID: 97 (code=exited, status=243/CREDENTIALS) × systemd-tmpfiles-setup-dev-early.service - Create Static Device Nodes in /dev gracefully Loaded: loaded (/usr/lib/systemd/system/systemd-tmpfiles-setup-dev-early.service; static) Active: failed (Result: exit-code) since Mon 2024-06-24 18:58:48 UTC; 1min 0s ago Invocation: 78e3c68cfa9a4a7982950b08c0f1385f Docs: man:tmpfiles.d(5) man:systemd-tmpfiles(8) Process: 73 ExecStart=systemd-tmpfiles --prefix=/dev --create --boot --graceful (code=exited, status=243/CREDENTIALS) Main PID: 73 (code=exited, status=243/CREDENTIALS) × systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev Loaded: loaded (/usr/lib/systemd/system/systemd-tmpfiles-setup-dev.service; static) Active: failed (Result: exit-code) since Mon 2024-06-24 18:58:48 UTC; 1min 0s ago Invocation: 46458c7b6e134ef8be299900db7cc288 Docs: man:tmpfiles.d(5) man:systemd-tmpfiles(8) Process: 98 ExecStart=systemd-tmpfiles --prefix=/dev --create --boot (code=exited, status=243/CREDENTIALS) Main PID: 98 (code=exited, status=243/CREDENTIALS) × systemd-tmpfiles-setup.service - Create Volatile Files and Directories Loaded: loaded (/usr/lib/systemd/system/systemd-tmpfiles-setup.service; static) Active: failed (Result: exit-code) since Mon 2024-06-24 18:58:48 UTC; 1min 0s ago Invocation: f4e64afdc8774170a9b29b8cf2919f46 Docs: man:tmpfiles.d(5) man:systemd-tmpfiles(8) Process: 147 ExecStart=systemd-tmpfiles --create --remove --boot --exclude-prefix=/dev (code=exited, status=243/CREDENTIALS) Main PID: 147 (code=exited, status=243/CREDENTIALS) × systemd-udev-load-credentials.service - Load udev Rules from Credentials Loaded: loaded (/usr/lib/systemd/system/systemd-udev-load-credentials.service; disabled; preset: enabled) Active: failed (Result: exit-code) since Mon 2024-06-24 18:58:48 UTC; 1min 0s ago Invocation: cb5a1f43cde248de80fcf701b4b5d381 Docs: man:udevadm(8) man:udev(7) man:systemd.system-credentials(7) Process: 75 ExecStart=udevadm control --load-credentials (code=exited, status=243/CREDENTIALS) Main PID: 75 (code=exited, status=243/CREDENTIALS) ○ systemd-tmpfiles-clean.service - Cleanup of Temporary Directories Loaded: loaded (/usr/lib/systemd/system/systemd-tmpfiles-clean.service; static) Active: inactive (dead) TriggeredBy: ● systemd-tmpfiles-clean.timer Docs: man:tmpfiles.d(5) man:systemd-tmpfiles(8) × systemd-networkd.service - Network Configuration Loaded: loaded (/usr/lib/systemd/system/systemd-networkd.service; enabled; preset: enabled) Active: failed (Result: exit-code) since Mon 2024-06-24 18:58:49 UTC; 59s ago Invocation: 5d960369ea944d5cbac4382e42ded1d0 TriggeredBy: × systemd-networkd.socket Docs: man:systemd-networkd.service(8) man:org.freedesktop.network1(5) Process: 280 ExecStart=/usr/lib/systemd/systemd-networkd (code=exited, status=243/CREDENTIALS) Main PID: 280 (code=exited, status=243/CREDENTIALS) FD Store: 0 (limit: 512) [Original Description] To demonstrate this, in an unprivileged LXD container, create the following unit (taken from the systemd test suite): $ cat > /etc/systemd/system/exec-set-credential.service << EOF # SPDX-License-Identifier: LGPL-2.1-or-later [Unit] Description=Test for SetCredential= [Service] ExecStart=/bin/sh -x -c 'test "$$(cat %d/test-execute.set-credential)" = "hoge"' ExecStartPost=/bin/sh -x -c 'test "$$(cat %d/test-execute.set-credential)" = "hoge"' ExecStop=/bin/sh -x -c 'test "$$(cat %d/test-execute.set-credential)" = "hoge"' ExecStopPost=/bin/sh -x -c 'test "$$(cat %d/test-execute.set-credential)" = "hoge"' Type=oneshot SetCredential=test-execute.set-credential:hoge EOF $ systemctl daemon-reload $ systemctl start exec-set-credential.service Job for exec-set-credential.service failed because the control process exited with error code. See "systemctl status exec-set-credential.service" and "journalctl -xeu exec-set-credential.service" for details. With debug logs enabled, we see: $ journalctl -u exec-set-credential.service -b --no-pager Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Trying to enqueue job exec-set-credential.service/start/replace Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Installed new job exec-set-credential.service/start as 2740 Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Enqueued job exec-set-credential.service/start as 2740 Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Will spawn child (service_enter_start): /bin/sh Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Failed to set 'trusted.invocation_id' xattr on control group /system.slice/exec-set-credential.service, ignoring: Operation not permitted Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Failed to remove 'trusted.delegate' xattr flag on control group /system.slice/exec-set-credential.service, ignoring: Operation not permitted Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Failed to remove 'trusted.survive_final_kill_signal' xattr flag on control group /system.slice/exec-set-credential.service, ignoring: Operation not permitted Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Passing 0 fds to service Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: About to execute: /bin/sh -x -c "test \"1031(cat /run/credentials/exec-set-credential.service/test-execute.set-credential)\" = \"hoge\"" Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Forked /bin/sh as 2183 Dec 14 19:24:24 noble (sh)[2183]: PR_SET_MM_ARG_START failed: Operation not permitted Dec 14 19:24:24 noble (sh)[2183]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy Dec 14 19:24:24 noble (sh)[2183]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Changed dead -> start Dec 14 19:24:24 noble systemd[1]: Starting exec-set-credential.service - Test for SetCredential=... Dec 14 19:24:24 noble (sh)[2183]: Successfully forked off '(sd-mkdcreds)' as PID 2184. Dec 14 19:24:24 noble (sd-[2184]: Changing mount propagation /dev (MS_REC|MS_SLAVE "") Dec 14 19:24:24 noble (sd-[2184]: Mounting ramfs (ramfs) on /dev/shm (MS_NOSUID|MS_NODEV|MS_NOEXEC|MS_NOSYMFOLLOW "mode=0700")... Dec 14 19:24:24 noble (sd-[2184]: Changing mount flags /dev/shm (MS_RDONLY|MS_NOSUID|MS_NODEV|MS_NOEXEC|MS_REMOUNT|MS_NOSYMFOLLOW|MS_BIND "")... Dec 14 19:24:24 noble (sd-[2184]: Failed to mount n/a (type n/a) on /dev/shm (MS_RDONLY|MS_NOSUID|MS_NODEV|MS_NOEXEC|MS_REMOUNT|MS_NOSYMFOLLOW|MS_BIND ""): Permission denied Dec 14 19:24:24 noble (sh)[2183]: (sd-mkdcreds) failed with exit status 1. Dec 14 19:24:24 noble (sh)[2183]: exec-set-credential.service: Failed to set up credentials: Protocol error Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Child 2183 belongs to exec-set-credential.service. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Main process exited, code=exited, status=243/CREDENTIALS Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Will spawn child (service_enter_stop_post): /bin/sh Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: About to execute: /bin/sh -x -c "test \"1031(cat /run/credentials/exec-set-credential.service/test-execute.set-credential)\" = \"hoge\"" Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Forked /bin/sh as 2186 Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Changed start -> stop-post Dec 14 19:24:24 noble (sh)[2186]: PR_SET_MM_ARG_START failed: Operation not permitted Dec 14 19:24:24 noble (sh)[2186]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy Dec 14 19:24:24 noble (sh)[2186]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy Dec 14 19:24:24 noble sh[2186]: + test 1031(cat /run/credentials/exec-set-credential.service/test-execute.set-credential) = hoge Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Child 2186 belongs to exec-set-credential.service. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Control process exited, code=exited, status=1/FAILURE Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Got final SIGCHLD for state stop-post. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Failed with result 'exit-code'. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Service will not restart (restart setting) Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Changed stop-post -> failed Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Job 2740 exec-set-credential.service/start finished, result=failed Dec 14 19:24:24 noble systemd[1]: Failed to start exec-set-credential.service - Test for SetCredential=. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Unit entered failed state. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Consumed 23ms CPU time. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Releasing resources... To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/2046486/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
