Just adding a comment on how to reproduce the bug, because I have lost a day trying to figure it out again. This fails only when not using secure boot because the device mapper state is kept from initrd to main boot.
-- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/2037281 Title: Shutdown when triggering daemon-reload eary in boot Status in systemd package in Ubuntu: Fix Released Status in systemd source package in Focal: New Status in systemd source package in Jammy: New Bug description: In Ubuntu Core 20, and Ubuntu Core 22, we are encountering an issue where if a service, started earlier than devices are processed by udev, does `systemctl daemon-reload`, the system shuts down. This is due to devices for mounted filesystem temporarily taken dead, which pulls most units down. This was fixed by upstream in https://github.com/systemd/systemd/pull/23218. But this was not backported to the versions used by Ubuntu packages for focal and jammy. The needed commit from that PR is the one with message `core/device: ignore DEVICE_FOUND_UDEV bit on switching root`. This patch applies to 245.4-4ubuntu3.22 (focal) without rebasing needed. And I suppose it does also for jammy. I have manually tested the fix with Ubuntu Core 20, and this fixes our issue. We would like this patch to be backported to focal-updates and jammy- updates. Thank you in advance. [ Impact ] If a user adds a service that calls `systemctl daemon-reload`, and if this service is started before systemd-udevd. And if the initrd is systemd (the case of Ubuntu Core), then most service will be stopped or cancel, and the machine will mostly shutdown everything and hang. The fix has been backported down to 250 upstream. It is already on kinetic and later. The fix only affects systems where systemd is used in initrd. [ Test Plan ] On Ubuntu Core 20 (with Core 22 kernel) or on Ubuntu Core 22. Or on any system that uses systemd in initrd. Add a systemd service that calls `systemctl daemon-reload`. The service should have `DefaultDependencies=no` in order to start as soon as possible and be enabled. Restart the machine. If fix is not applied, after the service is started, most of units with be shutdown, and the system will be unusable. [ Where problems could occur ] This should affect systems with systemd in initrd. There are risks on systems that have an udev rule in initrd not present in the main system. There are risks on systems that use db_persist in initrd where the device can potentially get dead state. Though this does not seem to happen on Ubuntu Core 22, even though we use db_persist for dev mapper devices. Regression is upstream bug #23429. Commits named "core/device: device_coldplug(): don't set DEVICE_DEAD" and "core/device: do not downgrade device state if it is already enumerated" could be applied as well. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/2037281/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp