I was bothered by the fact that we only sometimes see the double partition rescan in dmesg. If udev always rescans the partitions of a full block devices, shouldn't we always see those messages twice?
It turns out that udev doesn't rescan partitions just because a new block device appears. When it gets a uevent for a full block device, it registers an inotify watch on the device file for any process closing a file that was open in write mode. Presumably that process could've changed the partitions so, when that inotify event fires, it requests a partition rescan. In our case, losetup has opened /dev/loop0 in write mode. When it calls into the LOOP_CONFIGURE ioctl(), a uevent will fire for /dev/loop0. If udev processes that event and adds the inotify watch before losetup closes the device, then closing the device will trigger the inotify event and udev will rescan. I don't see a way to order operations to prevent this race. And I don't think we can "settle" our way out of it, because its not a uevent we're racing with, its the inotify event. But it looks like systemd may already have a solution for that: https://systemd.io/BLOCK_DEVICE_LOCKING/ I think as long as we run all of our commands that access the partition device files under `udevadm lock -d <fulldevice>`, then systemd should not trigger a partition reread underneath us. Here's what I'm thinking: https://code.launchpad.net/~dannf/livecd-rootfs/+git/livecd- rootfs/+merge/459549 Of course, udevadm lock didn't exist until after jammy. And all versions that do have it, including the one in v255 in noble-proposed, are broken and need this patch backport: https://github.com/systemd/systemd/commit/ba340e2a75a0a16031fcb7efa05cfd250e859f17 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2045586 Title: livecd-rootfs uses losetup -P for theoretically reliable/synchronous partition setup but it's not reliable Status in linux package in Ubuntu: New Status in livecd-rootfs package in Ubuntu: Fix Released Status in util-linux package in Ubuntu: New Bug description: In mantic, we migrated livecd-rootfs to use losetup -P instead of kpartx, with the expectation that this would give us a reliable, race- free way of loop-mounting partitions from a disk image during image build. In noble, we are finding that it is no longer reliable, and in fact fails rather often. It is most noticeable with riscv64 builds, which is the architecture where we most frequently ran into problems before with kpartx. The first riscv64+generic build in noble where the expected loop partition device is not available is https://launchpad.net/~ubuntu- cdimage/+livefs/ubuntu/noble/cpc/+build/531790 The failure is however not unique to riscv64, and the autopkgtest for the latest version of livecd-rootfs (24.04.7) - an update that specifically tries to add more debugging code for this scenario - has also failed on ppc64el. https://autopkgtest.ubuntu.com/packages/l/livecd- rootfs/noble/ppc64el The first failure happened on November 16. While there has been an update to the util-linux package in noble, this did not land until November 23. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2045586/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
