Note to self. umount.target A special target unit that unmounts all mount and automount points on system shutdown.
Mounts that shall be unmounted on system shutdown shall add Conflicts dependencies to this unit for their mount unit, which is implicitly done when DefaultDependencies=yes is set (the default). Given that /media/root-ro is a pre-requisite for /, imho it should be declared with DefaultDependencies=no. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1788188 Title: transient systemd ordering cycle in boot with overlayroot Status in systemd package in Ubuntu: Confirmed Bug description: open-iscsi test utilizes overlayroot to boot a cloud-image with root filesystem on a read-only iscsi server. The /etc/fstab file in the image looks like this: LABEL=cloudimg-rootfs / ext4 defaults 0 0 #LABEL=UEFI /boot/efi vfat defaults 0 0 when init takes over from the initramfs, we have /proc/cmdline: nomodeset iscsi_initiator=maas-enlist iscsi_target_name=tgt-boot-test-2abbnj iscsi_target_ip=10.0.12.2 iscsi_target_port=3260 iscsi_initiator=maas-enlist ip=::::maas-enlist:BOOTIF ro net.ifnames=0 BOOTIF_DEFAULT=eth0 root=/dev/disk/by-path/ip-10.0.12.2:3260-iscsi-tgt-boot-test-2abbnj-lun-1-part1 overlayroot=tmpfs console=ttyS0 ds=nocloud-net;seedfrom=http://10.0.12.2:32600/ init=/bin/bash /etc/fstab: # cat /etc/fstab # # This fstab is in an overlay. The real one can be found at # /media/root-ro/etc/fstab # The original entry for '/' and other mounts have been updated to be placed # under /media/root-ro. # To permanently modify this (or any other file), you should change-root into # a writable view of the underlying filesystem using: # sudo overlayroot-chroot # #LABEL=cloudimg-rootfs /media/root-ro/ ext4 ro,defaults,noauto 0 0 /media/root-ro/ / overlay lowerdir=/media/root-ro/,upperdir=/media/root-rw/over0 #LABEL=UEFI /boot/efi vfat defaults 0 0 /root/root-ro/etc/fstab: LABEL=cloudimg-rootfs / ext4 defaults 0 0 #LABEL=UEFI /boot/efi vfat defaults 0 0 /proc/mounts: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=233748k,nr_inodes=58437,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,noexec,relatime,size=49288k,mode=755 0 0 /dev/disk/by-path/ip-10.0.12.2:3260-iscsi-tgt-boot-test-2abbnj-lun-1-part1 /media/root-ro ext4 ro,relatime 0 0 tmpfs-root /media/root-rw tmpfs rw,relatime 0 0 overlayroot / overlay ro,relatime,lowerdir=/media/root-ro,upperdir=/media/root-rw/overlay,workdir=/media/root-rw/overlay-workdir/_ 0 0 /proc/1/mountinfo: 21 28 0:20 / /sys rw,nosuid,nodev,noexec,relatime - sysfs sysfs rw 22 28 0:4 / /proc rw,nosuid,nodev,noexec,relatime - proc proc rw 23 28 0:6 / /dev rw,nosuid,relatime - devtmpfs udev rw,size=233748k,nr_inodes=58437,mode=755 24 23 0:21 / /dev/pts rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=000 25 28 0:22 / /run rw,nosuid,noexec,relatime - tmpfs tmpfs rw,size=49288k,mode=755 26 28 8:1 / /media/root-ro ro,relatime - ext4 /dev/disk/by-path/ip-10.0.12.2:3260-iscsi-tgt-boot-test-2abbnj-lun-1-part1 ro 27 28 0:23 / /media/root-rw rw,relatime - tmpfs tmpfs-root rw 28 0 0:24 / / ro,relatime - overlay overlayroot ro,lowerdir=/media/root-ro,upperdir=/media/root-rw/overlay,workdir=/media/root-rw/overlay-workdir/_ overlayroot's scripts/init-bottom/overlayroot script [1] inherits a read-only mount of block device on mount point '$ROOTMNT' (ROOTMNT=/root). In order to set up the overlayroot, it does the following: mkdir /media/root-ro /media/root-rw # in the initramfs mount -t tmpfs tmpfs-root /media/root-rw mkdir /media/root-rw/overlay-workdir/_ mount --move $ROOTMNT /media/root-ro mount -t overlay -o lowerdir=/media/root-ro,upperdir=/media/root-rw/overlay,workdir=/media/root-rw/overlay-workdir/_ overlayroot /root mkdir $ROOTMNT/media/root-ro mkdir $ROOTMNT/media/root-rw mount --move /media/root-ro "${ROOTMNT}/media/root-ro" mount --move /media/root-rw "${ROOTMNT}/media/root-rw" # then, if 'ro' on the command line, it mounts /root read-only. mount -o remount,ro $ROOTMNT The script then exits, as ROOTMNT is now set up with a read-only mount of the overlayroot. All the mounts it has done have been moved under ROOTMNT. On failure systemd reports: [ 104.098833] systemd[1]: media-root\x2dro.mount: Found ordering cycle on -.mount/start [ 104.109897] systemd[1]: media-root\x2dro.mount: Found dependency on media-root\x2dro.mount/start [ 104.121386] systemd[1]: media-root\x2dro.mount: Unable to break cycle starting with media-root\x2dro.mount/start [ 104.137591] systemd[1]: Requested transaction contains an unfixable cyclic ordering dependency: Resource deadlock avoided On successful boot, we can login and see: $ find /run/systemd/ -name "*.mount" | xargs ls -l -rw-r--r-- 1 root root 322 Aug 21 13:55 /run/systemd/generator/-.mount lrwxrwxrwx 1 root root 10 Aug 21 13:55 /run/systemd/generator/local-fs.target.requires/-.mount -> ../-.mount lrwxrwxrwx 1 root root 32 Aug 21 13:55 /run/systemd/units/invocation:dev-hugepages.mount -> b847fc2d06b54b1ab86a6f526d149522 lrwxrwxrwx 1 root root 32 Aug 21 13:55 /run/systemd/units/invocation:dev-mqueue.mount -> f991d3bfe35d4e2bb9aa86677ff31d70 lrwxrwxrwx 1 root root 32 Aug 21 13:55 /run/systemd/units/invocation:sys-fs-fuse-connections.mount -> a561df591a28437f990f94269fa08be9 lrwxrwxrwx 1 root root 32 Aug 21 13:55 /run/systemd/units/invocation:sys-kernel-config.mount -> 141e9bbaca9041e48458749e29bc58fb lrwxrwxrwx 1 root root 32 Aug 21 13:55 /run/systemd/units/invocation:sys-kernel-debug.mount -> a406163f265c4b02b37e6f92ddbfc029 $ for f in $(find /run/systemd/ -name '*.mount' -type f); do echo == $f ==; cat $f; done == /run/systemd/generator/-.mount == # Automatically generated by systemd-fstab-generator [Unit] SourcePath=/etc/fstab Documentation=man:fstab(5) man:systemd-fstab-generator(8) Before=local-fs.target [Mount] Where=/ What=/media/root-ro/ Type=overlay Options=lowerdir=/media/root-ro/,upperdir=/media/root-rw/overlay/,workdir=/media/root-rw/overlay-workdir/_ -- [1] https://git.launchpad.net/cloud-initramfs-tools/tree/overlayroot/scripts/init-bottom/overlayroot Related bugs: * bug 1723183: [overlayroot] transient systemd ordering issue when using overlayroot ProblemType: Bug DistroRelease: Ubuntu 18.10 Package: systemd 237-3ubuntu10 ProcVersionSignature: Ubuntu 4.17.0-6.7-generic 4.17.9 Uname: Linux 4.17.0-6-generic x86_64 ApportVersion: 2.20.10-0ubuntu7 Architecture: amd64 Date: Tue Aug 21 14:06:24 2018 Lsusb: Error: command ['lsusb'] failed with exit code 1: MachineType: QEMU Standard PC (i440FX + PIIX, 1996) ProcEnviron: TERM=vt220 PATH=(custom, no user) LANG=C.UTF-8 SHELL=/bin/bash ProcKernelCmdLine: nomodeset iscsi_initiator=maas-enlist iscsi_target_name=tgt-boot-test-2abbnj iscsi_target_ip=10.0.12.2 iscsi_target_port=3260 iscsi_initiator=maas-enlist ip=::::maas-enlist:BOOTIF ro net.ifnames=0 BOOTIF_DEFAULT=eth0 root=/dev/disk/by-path/ip-10.0.12.2:3260-iscsi-tgt-boot-test-2abbnj-lun-1-part1 overlayroot=tmpfs console=ttyS0 ds=nocloud-net;seedfrom=http://10.0.12.2:32600/ init=/bin/bash SourcePackage: systemd UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 04/01/2014 dmi.bios.vendor: SeaBIOS dmi.bios.version: 1.11.1-1 dmi.chassis.type: 1 dmi.chassis.vendor: QEMU dmi.chassis.version: pc-i440fx-cosmic dmi.modalias: dmi:bvnSeaBIOS:bvr1.11.1-1:bd04/01/2014:svnQEMU:pnStandardPC(i440FX+PIIX,1996):pvrpc-i440fx-cosmic:cvnQEMU:ct1:cvrpc-i440fx-cosmic: dmi.product.name: Standard PC (i440FX + PIIX, 1996) dmi.product.version: pc-i440fx-cosmic dmi.sys.vendor: QEMU To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1788188/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp