** Changed in: lvm2 (Ubuntu Bionic) Status: Confirmed => In Progress
** Changed in: lvm2 (Ubuntu Disco) Status: Confirmed => In Progress ** Changed in: lvm2 (Ubuntu Disco) Assignee: (unassigned) => Eric Desrochers (slashd) ** Changed in: lvm2 (Ubuntu Bionic) Assignee: (unassigned) => Eric Desrochers (slashd) -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to lvm2 in Ubuntu. https://bugs.launchpad.net/bugs/1854981 Title: system doesn't properly boot as expected if /usr is on its own LV Status in initramfs-tools package in Ubuntu: Won't Fix Status in lvm2 package in Ubuntu: Fix Released Status in initramfs-tools source package in Xenial: Won't Fix Status in lvm2 source package in Xenial: Won't Fix Status in initramfs-tools source package in Bionic: Won't Fix Status in lvm2 source package in Bionic: In Progress Status in initramfs-tools source package in Disco: Won't Fix Status in lvm2 source package in Disco: In Progress Bug description: [Impact] In summary, the problem is a LVM 'auto-activation' issue. Basically the LVM activation doesn't occurs during the boot process (except for LVM root partition) and since /usr is an important component before the 'pivot'[0], the init inside the initramfs space tries to mount /usr. If /usr is a LVM volume then it will try to mount it while its backend device is not activated yet (due to the actual auto-activation issue), thus the mounting will fails and bring the user to the initramfs prompt for debugging. Workaround: At the initramfs prompt: lvm vgchange -ay and then 'ctrl-d' to resume the boot process. [Test Case] - Install Bionic or Disco - Create a VG with /usr on its on LV - Complete the installation - At reboot the system will be stuck at mounting /usr file system Extra notes: Since this problem is ONLY related to LVM activation. If one create a non-LVM /usr separate partition, everything will work as expected. Other separate LVM (e.g. /home, /var/, /srv) if on its own separate LVM volume will not be activate either, but since they don't need to be mounted in the initramfs space, it's not a problem, they can be activated later on (after the pivot) by systemd. So the only combination possible is when /usr is on its separate LV only. [Regression potential] I don't foresee any regression, the fix will instruct udev rule to pvscan activate volume based on their major/minor number if LVM is scanned. +RUN+="(LVM_EXEC)/lvm pvscan --cache --activate ay --major $major --minor $minor", ENV{LVM_SCANNED}="1" The current situation doesn't auto-activate at all. One downside, user will see this fixed, will still experience the situation in standard ISO, since I'm afraid there is no more point release for Bionic and Disco. So one would need to rely on the mini.iso for fetching the latest lvm2 package in the archive. or use the workaround in the [Impact] section and then apt-get dist- upgrade in order to get the latest lvm2. [Other information] This problem only exhibit in Bionic and Disco. https://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMerge/ https://wiki.debian.org/UsrMerge [Original Description] Only the lv for root volume get activated, because of the grub parameter that specifies/enforce it "root=/dev/mapper/ubuntu-vg-ubuntu --lv" http://man7.org/linux/man-pages/man7/bootparam.7.html 'root=...' This argument tells the kernel what device is to be used as the root filesystem while booting. If one add a separate LV for /usr, the system will go straight to initramfs prompt failling to mount /usr. At initramfs prompt, we notice that 'lv-usr' status is 'NOT available'. Performing 'lvm vgchange -ay' at initramfs prompt workaround the problem and allow one to successfully boot. Adding 'debug' parameter, we clearly we see /root being detected and mounted: initramfs.debug: => + mount -r -t ext4 /dev/mapper/ubuntu--vg-ubuntu--lv /root => + mountroot_status=0 + [ ] + log_end_msg + _log_msg done.\n + [ n = y ] + printf done.\n done. + read_fstab_entry /usr + found=1 + [ -f /root/etc/fstab ] + read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK + [ / = /usr ] + read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK + [ /usr = /usr ] + [ -n ] + found=0 + break 2 + return 0 + log_begin_msg Mounting /usr file system + _log_msg Begin: Mounting /usr file system ... then the code read /etc/fstab and specifically search for /usr (most likely because of the /usr binary merged) and try to mount if if found. initramfs-tools:init 271 maybe_break mount 272 log_begin_msg "Mounting root file system" 273 # Always load local and nfs (since these might be needed for /etc or 274 # /usr, irrespective of the boot script used to mount the rootfs). 275 . /scripts/local 276 . /scripts/nfs 277 . /scripts/${BOOT} 278 parse_numeric "${ROOT}" 279 maybe_break mountroot 280 mount_top 281 mount_premount 282 mountroot 283 log_end_msg 284 => 285 if read_fstab_entry /usr; then => 286 log_begin_msg "Mounting /usr file system" => 287 mountfs /usr => 288 log_end_msg => 289 fi In this case, /usr is present in /etc/fstab but the logical volume is not available, so it is mounting a filesystem without his backend device, thus goes straight to the initramfs prompt because /usr couldn't be mounted. It clearly seems to be an 'auto-activation' issue at boot. For other such as /home, /var, it's not a big deal cause they can be activated later on in the process and they are (I haven't check but I guess systemd or systemd unit/generator is taking care of it at some point), but /usr is a important piece to have mounted before the pivot since it contains most of the crucial binary, especially that nowadays /sbin, /bin, and /lib are pointing to /usr: lrwxrwxrwx 1 root root 10 Aug 26 13:51 libx32 -> usr/libx32 lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib64 -> usr/lib64 lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib32 -> usr/lib32 lrwxrwxrwx 1 root root 7 Aug 26 13:51 lib -> usr/lib lrwxrwxrwx 1 root root 7 Aug 26 13:51 bin -> usr/bin lrwxrwxrwx 1 root root 8 Aug 26 13:51 sbin -> usr/sbin NOTE: * That doesn't affect /usr found in /etc/fstab on its separate partition when no LVM involve (e.g. /dev/vdb /usr ext4 ....). It only cause issue when /usr is in a LVM context. * I was able to reproduce on Bionic and Disco so far. Eoan doesn't seem to exhibit the situation so far in my testing. * While certain release such as Bionic, Xenial doesn't come implicitly with the /usr merge approach. One can install package 'usrmerge' and convert their system /usr merged. I don't think removing the read_fstable_entry for /usr is an option here, as some user could potentially decide to convert their system with 'usrmerge' pkg. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/initramfs-tools/+bug/1854981/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp