Bug#902943: cryptsetup-initramfs: Encrypted rootfs in LVM is not found after upgrade
On Fri, 06 Jul 2018 at 00:51:02 +0200, Korbinian Demmel wrote: > I played around with the 'lvm2' script. Support to get the mangled > source device (LG/LV) for an given LV UUID is quite simple to add. It's not the LV UUID that we need here, but the LUKS UUID. And the script doesn't know which device to activate, so it'd need to activate all of them. But indeed it's quite simple to add, for instance by adding a default option in the `case "$dev" in … esac`: *) lvchange_activate;; We can do better however, and activate only the required LV rather than all of them. The boot script can't map the block device UUID to a LV, but on `update-initramfs -u` our hook can map it since at that point the LV is activated. In the upcoming 2:2.0.3-5 the hook substitutes the mapped source devices with /dev/mapper/$SOURCENAME in the list of devices to unlock at initramfs stage, which should work around this issue. >> I'd argue that this is a bug in lvm2's local-block script. The very >> same problem happens if /usr is held by a dedicated LV (whether / is >> held by a different LV in the same VG, in another VG, or by a non mapped >> device) and the FS is specified using its UUID in the fstab(5). > > My feeling is that the whole 'initramfs' process to get the rootfs is > not iterative (works not recursively) and is in some way "hard-wired" > for usual use cases. Some time ago I fiddled around with 'initramfs' > itself to support an image with an encrypted rootfs directly. To > accomplish that I added support for a list of arguments for 'rootfs', > 'rootflags', etc. But eventually it just did not work due to the > non-recursive architecture of initramfs. It is recursive now. Once we have we have the list of block devices holding the rootfs (for btrfs it's in /sys/fs/btrfs/$UUID/devices/*, for traditional file systems it's /proc/mounts' first column) we recursively traverse the sysfs hierarchy to find the list of underlying dm-crypt devices. It works for LV — dm-crypt — MD stacks in whichever combination. >> But coming up with our own logic to activate VGs is beyond the scope of >> cryptsetup IMHO. > > Unfortunately I am lost regarding the original issue: > As far as I understand, there is currently no way to use an encrypted > rootfs baked by LVM. Would be nice if that is mentioned in the > changelog at least. Or did I miss that? No you didn't, otherwise I would have closed the issue and not retitle it as a regression and tagged at as pending :-) 2:2.0.3-5 will contain a workaround. > Since I not really need LVM, I probably will just get rid of it for now. I'll upload 2:2.0.3-5 in the next day or two. As a workaround, you should be able to boot if you either 1/ add the catchall to the lvm2 local-block script; or 2/ use /dev/mapper/linux-debian as source in your crypttab *and* remove lines 131-132 in the cryptroot hook file: https://sources.debian.org/src/cryptsetup/2:2.0.3-4/debian/initramfs/hooks/cryptroot/#L131 -- Guilhem. signature.asc Description: PGP signature
Bug#902943: cryptsetup-initramfs: Encrypted rootfs in LVM is not found after upgrade
Hello Guilhem, thank you very much for the detailed response. Not forever, though. It drops to a debug shell after ‘rootdelay’ (default 180) seconds, [...] Right. Too long to wait ;) I read through your mail, but obviously lack some context. I played around with the 'lvm2' script. Support to get the mangled source device (LG/LV) for an given LV UUID is quite simple to add. But I ran into the issue that the very same UUID is also used afterwards for 'cryptsetup', which obviously won't work. I'd argue that this is a bug in lvm2's local-block script. The very same problem happens if /usr is held by a dedicated LV (whether / is held by a different LV in the same VG, in another VG, or by a non mapped device) and the FS is specified using its UUID in the fstab(5). My feeling is that the whole 'initramfs' process to get the rootfs is not iterative (works not recursively) and is in some way "hard-wired" for usual use cases. Some time ago I fiddled around with 'initramfs' itself to support an image with an encrypted rootfs directly. To accomplish that I added support for a list of arguments for 'rootfs', 'rootflags', etc. But eventually it just did not work due to the non-recursive architecture of initramfs. But coming up with our own logic to activate VGs is beyond the scope of cryptsetup IMHO. Unfortunately I am lost regarding the original issue: As far as I understand, there is currently no way to use an encrypted rootfs baked by LVM. Would be nice if that is mentioned in the changelog at least. Or did I miss that? Since I not really need LVM, I probably will just get rid of it for now. Regards, doak On 03.07.2018 22:58, Guilhem Moulin wrote: On Tue, 03 Jul 2018 at 20:01:02 +0200, doak wrote: After system upgrade the system is not bootable anymore due the initramfs is unable to find the "source" for the rootfs and boot hangs. Not forever, though. It drops to a debug shell after ‘rootdelay’ (default 180) seconds, unless you've set the ‘panic=’ boot parameter. linux-debian_crypt UUID=979d71b4-f9a9-45cb-b72e-6c81754ab504 none luks We're now relying on lvm2's /scripts/local-block/lvm2 to activate the VG holding our source device, which doesn't work with UUIDs: https://sources.debian.org/src/lvm2/2.02.176-4.1/debian/initramfs-tools/lvm2/scripts/local-block/lvm2/ I'd argue that this is a bug in lvm2's local-block script. The very same problem happens if /usr is held by a dedicated LV (whether / is held by a different LV in the same VG, in another VG, or by a non mapped device) and the FS is specified using its UUID in the fstab(5). If you specify the source device as /dev/mapper/linux-debian or /dev/linux/debian then you should be able to boot. “Should”, because for LUKS we're using UUIDs internally, so right now this won't work. When the source device is mapped we can use its (mangled) name instead. But coming up with our own logic to activate VGs is beyond the scope of cryptsetup IMHO. signature.asc Description: OpenPGP digital signature
Bug#902943: cryptsetup-initramfs: Encrypted rootfs in LVM is not found after upgrade
On Tue, 03 Jul 2018 at 20:01:02 +0200, doak wrote: > After system upgrade the system is not bootable anymore due the > initramfs is unable to find the "source" for the rootfs and boot > hangs. Not forever, though. It drops to a debug shell after ‘rootdelay’ (default 180) seconds, unless you've set the ‘panic=’ boot parameter. > linux-debian_crypt UUID=979d71b4-f9a9-45cb-b72e-6c81754ab504 none luks We're now relying on lvm2's /scripts/local-block/lvm2 to activate the VG holding our source device, which doesn't work with UUIDs: https://sources.debian.org/src/lvm2/2.02.176-4.1/debian/initramfs-tools/lvm2/scripts/local-block/lvm2/ I'd argue that this is a bug in lvm2's local-block script. The very same problem happens if /usr is held by a dedicated LV (whether / is held by a different LV in the same VG, in another VG, or by a non mapped device) and the FS is specified using its UUID in the fstab(5). If you specify the source device as /dev/mapper/linux-debian or /dev/linux/debian then you should be able to boot. “Should”, because for LUKS we're using UUIDs internally, so right now this won't work. When the source device is mapped we can use its (mangled) name instead. But coming up with our own logic to activate VGs is beyond the scope of cryptsetup IMHO. -- Guilhem. signature.asc Description: PGP signature
Bug#902943: cryptsetup-initramfs: Encrypted rootfs in LVM is not found after upgrade
Package: cryptsetup-initramfs Version: 2:2.0.3-4 Severity: critical Justification: breaks the whole system Dear Maintainer, After system upgrade the system is not bootable anymore due the initramfs is unable to find the "source" for the rootfs and boot hangs. Before the upgrade this worked seamlessly for a long time, UUID of FS did not change recently. When breaking into initramfs (kernel argument 'break'), I am able to activate the LVM logical volume manually with 'lvm lvchange -a y linux'. Resuming the boot after that works, i.e. the passphrase is asked and boot continues after successful decryption. Please let me know if I should provide more detailed debug information. Regards, doak -- Package-specific info: -- /proc/cmdline BOOT_IMAGE=/vmlinuz-4.16.0-2-amd64 root=/dev/mapper/linux-debian_crypt ro quiet -- /etc/crypttab linux-debian_crypt UUID=979d71b4-f9a9-45cb-b72e-6c81754ab504 none luks -- /etc/fstab # /etc/fstab: static file system information. # # Use 'blkid' to print the universally unique identifier for a # device; this may be used with UUID= as a more robust way to name devices # that works even if disks are added and removed. See fstab(5). # # /dev/mapper/linux-debian_crypt / ext4errors=remount-ro 0 1 # /boot was on /dev/sda5 during installation UUID=42edef9b-e7dc-4ceb-a5e1-05d958d6ea6f /boot ext4 defaults 0 2 /SWAP noneswapdefaults0 0 -- lsmod Module Size Used by ctr16384 6 ccm20480 9 cmac 16384 0 rfcomm 86016 0 fuse 118784 3 ipt_MASQUERADE 16384 1 nf_nat_masquerade_ipv416384 1 ipt_MASQUERADE nf_conntrack_netlink49152 0 nfnetlink 16384 2 nf_conntrack_netlink xfrm_user 45056 1 xfrm_algo 16384 1 xfrm_user iptable_nat16384 1 nf_conntrack_ipv4 16384 3 nf_defrag_ipv4 16384 1 nf_conntrack_ipv4 nf_nat_ipv416384 1 iptable_nat xt_addrtype16384 2 xt_conntrack 16384 1 nf_nat 36864 2 nf_nat_masquerade_ipv4,nf_nat_ipv4 nf_conntrack 151552 7 nf_conntrack_ipv4,ipt_MASQUERADE,nf_conntrack_netlink,nf_nat_masquerade_ipv4,xt_conntrack,nf_nat_ipv4,nf_nat libcrc32c 16384 2 nf_conntrack,nf_nat br_netfilter 24576 0 bridge184320 1 br_netfilter stp16384 1 bridge llc16384 2 bridge,stp bnep 24576 2 overlay 102400 0 ebtable_filter 16384 0 ebtables 36864 1 ebtable_filter devlink61440 0 ip6table_filter16384 0 ip6_tables 32768 1 ip6table_filter iptable_filter 16384 1 snd_hda_codec_hdmi 57344 1 binfmt_misc20480 1 intel_rapl 24576 0 x86_pkg_temp_thermal16384 0 intel_powerclamp 16384 0 coretemp 16384 0 kvm_intel 180224 0 uvcvideo 106496 0 arc4 16384 2 kvm 704512 1 kvm_intel videobuf2_vmalloc 16384 1 uvcvideo videobuf2_memops 16384 1 videobuf2_vmalloc videobuf2_v4l2 28672 1 uvcvideo videobuf2_common 49152 2 uvcvideo,videobuf2_v4l2 irqbypass 16384 1 kvm iTCO_wdt 16384 0 cdc_mbim 16384 0 wmi_bmof 16384 0 btusb 53248 0 iwlmvm290816 0 iTCO_vendor_support16384 1 iTCO_wdt videodev 204800 3 uvcvideo,videobuf2_common,videobuf2_v4l2 mac80211 802816 1 iwlmvm cdc_wdm24576 1 cdc_mbim btrtl 16384 1 btusb intel_cstate 16384 0 btbcm 16384 1 btusb media 45056 2 uvcvideo,videodev cdc_ncm40960 1 cdc_mbim btintel24576 1 btusb intel_uncore 131072 0 usbnet 49152 2 cdc_mbim,cdc_ncm iwlwifi 225280 1 iwlmvm i915 1650688 20 bluetooth 626688 14 btrtl,btintel,bnep,btbcm,rfcomm,btusb mii16384 1 usbnet cdc_acm32768 2 intel_rapl_perf16384 0 evdev 28672 29 joydev 24576 0 snd_hda_codec_realtek 110592 1 serio_raw 16384 0 snd_hda_codec_generic86016 1 snd_hda_codec_realtek pcspkr 16384 0 snd_hda_intel 45056 8 drbg 28672 1 ansi_cprng 16384 0 snd_hda_codec 151552 4 snd_hda_intel,snd_hda_codec_hdmi,snd_hda_codec_generic,snd_hda_codec_realtek ecdh_generic 24576 1 bluetooth cfg80211 737280 3 iwlmvm,iwlwifi,mac80211 sg 36864 0 rtsx_pci_ms20480 0 snd_hda_core 94208 5
