(I have already posted to the libguestfs mailing list, without any luck) I run a number of virtualisation hosts, the host OS is Debian 10 (tho this issue has also been present with 9) and the virtualisation tech is KVM. All packages are current & from 'stable'.
I am encountering a problem where Debian guests, specifically built using `virt-resize`, work flawlessly until the first time `update- initramfs` is run, when they then fail to boot with the following error: "mount: mounting /dev/vda1 on /root failed: No such device" after which the system fails to the initramfs Busybox debug shell. If I use the `break=premount` kernel param to drop out of boots to the debug shell before and after `update-initramfs` is run, and then compare the output of `dmesg` it is clear that the necessary IO drivers are not being loaded after the initramfs has been rebuilt: BEFORE `update-initramfs` [ 0.736291] Run /init as init process [ 0.795060] lpc_ich 0000:00:1f.0: I/O space for GPIO uninitialized [ 0.800259] PCI Interrupt Link [GSIA] enabled at IRQ 16 [ 0.800363] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt [ 0.811428] cryptd: max_cpu_qlen set to 1000 [ 0.821407] AVX2 version of gcm_enc/dec engaged. [ 0.821407] AES CTR mode by8 optimization enabled [ 0.823702] ACPI: bus type USB registered [ 0.823712] usbcore: registered new interface driver usbfs [ 0.823717] usbcore: registered new interface driver hub [ 0.823726] usbcore: registered new device driver usb [ 0.829766] SCSI subsystem initialized [ 0.830240] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 0.830430] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input2 [ 0.844882] virtio_blk virtio2: [vda] 41943040 512-byte logical blocks (21.5 GB/20.0 GiB) [ 0.854231] vda: vda1 vda2 [ 0.857633] xhci_hcd 0000:02:00.0: xHCI Host Controller [ 0.857638] xhci_hcd 0000:02:00.0: new USB bus registered, assigned bus number 1 [ 0.857900] xhci_hcd 0000:02:00.0: hcc params 0x00087001 hci version 0x100 quirks 0x0000000000000010 [ 0.859088] virtio_net virtio0 enp1s0: renamed from eth0 [ 0.860055] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 4.19 [ 0.860056] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 0.860056] usb usb1: Product: xHCI Host Controller [ 0.860057] usb usb1: Manufacturer: Linux 4.19.0-9-amd64 xhci-hcd [ 0.860057] usb usb1: SerialNumber: 0000:02:00.0 ... AFTER `update-initramfs` [ 0.743940] Run /init as init process [ 0.801833] lpc_ich 0000:00:1f.0: I/O space for GPIO uninitialized [ 0.814263] cryptd: max_cpu_qlen set to 1000 [ 0.817594] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 0.817785] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input2 [ 0.820087] PCI Interrupt Link [GSIA] enabled at IRQ 16 [ 0.820187] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt [ 0.829008] AVX2 version of gcm_enc/dec engaged. [ 0.829008] AES CTR mode by8 optimization enabled [ 0.839844] virtio_blk virtio2: [vda] 41943040 512-byte logical blocks (21.5 GB/20.0 GiB) [ 0.843734] vda: vda1 vda2 [ 0.846319] virtio_net virtio0 enp1s0: renamed from eth0 (In the BEFORE case I have snipped the output of `dmesg` as it just goes on to load the expected drivers, however in the case of AFTER, that is the end of the `dmesg` output) My workflow to create guests is as follows: Initially manually install a minimal 'reference' guest to a small LVM backed disk containing two partitions (/ & swap). The rest is then scripted each time I want to 'clone' a new guest from this reference: * take an LVM snapshot of the reference host disk volume * `kpartx -a` on the snapshot, to gain access to the partitions, mount / and modify two files to set hostname and IP * `kpartx -d` * create a new LVM volume at the desired size to be used as the disk for the finished guest * `virt-resize` to copy the partitions from the snapshot to the new volume, resizing swap to a fixed size and letting `virt-resize` grow / to fill the remaining space * `kpartx -a` on the new volume and then `mkswap` on the new swap partition * `kpartx -d` * define the new host in KVM using the new volume as the backing disk * `virt-sysprep -d <hostname> --enable bash- history,customize,logfiles,mail-spool,package-manager-cache,ssh- hostkeys,tmp-files` to clean up ahead of first boot At this point the guest will operate flawlessly, will boot & reboot without issue. However when update-initramfs first runs (usually as part of a kernel upgrade, but this happens if it is run directly as well), the problem is then present on next boot. CentOS guests built the same way do not encounter this problem. If I skip the `virt-resize` step and either use the snapshot as the backing disk for the new guest, or `dd` the reference disk volume to another volume and use that, Debian guests work fine in all cases. This problem is present when both Debian 9 or 10 are used for either the host or the guests. So it appears that `virt-resize` triggers a behavior where `update- initramfs` does not build the initramfs correctly. Could anybody give me any pointers as to either what may be wrong, or the best route to troubleshoot this further? Many thanks in advance.
