I successfully verified the SRU on my Raspberry Pi Zero 2W: ``` bdrung@zero2w:~$ sudo cp /boot/initrd.img-6.8.0-1007-raspi /boot/initrd.img-6.8.0-1007-raspi.current-noble bdrung@zero2w:~$ sudo sh -c '3cpio -t /boot/initrd.img-6.8.0-1007-raspi.current-noble > /boot/initrd.img-6.8.0-1007-raspi.current-noble.files' bdrung@zero2w:~$ sudo apt install dracut-install=060+5-1ubuntu3.2 [...] bdrung@zero2w:~$ sudo sh -c '3cpio -t /boot/initrd.img-6.8.0-1007-raspi > /boot/initrd.img-6.8.0-1007-raspi.files' bdrung@zero2w:~$ diff -u /boot/initrd.img-6.8.0-1007-raspi.current-noble.files /boot/initrd.img-6.8.0-1007-raspi.files bdrung@zero2w:~$ sudo hyperfine --warmup 1 -r 10 "update-initramfs -u" Benchmark 1: update-initramfs -u Time (mean ± σ): 232.001 s ± 5.678 s [User: 55.456 s, System: 166.510 s] Range (min … max): 222.120 s … 239.610 s 10 runs
bdrung@zero2w:~$ sudo apt install -t noble-proposed cryptsetup-initramfs lvm2 open-iscsi overlayroot thin-provisioning-tools initramfs-tools [...] bdrung@zero2w:~$ sudo hyperfine --warmup 1 -r 10 "update-initramfs -u" [sudo] password for bdrung: Benchmark 1: update-initramfs -u Time (mean ± σ): 214.585 s ± 6.790 s [User: 46.486 s, System: 157.656 s] Range (min … max): 206.479 s … 225.762 s 10 runs bdrung@zero2w:~$ dpkg -l | grep -E '(cryptsetup|initramfs-tools|dracut|open-iscsi|lvm2|thin-provisioning-tools|overlayroot)' ii cryptsetup 2:2.7.0-1ubuntu4.1 arm64 disk encryption support - startup scripts ii cryptsetup-bin 2:2.7.0-1ubuntu4.1 arm64 disk encryption support - command line tools ii cryptsetup-initramfs 2:2.7.0-1ubuntu4.1 all disk encryption support - initramfs integration ii dracut-install 060+5-1ubuntu3.2 arm64 dracut is an event driven initramfs infrastructure (dracut-install) ii initramfs-tools 0.142ubuntu25.2 all generic modular initramfs generator (automation) ii initramfs-tools-bin 0.142ubuntu25.2 arm64 binaries used by initramfs-tools ii initramfs-tools-core 0.142ubuntu25.2 all generic modular initramfs generator (core tools) ii libcryptsetup12:arm64 2:2.7.0-1ubuntu4.1 arm64 disk encryption support - shared library ii liblvm2cmd2.03:arm64 2.03.16-3ubuntu3.1 arm64 LVM2 command library ii lvm2 2.03.16-3ubuntu3.1 arm64 Linux Logical Volume Manager ii open-iscsi 2.1.9-3ubuntu5.1 arm64 iSCSI initiator tools ii overlayroot 0.48 all use an overlayfs on top of a read-only root filesystem ii thin-provisioning-tools 0.9.0-2ubuntu5.1 arm64 Tools for handling thinly provisioned device-mapper meta-data ``` ** Description changed: [ Impact ] When compared to Ubuntu 23.10, creating intramfs files with update- initramfs takes 2 to 5 times more time on ARM devices. IIUC, dracut-install usage was added to initramfs-tools to speed up the process. But now its way slower. Even running update-initramfs on jammy, which doesn't use dracut-install, is way faster then the time taken on Noble. first bad commit - https://github.com/dracutdevs/dracut/commit/3de4c7313260fb600507c9b87f780390b874c870 Updating the initrd on a Raspberry Pi Zero 2W on Ubuntu 24.04 (noble) with initramfs-tools 0.142ubuntu25.1 takes over six minutes: ``` bdrung@zero2w:~$ sudo hyperfine --warmup 1 -r 10 "update-initramfs -u" Benchmark 1: update-initramfs -u Time (mean ± σ): 402.751 s ± 5.592 s [User: 166.316 s, System: 228.909 s] Range (min … max): 394.380 s … 411.445 s 10 runs ``` [ Test Plan ] 1. Measure `update-initramfs -u` before the update. 2. Log the content of the initrd before the update: `lsinitramfs /boot/initrd.img` 3. update dracut-install / initramfs-tools-core 4. Measure `update-initramfs -u`. It should be faster (the performance improvements on amd64 should be very small and might be within the measurement uncertainty). 5. Check with lsinitramfs that the content of the newly generated initrd hasn't changed. [ Where problems could occur ] The code that is responsible for including the kernel modules into the initrd is touched. Negative consequences could be that some needed kernel modules will not be included any more (should be covered by the test case) or that building new initrds will fail. The initramfs-tools fix changes how manual_add_modules behaves. `manual_add_modules` does not copy kernel modules, but queues them for being copied when the newly added function `apply_add_modules` is called. I checked all instances of calls to `manual_add_modules` for possible regressions (see comment #15). Only miniramfs needs to be adjusted to also call `apply_add_modules`. But this change could break consumers of the `manual_add_modules` function that are outside of the Ubuntu archive. I googled for `apply_add_modules` but found no public outside users. [ Benchmarks ] Stock noble on a Raspberry Pi Zero 2W: ``` bdrung@zero2w:~$ sudo hyperfine -r 5 "update-initramfs -u" Benchmark 1: update-initramfs -u Time (mean ± σ): 415.664 s ± 6.015 s [User: 166.728 s, System: 232.523 s] Range (min … max): 409.139 s … 422.632 s 5 runs ``` noble with dracut-install 060+5-1ubuntu3.1 (with linux 6.8.0-1006.6 on 2024-07-01): ``` bdrung@zero2w:~$ sudo hyperfine --warmup 1 -r 10 "update-initramfs -u" Benchmark 1: update-initramfs -u Time (mean ± σ): 248.054 s ± 5.569 s [User: 67.410 s, System: 169.412 s] Range (min … max): 238.909 s … 257.384 s 10 runs ``` + noble with dracut-install 060+5-1ubuntu3.2 (with linux 6.8.0-1007 on + 2024-07-11): + + ``` + bdrung@zero2w:~$ sudo hyperfine --warmup 1 -r 10 "update-initramfs -u" + Benchmark 1: update-initramfs -u + Time (mean ± σ): 232.001 s ± 5.678 s [User: 55.456 s, System: 166.510 s] + Range (min … max): 222.120 s … 239.610 s 10 runs + ``` + [ Reduce manual_add_modules calls ] Besides making the dracut-install calls faster, group the dracut-install calls. Since the fix in oracular can cause regressions in custom hooks that rely on the current behavior, the SRU takes a safe approach which includes following packages (stating how many dracut-install calls are used): - * cryptsetup: 2 -> 1 - * lvm2: 8 -> 1 - * thin-provisioning-tools: 3 -> 1 - * open-iscsi: 9 -> 1 - * cloud-initramfs-tools: 5 -> 1 + * cryptsetup: 2 -> 1 + * lvm2: 8 -> 1 + * thin-provisioning-tools: 3 -> 1 + * open-iscsi: 9 -> 1 + * cloud-initramfs-tools: 5 -> 1 dracut-install calls on a Raspberry Pi Zero 2W: | area | before | noble SRU | oracular | |--------------------------------------|--------|-----------|----------| | auto_add_modules + apply_add_modules | 8 | 5 | 5 | | calls by hooks + apply_add_modules | 42 | 20 | 2 | | hidden_dep_add_modules | 1 | 1 | 1 | | total | 51 | 26 | 8 | [ Other Info ] $ lsb_release -rd No LSB modules are available. Description: Ubuntu 24.04 LTS Release: 24.04 $ apt-cache policy dracut-install dracut-install: Installed: 060+5-1ubuntu3 Candidate: 060+5-1ubuntu3 Version table: *** 060+5-1ubuntu3 500 500 http://ports.ubuntu.com/ubuntu-ports noble/main arm64 Packages 100 /var/lib/dpkg/status -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2065180 Title: performance regression in dracut-install 060 To manage notifications about this bug go to: https://bugs.launchpad.net/dracut/+bug/2065180/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
