This is a VM boot optimisation for Linux systems with transparent huge pages enabled for shared memory. It shaves about 1.4s off a 13s boot. For nested VMs it reduces a 22.5s boot by about 4.6s.
It applies to VMs using memory-backend-memfd, which LXD uses by default. Other backends can already use huge pages [1], but the memfd backend only will if allocations are aligned correctly. This patch fixes that. A drawback is that pc-dimm devices need to have sizes compatible with the new alignment. I could have limited the change to x86_64 + KVM only, which is the only configuration I've tested. It's already enabled for s390x + KVM. But I think any system can benefit from using huge pages. Disclaimer: I found this issue with LLM assistance. It located the part of the code responsible for alignment in the memfd backend, but I typed out the change myself. [1] https://linuxcontainers.org/incus/docs/main/explanation/performance_tuning/#virtual-machine-memory-allocation Signed-off-by: Jonathan Conder <[email protected]> --- Tested using the following script. It's AI-generated but I can vouch that it does the right thing. To run it, build QEMU, download a Debian image [2] and set /sys/kernel/mm/transparent_hugepage/shmem_enabled to "advise." Any image should work, provided you supply any flags it needs to boot properly. I ran it 3 times per configuration. Results: Before (bare metal): boot: 13135ms 12918ms 13046ms huge pages mapped: 0 0 0 After (bare metal): boot: 11694ms 11647ms 11687ms huge pages mapped: 264 269 265 Before (inside another VM): boot: 22528ms 22232ms 22327ms huge pages mapped: 0 0 0 After (inside another VM): boot: 17498ms 17677ms 18085ms huge pages mapped: 265 271 266 Reproducer: #!/bin/bash # Time how long a VM using memfd-backed RAM takes to boot. set -eo pipefail build=${1:-build} image=${2:-debian-13-nocloud-amd64-20261103-2651.qcow2} huge_pages_mapped() { awk '$1 == "thp_file_mapped" { print $2 }' /proc/vmstat } fifo=$(mktemp -u) mkfifo "$fifo" trap 'rm -f "$fifo"' EXIT before=$(huge_pages_mapped) start=$(date +%s%N) "$build/qemu-system-x86_64" \ -accel kvm -m 4G \ -machine q35,memory-backend=ram \ -object memory-backend-memfd,id=ram,size=4G \ -nodefaults -display none -serial stdio -snapshot \ -drive if=pflash,format=raw,file="$build/pc-bios/edk2-x86_64-code.fd" \ -drive if=virtio,file="$image" \ -smbios type=11,value=io.systemd.credential:firstboot.timezone=UTC \ -smbios type=11,value=io.systemd.credential:passwd.hashed-password.root= \ </dev/null >"$fifo" & grep -a -q -m1 'Reached target .*graphical\.target' "$fifo" end=$(date +%s%N) after=$(huge_pages_mapped) kill -KILL $! wait $! 2>/dev/null || true echo "boot: $(( (end - start) / 1000000 ))ms, huge pages mapped: $(( after - before ))" [2] https://cloud.debian.org/images/cloud/trixie/20261103-2651/debian-13-nocloud-amd64-20261103-2651.qcow2 system/physmem.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/system/physmem.c b/system/physmem.c index a7300bb9d9..758469bb3a 100644 --- a/system/physmem.c +++ b/system/physmem.c @@ -1730,11 +1730,7 @@ static void *file_ram_alloc(RAMBlock *block, return NULL; } block->mr->align = MAX(block->page_size, block->mr->align); -#if defined(__s390x__) - if (kvm_enabled()) { - block->mr->align = MAX(block->mr->align, QEMU_VMALLOC_ALIGN); - } -#endif + block->mr->align = MAX(block->mr->align, QEMU_VMALLOC_ALIGN); if (memory < block->page_size) { error_setg(errp, "memory size 0x" RAM_ADDR_FMT " must be equal to " -- 2.43.0
