This is a VM boot optimisation for Linux systems with transparent huge
pages enabled for shared memory. It shaves about 1.4s off a 13s boot.
For nested VMs it reduces a 22.5s boot by about 4.6s.

It applies to VMs using memory-backend-memfd, which LXD uses by default.
Other backends can already use huge pages [1], but the memfd backend
only will if allocations are aligned correctly. This patch fixes that.

A drawback is that pc-dimm devices need to have sizes compatible with
the new alignment. I could have limited the change to x86_64 + KVM only,
which is the only configuration I've tested. It's already enabled for
s390x + KVM. But I think any system can benefit from using huge pages.

Disclaimer: I found this issue with LLM assistance. It located the part
of the code responsible for alignment in the memfd backend, but I typed
out the change myself.

[1] 
https://linuxcontainers.org/incus/docs/main/explanation/performance_tuning/#virtual-machine-memory-allocation

Signed-off-by: Jonathan Conder <[email protected]>
---
Tested using the following script. It's AI-generated but I can vouch
that it does the right thing.

To run it, build QEMU, download a Debian image [2] and set
/sys/kernel/mm/transparent_hugepage/shmem_enabled to "advise." Any image
should work, provided you supply any flags it needs to boot properly.

I ran it 3 times per configuration. Results:

Before (bare metal):
boot: 13135ms 12918ms 13046ms
huge pages mapped: 0 0 0

After (bare metal):
boot: 11694ms 11647ms 11687ms
huge pages mapped: 264 269 265

Before (inside another VM):
boot: 22528ms 22232ms 22327ms
huge pages mapped: 0 0 0

After (inside another VM):
boot: 17498ms 17677ms 18085ms
huge pages mapped: 265 271 266

Reproducer:
#!/bin/bash
# Time how long a VM using memfd-backed RAM takes to boot.

set -eo pipefail

build=${1:-build}
image=${2:-debian-13-nocloud-amd64-20261103-2651.qcow2}

huge_pages_mapped() {
        awk '$1 == "thp_file_mapped" { print $2 }' /proc/vmstat
}

fifo=$(mktemp -u)
mkfifo "$fifo"
trap 'rm -f "$fifo"' EXIT

before=$(huge_pages_mapped)
start=$(date +%s%N)

"$build/qemu-system-x86_64" \
        -accel kvm -m 4G \
        -machine q35,memory-backend=ram \
        -object memory-backend-memfd,id=ram,size=4G \
        -nodefaults -display none -serial stdio -snapshot \
        -drive if=pflash,format=raw,file="$build/pc-bios/edk2-x86_64-code.fd" \
        -drive if=virtio,file="$image" \
        -smbios type=11,value=io.systemd.credential:firstboot.timezone=UTC \
        -smbios 
type=11,value=io.systemd.credential:passwd.hashed-password.root= \
        </dev/null >"$fifo" &
grep -a -q -m1 'Reached target .*graphical\.target' "$fifo"

end=$(date +%s%N)
after=$(huge_pages_mapped)

kill -KILL $!
wait $! 2>/dev/null || true

echo "boot: $(( (end - start) / 1000000 ))ms, huge pages mapped: $(( after - 
before ))"

[2] 
https://cloud.debian.org/images/cloud/trixie/20261103-2651/debian-13-nocloud-amd64-20261103-2651.qcow2

 system/physmem.c | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/system/physmem.c b/system/physmem.c
index a7300bb9d9..758469bb3a 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -1730,11 +1730,7 @@ static void *file_ram_alloc(RAMBlock *block,
         return NULL;
     }
     block->mr->align = MAX(block->page_size, block->mr->align);
-#if defined(__s390x__)
-    if (kvm_enabled()) {
-        block->mr->align = MAX(block->mr->align, QEMU_VMALLOC_ALIGN);
-    }
-#endif
+    block->mr->align = MAX(block->mr->align, QEMU_VMALLOC_ALIGN);
 
     if (memory < block->page_size) {
         error_setg(errp, "memory size 0x" RAM_ADDR_FMT " must be equal to "
-- 
2.43.0


Reply via email to