This series adds batch register access support to QEMU's KVM GICv3
register synchronization path, reducing syscall overhead during
ARM64 live migration.
Currently, QEMU issues one ioctl per register when saving/restoring VGIC
state. On large VM configurations this means tens of thousands of syscalls,
where lock acquisition and context switch overhead dominates migration
downtime. Thus, we provide a batch register method to allow userspace
read/write multiple distributor and redistributor registers in a single call.
In this way, we can significantly reduce syscalls and migration downtime.
All batch paths probe kernel capability at runtime and fall back to the
original one-by-one path when the kernel does not support batch access,
ensuring backward compatibility.
This series depends on the corresponding kernel patches that add
KVM_DEV_ARM_VGIC_GRP_BATCH_REGS support.
Test the VM migration time under pressure conditions.
The VM specifications for migration are as follows:
- VM use 4-K page;
- the number of VCPU is 160;
- the total memory is 320Gigabit;
- use 'Redis SET-benchmark' to pressurize VM;
Performance results (3-run average, ms):
| Metric | Without patch | With patch | Improvement |
|---------------------|---------------|------------|-------------|
| Migration downtime | 536 | 321 | 40% |
| Source (total) | 344 | 230 | 33% |
| - VGIC put | 158 | 40 | 75% |
| - VGIC get | 120 | 19 | 84% |
| Destination (total) | 192 | 91 | 53% |
| - VGIC put | 132 | 27 | 80% |
Yize Wang (4):
Sync KVM ARM VGIC batch register UAPI to QEMU headers
hw/intc/arm_gicv3_kvm: Add batch helper structures and functions
hw/intc/arm_gicv3_kvm: Refactor kvm_arm_gicv3_put for batch register access
hw/intc/arm_gicv3_kvm: Refactor kvm_arm_gicv3_get for batch register access
hw/intc/arm_gicv3_kvm.c | 1087 ++++++++++++++++++++++++++++++++++-------
linux-headers/asm-arm64/kvm.h | 9 +
2 files changed, 925 insertions(+), 171 deletions(-)
References:
[1] [PATCH 0/2] Batch register access for live migration optimization
https://lore.kernel.org/kvmarm/[email protected]/T/#t
