This patch series implements two new Hyper-V hypercalls. The primary goal is to support "zeroed memory" enlightenment via HvExtCallGetBootZeroedMemory(). To do so, we also need to implement HvExtCallQueryCapabilities(). While HvExtCallQueryCapabilities() uses another bit in EBX on CPUID leaf 0x40000003, it then allows the guest to query for further "extended" hypercalls via the return value of that call.
HvExtCallGetBootZeroedMemory() allows a Windows guest to inquire which areas of memory that were provided by the hypervisor are already zeroed out, meaning that there's no need for the guest to do so again. This has obvious performance benefits. To get a rough idea of the performance benefit, looking at a 4-core, 64GB Windows 11 guest, we can see that an otherwise idle guest takes a bit less than 90 seconds to finish zeroing and settle down from the time the qemu process is started. If we look at CPU usage for each vCPU task 90 seconds after starting, we get: 1559 498 1294 343 1451 314 4015 2729 Conversely, after enabling HvExtCallGetBootZeroedMemory(), CPU usage is reduced: 1583 458 1441 361 1279 312 1337 264 These are taken from the respective /proc/<pid>/task/<tid>/stat entries, denoting user and system time per "CPU X/KVM" task, in ticks. We can clearly see which to which vCPU fell the tasks of zeroing. It also has the side benefit of not touching all pages at boot. This can be useful if, for example, the VM is started with a balloon target already set. In this case, we don't risk memory usage spikes on the host if the VM touches all memory before the balloon can catch up. Note that this is still an RFC series. There are a number of open questions, most importantly, which memory areas to send back to the guest as pre-zeroed. The Microsoft documentation is a bit sparse, but points out: > Ranges can include memory that don’t exist and can overlap. https://learn.microsoft.com/en-us/virtualization/hyper-v-on-windows/tlfs/hypercalls/hvextcallgetbootzeroedmemory So for testing purposes, I just set all memory from 0 to the end of the 64-bit address space as pre-zeroed. Anecdotally, this worked fine in my preliminary testing so far, though it feels a bit cheeky. I'd appreciate some suggestions here on how to handle this in the most robust way. For example, I was worried about mapped memory, legacy craziness in the first MB of address space, etc. As a side note, I could not get this to work for hotplugging memory via adding a memdimm0 at runtime; that memory would be zeroed out by the guest. It may indeed just be about _BOOT_ZeroedMemory... I also wonder whether HvExtCallQueryCapabilities() should have a prop bit, like HvExtCallGetBootZeroedMemory(). Right now, HvExtCallQueryCapabilities() is always enabled after this patch, and simply returns "no ext capabilities" if HvExtCallGetBootZeroedMemory() is not enabled. This does technically change guest-visible behaviour. [1] But then we'll have to implement a dependency between the props, because you can't have HvExtCallGetBootZeroedMemory() without HvExtCallQueryCapabilities(), because the guest won't have a way to figure out it's available. [1] There's a little quirk here, in case someone else gets stung by this: Windows seems to have a long-standing bug/"feature", where it _always_ issues a HvExtCallQueryCapabilities() hypercall, even if it's not signalled in the CPUID leaf. Except in those cases, it seems to ignore the return value. See here for someone else who noticed that years ago: https://lists.xenproject.org/archives/html/xen-devel/2017-03/msg02809.html And it's still happening to this day. Florian Schmidt (2): Add HvExtCallQueryCapabilities Add HvExtCallGetBootZeroedMemory docs/system/i386/hyperv.rst | 5 +++ hw/hyperv/hyperv.c | 62 ++++++++++++++++++++++++++++++++ include/hw/hyperv/hyperv-proto.h | 12 +++++++ include/hw/hyperv/hyperv.h | 10 ++++++ target/i386/cpu.c | 2 ++ target/i386/cpu.h | 1 + target/i386/kvm/hyperv-proto.h | 6 ++++ target/i386/kvm/hyperv.c | 24 +++++++++++++ target/i386/kvm/kvm.c | 3 ++ 9 files changed, 125 insertions(+) -- 2.39.5
