Hi, On Sun, May 21, 2023 at 02:01:22PM +0000, Mathew, Cherry G.* wrote: > I'm wondering if there are any nvmm(4) users out there - I'd like to > understand what your user experience is - expecially for multiple VMs > running simultaneously. > > Specifically, I'd like to understand if nvmm based qemu VMs have > interactive "jitter" or any such scheduling related effects. > > I tried 10.0_BETA with nvmm and it was unusable for 3guests that I > migrated from XEN, so I had to fall back. > > Just looking for experiences from any users of nvmm(4) on NetBSD (any > version, including -current is fine).
How many CPUs does each instance have and how many CPUs does your machine have? It could be, but thats just a hunch reading the code since I'm not that deep in NVMM (!), that possibly more than one Qemu has vCPUs activated and all host CPUs are busy? This will mean that the other Qemu processes are not run (for I/O etc) until a tick interrupt comes by or an userland event like IO read/pagefault comes by and one vCPU is aborted. This will be handled in the kernel and control will be given back to its nvmm userland part and the qemu handler might interrupt the other vCPUs to trigger their virtual interrupt handling. Note that while running a vCPU the kernel preemption is switched off. This might give the irregular IO performance you see as the qemu IO threads are not sheduled often enough. A solution might be that the nvmm code in *_vcpu_run() sets a timer interrupt with a dummy handler on its CPU and on its forced vmreturn checks the number of cpus currently not running NVMM cpus and if that number is lower than the number of virtual machines that are declared and running it yields by returning to userland to have at least one CPU for each of the Qemus. Otherwise it can continue as before without making the expensive switch back to userland. Ideally we ought to ask Maxime Villard who wrote the code and check/coordinate with DragonFlyBSD but he also isn't with DragonFly anymore IIRC. So, I might be barking up the wrong tree but this might be it. With regards, Reinoud