[libvirt] [PATCH 8/8] qemu: set qemu process' RLIMIT_MEMLOCK when VFIO is used
VFIO requires all of the guest's memory and IO space to be lockable in RAM. The domain's max_balloon is the maximum amount of memory the domain can have (in KiB). We add a generous 1GiB to that for IO space (still much better than KVM device assignment, where the KVM module actually *ignores* the process limits and locks everything anyway), and convert from KiB to bytes. In the case of hotplug, we are changing the limit for the already existing qemu process (prlimit() is used under the hood), and for regular commandline additions of vfio devices, we schedule a call to setrlimit() that will happen after the qemu process is forked. --- src/qemu/qemu_command.c | 22 +++--- src/qemu/qemu_hotplug.c | 24 +--- 2 files changed, 32 insertions(+), 14 deletions(-) diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 761291c..80a0dec 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -7906,13 +7906,21 @@ qemuBuildCommandLine(virConnectPtr conn, if (hostdev-mode == VIR_DOMAIN_HOSTDEV_MODE_SUBSYS hostdev-source.subsys.type == VIR_DOMAIN_HOSTDEV_SUBSYS_TYPE_PCI) { -if ((hostdev-source.subsys.u.pci.backend - == VIR_DOMAIN_HOSTDEV_PCI_BACKEND_TYPE_VFIO) -!virQEMUCapsGet(qemuCaps, QEMU_CAPS_DEVICE_VFIO_PCI)) { -virReportError(VIR_ERR_CONFIG_UNSUPPORTED, %s, - _(VFIO PCI device assignment is not supported by - this version of qemu)); -goto error; +if (hostdev-source.subsys.u.pci.backend +== VIR_DOMAIN_HOSTDEV_PCI_BACKEND_TYPE_VFIO) { +if (!virQEMUCapsGet(qemuCaps, QEMU_CAPS_DEVICE_VFIO_PCI)) { +virReportError(VIR_ERR_CONFIG_UNSUPPORTED, %s, + _(VFIO PCI device assignment is not + supported by this version of qemu)); +goto error; +} +/* VFIO requires all of the guest's memory to be + * locked resident, plus some amount for IO + * space. Alex Williamson suggested adding 1GiB for IO + * space just to be safe (some finer tuning might be + * nice, though). + */ +virCommandSetMaxMemLock(cmd, ((unsigned long long)def-mem.max_balloon + (1024 * 1024)) * 1024); } if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_DEVICE)) { diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index 30c8bda..af0b8a6 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -999,13 +999,23 @@ int qemuDomainAttachHostPciDevice(virQEMUDriverPtr driver, hostdev, 1) 0) return -1; -if ((hostdev-source.subsys.u.pci.backend - == VIR_DOMAIN_HOSTDEV_PCI_BACKEND_TYPE_VFIO) -!virQEMUCapsGet(priv-qemuCaps, QEMU_CAPS_DEVICE_VFIO_PCI)) { -virReportError(VIR_ERR_CONFIG_UNSUPPORTED, %s, - _(VFIO PCI device assignment is not supported by - this version of qemu)); -goto error; +if (hostdev-source.subsys.u.pci.backend +== VIR_DOMAIN_HOSTDEV_PCI_BACKEND_TYPE_VFIO) { +if (!virQEMUCapsGet(priv-qemuCaps, QEMU_CAPS_DEVICE_VFIO_PCI)) { +virReportError(VIR_ERR_CONFIG_UNSUPPORTED, %s, + _(VFIO PCI device assignment is not + supported by this version of qemu)); +goto error; +} +/* VFIO requires all of the guest's memory to be locked + * resident, plus some amount for IO space. Alex Williamson + * suggested adding 1GiB for IO space just to be safe (some + * finer tuning might be nice, though). + * In this case, the guest's memory may already be locked, but + * it doesn't hurt to change the limit to the same value. + */ +virSetMaxMemLock(vm-pid, + ((unsigned long long)vm-def-mem.max_balloon + (1024 * 1024)) * 1024); } if (virQEMUCapsGet(priv-qemuCaps, QEMU_CAPS_DEVICE)) { -- 1.7.11.7 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 8/8] qemu: set qemu process' RLIMIT_MEMLOCK when VFIO is used
On 04/25/2013 11:57 AM, Laine Stump wrote: VFIO requires all of the guest's memory and IO space to be lockable in RAM. The domain's max_balloon is the maximum amount of memory the domain can have (in KiB). We add a generous 1GiB to that for IO space (still much better than KVM device assignment, where the KVM module actually *ignores* the process limits and locks everything anyway), and convert from KiB to bytes. In the case of hotplug, we are changing the limit for the already existing qemu process (prlimit() is used under the hood), and for regular commandline additions of vfio devices, we schedule a call to setrlimit() that will happen after the qemu process is forked. --- src/qemu/qemu_command.c | 22 +++--- src/qemu/qemu_hotplug.c | 24 +--- 2 files changed, 32 insertions(+), 14 deletions(-) +/* VFIO requires all of the guest's memory to be + * locked resident, plus some amount for IO + * space. Alex Williamson suggested adding 1GiB for IO + * space just to be safe (some finer tuning might be + * nice, though). + */ +virCommandSetMaxMemLock(cmd, ((unsigned long long)def-mem.max_balloon + (1024 * 1024)) * 1024); Worth wrapping the long line, perhaps at the + operator? Or even using a temporary: unsigned long long mem_kbytes = def-mem.max_balloon + (1024 * 1024); virCommandSetMaxMemLock(cmd, mem_kbytes * 1024); +virSetMaxMemLock(vm-pid, + ((unsigned long long)vm-def-mem.max_balloon + (1024 * 1024)) * 1024); and again. Line wrapping is trivial, though, so: ACK. -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 8/8] qemu: set qemu process' RLIMIT_MEMLOCK when VFIO is used
On 04/25/2013 06:14 PM, Eric Blake wrote: On 04/25/2013 11:57 AM, Laine Stump wrote: VFIO requires all of the guest's memory and IO space to be lockable in RAM. The domain's max_balloon is the maximum amount of memory the domain can have (in KiB). We add a generous 1GiB to that for IO space (still much better than KVM device assignment, where the KVM module actually *ignores* the process limits and locks everything anyway), and convert from KiB to bytes. In the case of hotplug, we are changing the limit for the already existing qemu process (prlimit() is used under the hood), and for regular commandline additions of vfio devices, we schedule a call to setrlimit() that will happen after the qemu process is forked. --- src/qemu/qemu_command.c | 22 +++--- src/qemu/qemu_hotplug.c | 24 +--- 2 files changed, 32 insertions(+), 14 deletions(-) +/* VFIO requires all of the guest's memory to be + * locked resident, plus some amount for IO + * space. Alex Williamson suggested adding 1GiB for IO + * space just to be safe (some finer tuning might be + * nice, though). + */ +virCommandSetMaxMemLock(cmd, ((unsigned long long)def-mem.max_balloon + (1024 * 1024)) * 1024); Worth wrapping the long line, perhaps at the + operator? Or even using a temporary: unsigned long long mem_kbytes = def-mem.max_balloon + (1024 * 1024); virCommandSetMaxMemLock(cmd, mem_kbytes * 1024); I did the latter. +virSetMaxMemLock(vm-pid, + ((unsigned long long)vm-def-mem.max_balloon + (1024 * 1024)) * 1024); and again. Same here. Line wrapping is trivial, though, so: ACK. -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list