Re: [libvirt] [RFC] On present using dummy hostdev usb device
On Fri, Aug 30, 2019 at 04:19:23PM +0100, Daniel P. Berrangé wrote: > On Fri, Aug 30, 2019 at 05:08:52PM +0200, Michal Privoznik wrote: > > On 8/30/19 2:30 PM, Roman Kagan wrote: > > > On Fri, Aug 30, 2019 at 10:09:06AM +0100, Daniel P. Berrangé wrote: > > > > On Fri, Aug 30, 2019 at 08:44:03AM +, Nikolay Shirokovskiy wrote: > > > > > Hi, all! > > > > > > > > > >We use an interesting approach when starting/migrating/etc domain > > > > > with usb > > > > > hostdev with startupPolicy=optional. We add qemu usb-host device with > > > > > missing hostaddr/hostbus parameters (dummy device). I guess there are > > > > > 2 reasons why we do it. First without dummy device migration will > > > > > fail as > > > > > described in [1]. Second is an interesting property of dummy device > > > > > that > > > > > qemu starts to monitor for attaching of usb devices and binds the > > > > > first > > > > > attached to node to the dummy device. So one can start a domain with > > > > > missing hostdev and attach it later or migrate a domain then detach > > > > > hostdev on source and attach it on destination. But as qemu binds the > > > > > first attached device this is not reliable, to say the least. And > > > > > after > > > > > all this does not work if domain uses distinct mount namespace which > > > > > is default. > > > > > > > > Even without mount namespaces, it should fail as QEMU is running > > > > non-root > > > > and libvirt won't have granted access to any host USB devices in /dev, > > > > and > > > > also SELinux policy will forbid this. > > > > > > Right, but the case with mount namespaces is particularly problematic: > > > if the device open fails due to missing device node, libusb removes the > > > device from its internal device list. This results in the following > > > scenario: > > > > > > - libvirt adds a dummy usb-host device to QEMU in place of a missing > > >device > > > > > > - QEMU (via libusb) installs a watch for udev add events > > > > > > - the physical device is plugged into the host > > > > > > - QEMU detects the addition of the device and, since the dummy device > > >matches everything, tries to open it > > > > > > - by this time libvirt may have not created a device node in QEMU's > > >mount namespace, so the open fails due to missing device node, and > > >libusb removes the device from its internal list > > > > > > - libvirt removes the dummy usb-host device and adds the actual usb-host > > >device > > > > > > - QEMU fails to open it because it's no longer seen by libusb > > > > There is a bug filed against libusb exactly for this: > > > > https://bugzilla.redhat.com/show_bug.cgi?id=1595525 > > > > BTW: you don't have to migrate, it's sufficient to start a domain with a > > missing USB and startupPolicy='optional' and then physically plug it into > > the host and then try to hotplug it into the domain. > > AFAIR, the startupPolicy=optional was never really intended to allow > for the device to be dynamically added after startup. It was only > trying drop the device if it didn't exist at startup. > > If we do want a way to dynamically add after startup, then libvirt itself > would have to monitor the USB devices on the host, and then perform QMP > commands to hot-add to QEMU. This is actually what Nikolay was trying to achieve with his patchset (https://www.redhat.com/archives/libvir-list/2019-August/msg01413.html). Is this a NAK to it, with a suggestion that it's the upper layer's responsibility? Thanks, Roman. -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [RFC] On present using dummy hostdev usb device
On Fri, Aug 30, 2019 at 10:09:06AM +0100, Daniel P. Berrangé wrote: > On Fri, Aug 30, 2019 at 08:44:03AM +, Nikolay Shirokovskiy wrote: > > Hi, all! > > > > We use an interesting approach when starting/migrating/etc domain with usb > > hostdev with startupPolicy=optional. We add qemu usb-host device with > > missing hostaddr/hostbus parameters (dummy device). I guess there are > > 2 reasons why we do it. First without dummy device migration will fail as > > described in [1]. Second is an interesting property of dummy device that > > qemu starts to monitor for attaching of usb devices and binds the first > > attached to node to the dummy device. So one can start a domain with > > missing hostdev and attach it later or migrate a domain then detach > > hostdev on source and attach it on destination. But as qemu binds the > > first attached device this is not reliable, to say the least. And after > > all this does not work if domain uses distinct mount namespace which > > is default. > > Even without mount namespaces, it should fail as QEMU is running non-root > and libvirt won't have granted access to any host USB devices in /dev, and > also SELinux policy will forbid this. Right, but the case with mount namespaces is particularly problematic: if the device open fails due to missing device node, libusb removes the device from its internal device list. This results in the following scenario: - libvirt adds a dummy usb-host device to QEMU in place of a missing device - QEMU (via libusb) installs a watch for udev add events - the physical device is plugged into the host - QEMU detects the addition of the device and, since the dummy device matches everything, tries to open it - by this time libvirt may have not created a device node in QEMU's mount namespace, so the open fails due to missing device node, and libusb removes the device from its internal list - libvirt removes the dummy usb-host device and adds the actual usb-host device - QEMU fails to open it because it's no longer seen by libusb IOW a usb-host device with missing=true can't (reliably, because sometimes libvirt is quick enough to create the device node before QEMU gives up opening it) turn into a working one without QEMU restart. > > So I question does it make sense to use dummy device at all? In case of > > migration/resume from suspend/revert to snapshot we can either fix qemu to > > ignore incoming missing hostdev data or add dummy device temporarily. The > > latter solution is worse as it brings dummy device behaviour even for a > > short > > period of time. However having a temporary dummy device is neccessary step > > towards the time when all supported versions of qemu do the mentioned > > ignoring. > > As to handling attaching of missing hostdev device to node it should be > > done in > > libvirt which can do necessary mount namespace actions. (Actually I > > developing > > such patches right now but some peculiarities of dummy device bring me > > here). > > The problems around host USB device passthrough are conceptually similar > to the problems of hots PCI device passthrough. > > In both cases we cannot assume the device present on the source device > exists on the target device in the same way. > > In both cases, even if the device does exist on the target, we cannot > serialize the state of the host device across the migration. Right. > For PCI devices we simply refuse to initiate the migration if any host > PCI devices are attached. The mgmt app has to hot-unplug all devices > before migration, and hot-plug new devices after migration if desired. > > I'm inclined to suggest that same approach of hotunplug + hotplug either > side of migration is the only viable option for host USB devices too. > > As such any mgmt app could do this dance today without any changes in > libvirt. Are you trying to say that the mgmt app should just refrain from creating usb-host devices with missing=true? > If we turned host USB devices into a migration blocker though, that > could be considered a significant change of behaviour for mgmt apps, > even though this dummy USB device is effectively useless due to our > security policies. I'm afraid the issue is a bit more severe: the dummy device isn't just useless, it stands in the way of the real device later on. Thanks, Roman. -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 0/3] conf: qemu: support new Hyper-V enlightenments
On Thu, Aug 09, 2018 at 03:14:18PM +0200, Vitaly Kuznetsov wrote: > Several new Hyper-V enlightenments were recently added to Qemu: > - hv-frequencies > - hv-reenlightenment > - hv-tlbflush > > Support these in libvirt. > > Vitaly Kuznetsov (3): > conf: qemu: add support for Hyper-V frequency MSRs > conf: qemu: add support for Hyper-V reenlightenment notifications > conf: qemu: add support for Hyper-V PV TLB flush > > docs/formatdomain.html.in | 21 + > docs/schemas/domaincommon.rng | 15 +++ > src/conf/domain_conf.c | 14 +- > src/conf/domain_conf.h | 3 +++ > src/cpu/cpu_x86.c | 9 + > src/cpu/cpu_x86_data.h | 3 +++ > src/qemu/qemu_command.c | 3 +++ > src/qemu/qemu_parse_command.c | 3 +++ > src/qemu/qemu_process.c | 3 +++ > tests/qemuxml2argvdata/hyperv-off.xml | 3 +++ > tests/qemuxml2argvdata/hyperv.args | 3 ++- > tests/qemuxml2argvdata/hyperv.xml | 3 +++ > tests/qemuxml2xmloutdata/hyperv-off.xml | 3 +++ > tests/qemuxml2xmloutdata/hyperv.xml | 3 +++ > 14 files changed, 87 insertions(+), 2 deletions(-) Bringing this to the attention of our libvirt guru... Roman. -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
