Re: [ovs-dev] [RFC] Vhost-user backends cross-version migration support
On Fri, Feb 03, 2017 at 05:34:07PM +0200, Michael S. Tsirkin wrote: > On Fri, Feb 03, 2017 at 03:11:10PM +0100, Maxime Coquelin wrote: > > Hi, > > > > On 02/01/2017 09:35 AM, Maxime Coquelin wrote: > > > Hi, > > > > > > Few months ago, Michael reported a problem about migrating VMs relying > > > on vhost-user between hosts supporting different backend versions: > > > - Message-Id: <20161011173526-mutt-send-email-...@kernel.org> > > > - https://lists.gnu.org/archive/html/qemu-devel/2016-10/msg03026.html > > > > > > The goal of this thread is to draft a proposal based on the outcomes > > > of discussions with contributors of the different parties (DPDK/OVS > > > /libvirt/...). > > > > Thanks the first feedback. It seems to converge that this is Nova's > > role, but not Libvirt one to manage these versions from management tool > > layer. > > > I think the conclusion is not that it should go up the stack. I think > this will just get broken all the time. No one understands versions and > stuff. Even QEMU developers get confused and break compatibility once in > a while. I don't think it is really true, nor is it really specific to migration, it can hit any time you have a network connection that is re-opened even without migration. Version compatibility negotiations are an inherant part of any network protocol that is expected to evolve over time. > My conclusion is that doing it from OVS side is wrong. Migration is not > an OVS thing, it's a QEMU thing, and libvirt abstracts QEMU.People > just want migration to work, ok? It's our job to do it, we do not really > need a "make things work" flag. > > If libvirt does not want to use the vhost-user protocol (which sounds > reasonable, it's rather complex) how about qemu providing a small > utility to query the port? We could output json or whatever. > > This can help with MTU as well. > > And maybe it will help with nowait support - if someone uses the utility > to dump backend config once, QEMU can later start the device without > feature queries. I don't think it is QEMU's job to deal with external components in this way and don't think QEMU vhost-user should be treated as special. This general scenario arises any time there is a QEMU backend is talking to an external service/process. For example, a virtio-serial talking to a daemon in the host. This daemon can support different versions of the protocol being spoken, so we have the same compat issue on migration. Or a traditional serial device, which the guest is using to talk to an external daemon the host. In a few cases we might know what the protocol is, but in general the data stream is opaque to QEMU. QEMU should not have need to learn about every single protocol that might be used to communicate with some external service. This is an unbounded set of possibilities to deal with, some of which will not even be open source. This all needs to be delegated to whatever mgmt app is responsible for setting up these external services, as it has the right domain knowledge about the applications being used, to make the policy decisions that are suitable for its usage scenario. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://entangle-photo.org -o-http://search.cpan.org/~danberr/ :| ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [libvirt] [RFC] Vhost-user backends cross-version migration support
On Fri, Feb 03, 2017 at 10:41:12AM +0100, Maxime Coquelin wrote: > > > On 02/03/2017 10:27 AM, Daniel P. Berrange wrote: > > On Thu, Feb 02, 2017 at 08:27:18PM +0200, Michael S. Tsirkin wrote: > > > On Thu, Feb 02, 2017 at 06:21:55PM +, Daniel P. Berrange wrote: > > > > On Thu, Feb 02, 2017 at 07:31:49PM +0200, Michael S. Tsirkin wrote: > > > > > On Thu, Feb 02, 2017 at 05:29:08PM +, Daniel P. Berrange wrote: > > > > > > On Thu, Feb 02, 2017 at 07:20:35PM +0200, Michael S. Tsirkin wrote: > > > > > > > On Thu, Feb 02, 2017 at 05:10:28PM +, Daniel P. Berrange > > > > > > > wrote: > > > > > > > > On Thu, Feb 02, 2017 at 06:21:55PM +0200, Michael S. Tsirkin > > > > > > > > wrote: > > > > > > > > > On Thu, Feb 02, 2017 at 03:06:21PM +, Daniel P. Berrange > > > > > > > > > wrote: > > > > > > > > > > On Thu, Feb 02, 2017 at 03:14:01PM +0100, Maxime Coquelin > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 02/01/2017 12:41 PM, Daniel P. Berrange wrote: > > > > > > > > > > > > > > > > > > > > > > > > It depends where / how in OVS it needs to be set. The > > > > > > > > > > > > only stuff libvirt > > > > > > > > > > > > does with OVS is to run 'add-port' and 'del-port' > > > > > > > > > > > > commands via the ovs > > > > > > > > > > > > cli tool. We pass through arguments from the port > > > > > > > > > > > > profile stored in the > > > > > > > > > > > > XML config. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > interfaceid='09b11c53-8b5c-4eeb-8f00-d84eaa0aaa4f'/> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > eg those things in get passed as cli args > > > > > > > > > > > > to the 'add-port' > > > > > > > > > > > > command. Soo if add-port needs this new version string, > > > > > > > > > > > > then we'd need > > > > > > > > > > > > to add the version to the openvswitch virtualport XML. > > > > > > > > > > > > > > > > > > > > > > > > If the version is provided to OVS in a different > > > > > > > > > > > > command, then it would > > > > > > > > > > > > probably be outside scope of libvirt. > > > > > > > > > > > > > > > > > > > > > > I think it would make sense to be a parameter of the > > > > > > > > > > > add-port command. > > > > > > > > > > > But it would be for vhost-user related add-port command, > > > > > > > > > > > I didn't find > > > > > > > > > > > where/if this is managed in libvirt XML. > > > > > > > > > > > > > > > > > > > > For vhost-user, libvirt does not have any interaction with > > > > > > > > > > OVS at > > > > > > > > > > all. If the thing that's using the vhost-user UNIX socket, > > > > > > > > > > in turn > > > > > > > > > > connects to OVS, that's outside scope of libvirt. IOW, for > > > > > > > > > > vhost-user > > > > > > > > > > OVS it seems like that job is for Nova / os-vif to solve. > > > > > > > > > > > > > > > > > > I don't think they currently understand the issues involved in > > > >
Re: [ovs-dev] [libvirt] [RFC] Vhost-user backends cross-version migration support
On Thu, Feb 02, 2017 at 08:27:18PM +0200, Michael S. Tsirkin wrote: > On Thu, Feb 02, 2017 at 06:21:55PM +0000, Daniel P. Berrange wrote: > > On Thu, Feb 02, 2017 at 07:31:49PM +0200, Michael S. Tsirkin wrote: > > > On Thu, Feb 02, 2017 at 05:29:08PM +, Daniel P. Berrange wrote: > > > > On Thu, Feb 02, 2017 at 07:20:35PM +0200, Michael S. Tsirkin wrote: > > > > > On Thu, Feb 02, 2017 at 05:10:28PM +, Daniel P. Berrange wrote: > > > > > > On Thu, Feb 02, 2017 at 06:21:55PM +0200, Michael S. Tsirkin wrote: > > > > > > > On Thu, Feb 02, 2017 at 03:06:21PM +, Daniel P. Berrange > > > > > > > wrote: > > > > > > > > On Thu, Feb 02, 2017 at 03:14:01PM +0100, Maxime Coquelin wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > On 02/01/2017 12:41 PM, Daniel P. Berrange wrote: > > > > > > > > > > > > > > > > > > > > It depends where / how in OVS it needs to be set. The only > > > > > > > > > > stuff libvirt > > > > > > > > > > does with OVS is to run 'add-port' and 'del-port' commands > > > > > > > > > > via the ovs > > > > > > > > > > cli tool. We pass through arguments from the port profile > > > > > > > > > > stored in the > > > > > > > > > > XML config. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > interfaceid='09b11c53-8b5c-4eeb-8f00-d84eaa0aaa4f'/> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > eg those things in get passed as cli args to > > > > > > > > > > the 'add-port' > > > > > > > > > > command. Soo if add-port needs this new version string, > > > > > > > > > > then we'd need > > > > > > > > > > to add the version to the openvswitch virtualport XML. > > > > > > > > > > > > > > > > > > > > If the version is provided to OVS in a different command, > > > > > > > > > > then it would > > > > > > > > > > probably be outside scope of libvirt. > > > > > > > > > > > > > > > > > > I think it would make sense to be a parameter of the add-port > > > > > > > > > command. > > > > > > > > > But it would be for vhost-user related add-port command, I > > > > > > > > > didn't find > > > > > > > > > where/if this is managed in libvirt XML. > > > > > > > > > > > > > > > > For vhost-user, libvirt does not have any interaction with OVS > > > > > > > > at > > > > > > > > all. If the thing that's using the vhost-user UNIX socket, in > > > > > > > > turn > > > > > > > > connects to OVS, that's outside scope of libvirt. IOW, for > > > > > > > > vhost-user > > > > > > > > OVS it seems like that job is for Nova / os-vif to solve. > > > > > > > > > > > > > > I don't think they currently understand the issues involved in > > > > > > > cross-version migration though. This is a complex subject and > > > > > > > easy to > > > > > > > get wrong. It would be significantly better to figure it out at > > > > > > > libvirt > > > > > > > level since it already deals with cross-version migration issues. > > > > > > > > > > > > Libvirt considers vhost-user to be a blackbox though - it just > > > > > > exposes > > > > > > a UNIX socket, and whatever is on the other end is completely > > > > > > opaque. > > > > > > The fact that the other end might plumb the data stream into > > > > > > openvswitch > > >
Re: [ovs-dev] [libvirt] [RFC] Vhost-user backends cross-version migration support
On Thu, Feb 02, 2017 at 07:31:49PM +0200, Michael S. Tsirkin wrote: > On Thu, Feb 02, 2017 at 05:29:08PM +0000, Daniel P. Berrange wrote: > > On Thu, Feb 02, 2017 at 07:20:35PM +0200, Michael S. Tsirkin wrote: > > > On Thu, Feb 02, 2017 at 05:10:28PM +, Daniel P. Berrange wrote: > > > > On Thu, Feb 02, 2017 at 06:21:55PM +0200, Michael S. Tsirkin wrote: > > > > > On Thu, Feb 02, 2017 at 03:06:21PM +, Daniel P. Berrange wrote: > > > > > > On Thu, Feb 02, 2017 at 03:14:01PM +0100, Maxime Coquelin wrote: > > > > > > > > > > > > > > > > > > > > > On 02/01/2017 12:41 PM, Daniel P. Berrange wrote: > > > > > > > > > > > > > > > > It depends where / how in OVS it needs to be set. The only > > > > > > > > stuff libvirt > > > > > > > > does with OVS is to run 'add-port' and 'del-port' commands via > > > > > > > > the ovs > > > > > > > > cli tool. We pass through arguments from the port profile > > > > > > > > stored in the > > > > > > > > XML config. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > > > > > > interfaceid='09b11c53-8b5c-4eeb-8f00-d84eaa0aaa4f'/> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > eg those things in get passed as cli args to the > > > > > > > > 'add-port' > > > > > > > > command. Soo if add-port needs this new version string, then > > > > > > > > we'd need > > > > > > > > to add the version to the openvswitch virtualport XML. > > > > > > > > > > > > > > > > If the version is provided to OVS in a different command, then > > > > > > > > it would > > > > > > > > probably be outside scope of libvirt. > > > > > > > > > > > > > > I think it would make sense to be a parameter of the add-port > > > > > > > command. > > > > > > > But it would be for vhost-user related add-port command, I didn't > > > > > > > find > > > > > > > where/if this is managed in libvirt XML. > > > > > > > > > > > > For vhost-user, libvirt does not have any interaction with OVS at > > > > > > all. If the thing that's using the vhost-user UNIX socket, in turn > > > > > > connects to OVS, that's outside scope of libvirt. IOW, for > > > > > > vhost-user > > > > > > OVS it seems like that job is for Nova / os-vif to solve. > > > > > > > > > > I don't think they currently understand the issues involved in > > > > > cross-version migration though. This is a complex subject and easy to > > > > > get wrong. It would be significantly better to figure it out at > > > > > libvirt > > > > > level since it already deals with cross-version migration issues. > > > > > > > > Libvirt considers vhost-user to be a blackbox though - it just exposes > > > > a UNIX socket, and whatever is on the other end is completely opaque. > > > > The fact that the other end might plumb the data stream into openvswitch > > > > is not something libvirt should know, as we don't want to end up having > > > > to add custom code to libvirt for every different vhost-user server > > > > impl. > > > > > > > > IOW, if the version str can be passed to QEMU, and then onto vhost-user > > > > backend in QEMU, then libvirt can be involved. If the version str has > > > > to be given to openvswitch that's not for libvirt to deall with. > > > > > > > > Regards, > > > > Daniel > > > > > > It's not just about passing it to QEMU. The following is needed: > > > - need to query version when creating VM/device > > > - need to query supported versions when migrating VM > > > > Those are both things that nova can do, since it knows what the vhost-user > > device in question is connected to, and so can query the versions, and check > > versions before triggering migration with libvirt > > It can, but then it will need to query libvirt or source for the version > string since it's in the XML. No, it wouldn't be in the XML at all. Nova on the source queries what vhostuser version it has and what the target host has, and can prevent the migration if they're incompatible. I dont think libvirt has to be involved at all for this, as all the info can be obtained by nova/os-vif from the vhostuser impl it has configured. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://entangle-photo.org -o-http://search.cpan.org/~danberr/ :| ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [libvirt] [RFC] Vhost-user backends cross-version migration support
On Thu, Feb 02, 2017 at 07:20:35PM +0200, Michael S. Tsirkin wrote: > On Thu, Feb 02, 2017 at 05:10:28PM +0000, Daniel P. Berrange wrote: > > On Thu, Feb 02, 2017 at 06:21:55PM +0200, Michael S. Tsirkin wrote: > > > On Thu, Feb 02, 2017 at 03:06:21PM +, Daniel P. Berrange wrote: > > > > On Thu, Feb 02, 2017 at 03:14:01PM +0100, Maxime Coquelin wrote: > > > > > > > > > > > > > > > On 02/01/2017 12:41 PM, Daniel P. Berrange wrote: > > > > > > > > > > > > It depends where / how in OVS it needs to be set. The only stuff > > > > > > libvirt > > > > > > does with OVS is to run 'add-port' and 'del-port' commands via the > > > > > > ovs > > > > > > cli tool. We pass through arguments from the port profile stored in > > > > > > the > > > > > > XML config. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > > > > interfaceid='09b11c53-8b5c-4eeb-8f00-d84eaa0aaa4f'/> > > > > > > > > > > > > > > > > > > > > > > > > eg those things in get passed as cli args to the > > > > > > 'add-port' > > > > > > command. Soo if add-port needs this new version string, then we'd > > > > > > need > > > > > > to add the version to the openvswitch virtualport XML. > > > > > > > > > > > > If the version is provided to OVS in a different command, then it > > > > > > would > > > > > > probably be outside scope of libvirt. > > > > > > > > > > I think it would make sense to be a parameter of the add-port command. > > > > > But it would be for vhost-user related add-port command, I didn't find > > > > > where/if this is managed in libvirt XML. > > > > > > > > For vhost-user, libvirt does not have any interaction with OVS at > > > > all. If the thing that's using the vhost-user UNIX socket, in turn > > > > connects to OVS, that's outside scope of libvirt. IOW, for vhost-user > > > > OVS it seems like that job is for Nova / os-vif to solve. > > > > > > I don't think they currently understand the issues involved in > > > cross-version migration though. This is a complex subject and easy to > > > get wrong. It would be significantly better to figure it out at libvirt > > > level since it already deals with cross-version migration issues. > > > > Libvirt considers vhost-user to be a blackbox though - it just exposes > > a UNIX socket, and whatever is on the other end is completely opaque. > > The fact that the other end might plumb the data stream into openvswitch > > is not something libvirt should know, as we don't want to end up having > > to add custom code to libvirt for every different vhost-user server > > impl. > > > > IOW, if the version str can be passed to QEMU, and then onto vhost-user > > backend in QEMU, then libvirt can be involved. If the version str has > > to be given to openvswitch that's not for libvirt to deall with. > > > > Regards, > > Daniel > > It's not just about passing it to QEMU. The following is needed: > - need to query version when creating VM/device > - need to query supported versions when migrating VM Those are both things that nova can do, since it knows what the vhost-user device in question is connected to, and so can query the versions, and check versions before triggering migration with libvirt Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://entangle-photo.org -o-http://search.cpan.org/~danberr/ :| ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [libvirt] [RFC] Vhost-user backends cross-version migration support
On Thu, Feb 02, 2017 at 07:09:24PM +0200, Michael S. Tsirkin wrote: > On Thu, Feb 02, 2017 at 11:47:57AM -0500, Laine Stump wrote: > > On 02/02/2017 10:06 AM, Daniel P. Berrange wrote: > > > On Thu, Feb 02, 2017 at 03:14:01PM +0100, Maxime Coquelin wrote: > > > > > > > > On 02/01/2017 12:41 PM, Daniel P. Berrange wrote: > > > > > It depends where / how in OVS it needs to be set. The only stuff > > > > > libvirt > > > > > does with OVS is to run 'add-port' and 'del-port' commands via the ovs > > > > > cli tool. > > > > (aside note: the code that exec's ovs-vsctl was written back when there was > > no standardized API for performing such operations. libvirt would prefer to > > not exec external programs though, and I've heard that OVS may now have an > > official API of some sort for doing things like this (maybe via netlink or > > dbus or something?) If that's the case, can someone point me in the right > > direction?) > > > > > > > We pass through arguments from the port profile stored in the > > > > > XML config. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > interfaceid='09b11c53-8b5c-4eeb-8f00-d84eaa0aaa4f'/> > > > > > > > > > > > > > > > > > > > > eg those things in get passed as cli args to the > > > > > 'add-port' > > > > > command. Soo if add-port needs this new version string, then we'd need > > > > > to add the version to the openvswitch virtualport XML. > > > > > > > > > > If the version is provided to OVS in a different command, then it > > > > > would > > > > > probably be outside scope of libvirt. > > > > I think it would make sense to be a parameter of the add-port command. > > > > But it would be for vhost-user related add-port command, I didn't find > > > > where/if this is managed in libvirt XML. > > > For vhost-user, libvirt does not have any interaction with OVS at > > > all. If the thing that's using the vhost-user UNIX socket, in turn > > > connects to OVS, that's outside scope of libvirt. IOW, for vhost-user > > > OVS it seems like that job is for Nova / os-vif to solve. > > > > This brings up another tangentially related question I came up against last > > night - qemu now has an option to report the host's MTU to the guest for > > virtio and vhost-user interfaces, and Michal Privoznik recently pushed > > patches to set the MTU sent to the guest via an explicit in > > libvirt's interface config: > > > >https://bugzilla.redhat.com/1408701 > > > > But it would be much nicer if libvirt could learn the MTU of [that stuff at > > the other end of the unix socket] without requiring intervention in > > libvirt's config. For example, I'm just now testing patches for tap-based > > interfaces (connecting to Linux host bridges or OVS switches) that query the > > current MTU of the bridge and report that to qemu; this eliminates the > > burden of configuring each interface of each guest individually (and > > changing that config in all those places if someone ever wants to change the > > MTU of the bridge). > > > > As Dan says, though, libvirt's only interaction in the case of vhost-user is > > with the unix socket. Is there any way to learn what is the appropriate MTU > > from OVS in these cases? Or must Nova (or ovirt or some poor user) set that > > up in the libvirt config for every single interface? > > We could add commands for all kind of queries to the vhost-user > protocol. libvirt would have to learn the vhost-user protocol though. > Interested? No, I really don't think libvirt should implement the vhost-user protocol Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://entangle-photo.org -o-http://search.cpan.org/~danberr/ :| ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [libvirt] [RFC] Vhost-user backends cross-version migration support
On Thu, Feb 02, 2017 at 06:21:55PM +0200, Michael S. Tsirkin wrote: > On Thu, Feb 02, 2017 at 03:06:21PM +0000, Daniel P. Berrange wrote: > > On Thu, Feb 02, 2017 at 03:14:01PM +0100, Maxime Coquelin wrote: > > > > > > > > > On 02/01/2017 12:41 PM, Daniel P. Berrange wrote: > > > > > > > > It depends where / how in OVS it needs to be set. The only stuff libvirt > > > > does with OVS is to run 'add-port' and 'del-port' commands via the ovs > > > > cli tool. We pass through arguments from the port profile stored in the > > > > XML config. > > > > > > > > > > > > > > > > > > > >> > > interfaceid='09b11c53-8b5c-4eeb-8f00-d84eaa0aaa4f'/> > > > > > > > > > > > > > > > > eg those things in get passed as cli args to the > > > > 'add-port' > > > > command. Soo if add-port needs this new version string, then we'd need > > > > to add the version to the openvswitch virtualport XML. > > > > > > > > If the version is provided to OVS in a different command, then it would > > > > probably be outside scope of libvirt. > > > > > > I think it would make sense to be a parameter of the add-port command. > > > But it would be for vhost-user related add-port command, I didn't find > > > where/if this is managed in libvirt XML. > > > > For vhost-user, libvirt does not have any interaction with OVS at > > all. If the thing that's using the vhost-user UNIX socket, in turn > > connects to OVS, that's outside scope of libvirt. IOW, for vhost-user > > OVS it seems like that job is for Nova / os-vif to solve. > > I don't think they currently understand the issues involved in > cross-version migration though. This is a complex subject and easy to > get wrong. It would be significantly better to figure it out at libvirt > level since it already deals with cross-version migration issues. Libvirt considers vhost-user to be a blackbox though - it just exposes a UNIX socket, and whatever is on the other end is completely opaque. The fact that the other end might plumb the data stream into openvswitch is not something libvirt should know, as we don't want to end up having to add custom code to libvirt for every different vhost-user server impl. IOW, if the version str can be passed to QEMU, and then onto vhost-user backend in QEMU, then libvirt can be involved. If the version str has to be given to openvswitch that's not for libvirt to deall with. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://entangle-photo.org -o-http://search.cpan.org/~danberr/ :| ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [libvirt] [RFC] Vhost-user backends cross-version migration support
On Thu, Feb 02, 2017 at 03:14:01PM +0100, Maxime Coquelin wrote: > > > On 02/01/2017 12:41 PM, Daniel P. Berrange wrote: > > > > It depends where / how in OVS it needs to be set. The only stuff libvirt > > does with OVS is to run 'add-port' and 'del-port' commands via the ovs > > cli tool. We pass through arguments from the port profile stored in the > > XML config. > > > > > > > > > >> interfaceid='09b11c53-8b5c-4eeb-8f00-d84eaa0aaa4f'/> > > > > > > > > eg those things in get passed as cli args to the 'add-port' > > command. Soo if add-port needs this new version string, then we'd need > > to add the version to the openvswitch virtualport XML. > > > > If the version is provided to OVS in a different command, then it would > > probably be outside scope of libvirt. > > I think it would make sense to be a parameter of the add-port command. > But it would be for vhost-user related add-port command, I didn't find > where/if this is managed in libvirt XML. For vhost-user, libvirt does not have any interaction with OVS at all. If the thing that's using the vhost-user UNIX socket, in turn connects to OVS, that's outside scope of libvirt. IOW, for vhost-user OVS it seems like that job is for Nova / os-vif to solve. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://entangle-photo.org -o-http://search.cpan.org/~danberr/ :| ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [libvirt] [RFC] Vhost-user backends cross-version migration support
On Wed, Feb 01, 2017 at 12:33:22PM +0100, Maxime Coquelin wrote: > > > On 02/01/2017 10:43 AM, Daniel P. Berrange wrote: > > On Wed, Feb 01, 2017 at 10:14:54AM +0100, Michal Privoznik wrote: > > > On 02/01/2017 09:35 AM, Maxime Coquelin wrote: > > > > > > Solution 3: Libvirt queries OVS for vhost backend version string: *OK* > > > > == > > > > > > > > > > > > The idea is to have a table of supported versions, associated to > > > > key/value pairs. Libvirt could query the list of supported versions > > > > strings for each hosts, and select the first common one among all hosts. > > > > > > How does libvirt know what hosts to ask? Libvirt aims on managing a > > > single host. It has no knowledge of other hosts on the network. That's > > > task for upper layers like RHEV, OpenStack, etc. > > > > > > > > > > > Then, libvirt would ask OVS to probe the vhost-user interfaces in the > > > > selected version (compatibility mode). For example host A runs OVS-2.7, > > > > and host B OVS-2.6. Host A's OVS-2.7 has an OVS-2.6 compatibility mode > > > > (e.g. with indirect descriptors disabled), which should be selected at > > > > vhost-user interface probe time. > > > > > > > > Advantage of doing so is that libvirt does not need any update if new > > > > keys are introduced (i.e. it does not need to know how the new keys have > > > > to be handled), all these checks remain in OVS's vhost-user > > > > implementation. > > > > > > And that's where they should stay. Duplicating code between projects > > > will inevitably lead to a divergence. > > > > > > > > > > > Ideally, we would support per vhost-user interface compatibility mode, > > > > which may have an impact also on DPDK API, as the Virtio feature update > > > > API is global, and not per port. > > > > > > In general, I don't think we want any kind of this logic in libvirt. > > > Either: > > > > > > a) fallback logic should be implemented in qemu (e.g. upon migration it > > > should detect that the migrated guest uses certain version and thus set > > > backend to use that version or error out and cancel migration), or > > > > > > b) libvirt would grew another element/attribute to specify version of > > > vhost-user backend in use and do nothing more than pass it to qemu. At > > > the same time, we can provide an API (or extend and existing one, e.g. > > > virsh domcapabilities) to list all available versions on given host. > > > Upper layer, which knows what are the possible hosts suitable for > > > virtualization, can then use this API to ask all the hosts, construct > > > the matrix, select preferred version and put it into libvirt's domain XML. > > > > > > But frankly, I don't like b) that much. Lets put the fact this is OVS > > > aside for a moment. Just pretend this is a generic device in qemu. Would > > > we do the same magic with it? No! Or lets talk about machine types. You > > > spawn -M type$((X+1)) guest and then decide to migrate it to a host with > > > older qemu wich supports just typeX. Well, you get an error. Do we care? > > > Not at all! It's your responsibility (as user/admin) to upgrade the qemu > > > so that it supports new machine type. I think the same applies to OVS. > > > > With machine types, if the latest machine type is X, libvirt allows > > the mgmt app to spawn a guest with mcahine type X-1, so that you can > > later migrate the VM to a host with older QEMU. > > > > With the vhost user device, the VM will always be launched with version > > Y. There's currently no way to request launching the vhost user device > > wtih version Y-1. So even if you set your machine type to X-1 for > > compat with older host, vhost user will be stuck at version Y preventing > > the migration. > > > > One argument would be to say that the vhost user featureset should be > > determined by the machine type version, instead of introducing a new > > version. The complexity is that vhost-user is a pretty dumb device > > from QEMUs POV - most of the interesting logic & the features that > > need to be constrained lie in code outside of QEMU, in whatever > > server is connected to the vhost user socket. > > > > So I can see the value in
Re: [ovs-dev] [libvirt] [RFC] Vhost-user backends cross-version migration support
On Wed, Feb 01, 2017 at 10:14:54AM +0100, Michal Privoznik wrote: > On 02/01/2017 09:35 AM, Maxime Coquelin wrote: > > Solution 3: Libvirt queries OVS for vhost backend version string: *OK* > > == > > > > > > The idea is to have a table of supported versions, associated to > > key/value pairs. Libvirt could query the list of supported versions > > strings for each hosts, and select the first common one among all hosts. > > How does libvirt know what hosts to ask? Libvirt aims on managing a > single host. It has no knowledge of other hosts on the network. That's > task for upper layers like RHEV, OpenStack, etc. > > > > > Then, libvirt would ask OVS to probe the vhost-user interfaces in the > > selected version (compatibility mode). For example host A runs OVS-2.7, > > and host B OVS-2.6. Host A's OVS-2.7 has an OVS-2.6 compatibility mode > > (e.g. with indirect descriptors disabled), which should be selected at > > vhost-user interface probe time. > > > > Advantage of doing so is that libvirt does not need any update if new > > keys are introduced (i.e. it does not need to know how the new keys have > > to be handled), all these checks remain in OVS's vhost-user implementation. > > And that's where they should stay. Duplicating code between projects > will inevitably lead to a divergence. > > > > > Ideally, we would support per vhost-user interface compatibility mode, > > which may have an impact also on DPDK API, as the Virtio feature update > > API is global, and not per port. > > In general, I don't think we want any kind of this logic in libvirt. Either: > > a) fallback logic should be implemented in qemu (e.g. upon migration it > should detect that the migrated guest uses certain version and thus set > backend to use that version or error out and cancel migration), or > > b) libvirt would grew another element/attribute to specify version of > vhost-user backend in use and do nothing more than pass it to qemu. At > the same time, we can provide an API (or extend and existing one, e.g. > virsh domcapabilities) to list all available versions on given host. > Upper layer, which knows what are the possible hosts suitable for > virtualization, can then use this API to ask all the hosts, construct > the matrix, select preferred version and put it into libvirt's domain XML. > > But frankly, I don't like b) that much. Lets put the fact this is OVS > aside for a moment. Just pretend this is a generic device in qemu. Would > we do the same magic with it? No! Or lets talk about machine types. You > spawn -M type$((X+1)) guest and then decide to migrate it to a host with > older qemu wich supports just typeX. Well, you get an error. Do we care? > Not at all! It's your responsibility (as user/admin) to upgrade the qemu > so that it supports new machine type. I think the same applies to OVS. With machine types, if the latest machine type is X, libvirt allows the mgmt app to spawn a guest with mcahine type X-1, so that you can later migrate the VM to a host with older QEMU. With the vhost user device, the VM will always be launched with version Y. There's currently no way to request launching the vhost user device wtih version Y-1. So even if you set your machine type to X-1 for compat with older host, vhost user will be stuck at version Y preventing the migration. One argument would be to say that the vhost user featureset should be determined by the machine type version, instead of introducing a new version. The complexity is that vhost-user is a pretty dumb device from QEMUs POV - most of the interesting logic & the features that need to be constrained lie in code outside of QEMU, in whatever server is connected to the vhost user socket. So I can see the value in allowing a simple version string to be associated with the vhost-user NIC. What I'm unclear about is how we would be able to report capabilities for a host to enumerate what versions were possible. Libvirt doesn't interact with any of the 3rd party vhost user servers, so it is probably out of scope - it'd be upto the higher level mgmt app to talk to DPDK and figure out what versions it supports. That does make me wonder though if libvirt & QEMU need to be involved at all in any part. When provisioning a new guest, the mgmt app presumably has to talk to DPDK to setup the NIC port, so DPDK is ready when QEMU launches and connects. Surely as part of that NIC port setup, it could directly tell DPDK which version or featureset to permit on the port ? It is not obvious why the version string has to be fed in via QEMU. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://entangle-photo.org -o-http://search.cpan.org/~danberr/ :| ___ dev mailing list d...@openvswitch.org https://mail.ope
