Re: [openstack-dev] [nova] [neutron] Adding neutron VIF NUMA locality support
Stephen- FYI, we're teeing up (anti-)affinity and neutron-nova interaction topics for the "generic device management" discussions at the PTG. Please jump on the etherpad [1] and expand/expound in the relevant spots, which, as of this writing, you can find by searching for "aggregate", "vf selection policy", and "interplay". Thanks, Eric [1] https://etherpad.openstack.org/p/nova-ptg-queens-generic-device-management On 09/07/2017 01:07 PM, Mooney, Sean K wrote: > > >> -Original Message- >> From: Stephen Finucane [mailto:sfinu...@redhat.com] >> Sent: Thursday, September 7, 2017 5:42 PM >> To: OpenStack Development Mailing List (not for usage questions) >> >> Cc: Jakub Libosvar ; Karthik Sundaravel >> ; Mooney, Sean K >> Subject: [nova] [neutron] Adding neutron VIF NUMA locality support >> >> Hey, >> >> NUMA locality matters as much for NICs used e.g for Open vSwitch as for >> SR-IOV devices. At the moment, nova support NUMA affinity for PCI >> passthrough devices and SR-IOV devices, but it makes no attempt to do >> the same for other NICs. In the name of NFV enablement, we should >> probably close this gap. > [Mooney, Sean K] I like this idea in general, that said in ovs-dpdk we > modified > ovs to schedule the vhost-user port to be processed on a pmd that is on the > same > Numa node as the vm and reallocate the vhsot user port memory where possible > To also have the same affinity. >> >> I have some ideas around how this could work, but they're fuzzy enough >> and involve exchanging os-vif objects between nova and neutron. This is >> probably the most difficult path as we've been trying to get os-vif >> objects over the nova-neutron wire for a while now, to no success. > [Mooney, Sean K] actually we have so poc code you should proably review > This topic. > https://blueprints.launchpad.net/os-vif/+spec/vif-port-profile > https://review.openstack.org/#/c/490829/ > https://review.openstack.org/#/c/490819/ > https://review.openstack.org/#/c/441590/ > the first patch of the neutron side poc should be up before the ptg. > >> >> Anyone else keen on such a feature? Given that there are a significant >> amount of people from nova, neutron, and general NFV backgrounds at the >> PTG next week, we have a very good opportunity to talk about this >> (either in the nova- neutron sync, if that's not already full, or in >> some hallway somewhere). > [Mooney, Sean K] in terms of basic numa affinity this is not as important > With ovs-dpdk because we make best effort to fix it in ovs this is less > pressing > Then it used to be. It is still important for other backbends but we need > Also have a mechanism to control numa affinity policy like > https://review.openstack.org/#/c/361140/ to not break existing deployments. > > I have some taught about modeling network backbends > in placement and also passing traits requests for neutron that this would dove > tail with so would love to talk to anyone who is interested in this. > By modeling ovs and other network backend in placement and combining that > With traits and the nova-neutron negotiation protocol we support several > Advance usescase. > > By the way ovs-dpdk allow you to specify vhost-port rx/tx queue mapping > to pmd which could give a nice performance boost if done correctly. It > might be worth extending os-vif to do that in the future though this could > equally be handeled by the neutron ovs l2 agent. >> >> At this point in the day, this is probably very much a Rocky feature, >> but we could definitely put in whatever groundwork is necessary this >> cycle to make the work in Rocky as easy possible. > [Mooney, Sean K] I'm hoping we can get the nova neutron negotiation done in > queens. >> >> Cheers, >> Stephen > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [neutron] Adding neutron VIF NUMA locality support
> -Original Message- > From: Stephen Finucane [mailto:sfinu...@redhat.com] > Sent: Thursday, September 7, 2017 5:42 PM > To: OpenStack Development Mailing List (not for usage questions) > > Cc: Jakub Libosvar ; Karthik Sundaravel > ; Mooney, Sean K > Subject: [nova] [neutron] Adding neutron VIF NUMA locality support > > Hey, > > NUMA locality matters as much for NICs used e.g for Open vSwitch as for > SR-IOV devices. At the moment, nova support NUMA affinity for PCI > passthrough devices and SR-IOV devices, but it makes no attempt to do > the same for other NICs. In the name of NFV enablement, we should > probably close this gap. [Mooney, Sean K] I like this idea in general, that said in ovs-dpdk we modified ovs to schedule the vhost-user port to be processed on a pmd that is on the same Numa node as the vm and reallocate the vhsot user port memory where possible To also have the same affinity. > > I have some ideas around how this could work, but they're fuzzy enough > and involve exchanging os-vif objects between nova and neutron. This is > probably the most difficult path as we've been trying to get os-vif > objects over the nova-neutron wire for a while now, to no success. [Mooney, Sean K] actually we have so poc code you should proably review This topic. https://blueprints.launchpad.net/os-vif/+spec/vif-port-profile https://review.openstack.org/#/c/490829/ https://review.openstack.org/#/c/490819/ https://review.openstack.org/#/c/441590/ the first patch of the neutron side poc should be up before the ptg. > > Anyone else keen on such a feature? Given that there are a significant > amount of people from nova, neutron, and general NFV backgrounds at the > PTG next week, we have a very good opportunity to talk about this > (either in the nova- neutron sync, if that's not already full, or in > some hallway somewhere). [Mooney, Sean K] in terms of basic numa affinity this is not as important With ovs-dpdk because we make best effort to fix it in ovs this is less pressing Then it used to be. It is still important for other backbends but we need Also have a mechanism to control numa affinity policy like https://review.openstack.org/#/c/361140/ to not break existing deployments. I have some taught about modeling network backbends in placement and also passing traits requests for neutron that this would dove tail with so would love to talk to anyone who is interested in this. By modeling ovs and other network backend in placement and combining that With traits and the nova-neutron negotiation protocol we support several Advance usescase. By the way ovs-dpdk allow you to specify vhost-port rx/tx queue mapping to pmd which could give a nice performance boost if done correctly. It might be worth extending os-vif to do that in the future though this could equally be handeled by the neutron ovs l2 agent. > > At this point in the day, this is probably very much a Rocky feature, > but we could definitely put in whatever groundwork is necessary this > cycle to make the work in Rocky as easy possible. [Mooney, Sean K] I'm hoping we can get the nova neutron negotiation done in queens. > > Cheers, > Stephen __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] [neutron] Adding neutron VIF NUMA locality support
Hey, NUMA locality matters as much for NICs used e.g for Open vSwitch as for SR-IOV devices. At the moment, nova support NUMA affinity for PCI passthrough devices and SR-IOV devices, but it makes no attempt to do the same for other NICs. In the name of NFV enablement, we should probably close this gap. I have some ideas around how this could work, but they're fuzzy enough and involve exchanging os-vif objects between nova and neutron. This is probably the most difficult path as we've been trying to get os-vif objects over the nova-neutron wire for a while now, to no success. Anyone else keen on such a feature? Given that there are a significant amount of people from nova, neutron, and general NFV backgrounds at the PTG next week, we have a very good opportunity to talk about this (either in the nova- neutron sync, if that's not already full, or in some hallway somewhere). At this point in the day, this is probably very much a Rocky feature, but we could definitely put in whatever groundwork is necessary this cycle to make the work in Rocky as easy possible. Cheers, Stephen __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev