Re: [ovs-discuss] [External] : Re: SR-IOV OVN OpenStack Mellanox
Thanks, that's been very helpful
On 15/02/2022 19:13, Piotr Misiak wrote:
Hi Tony,
AFAIK Mellanox ConnectX-5 and newer supports OVS DP offloading but it
may be not fully supported (I mean not all match ops).
On Tue, 15 Feb 2022 at 20:02 Tony Liu wrote:
You want virtual networking data plane on SmartNIC.
This idea came up a few years ago, but I am not aware of any
SmartNIC supporting
any virtual networking implementation. I think, it's because that,
it's lots easier to
achieve the same with physical networking device.
Tony
From: Brendan Doyle
Sent: February 15, 2022 10:06 AM
To: Tony Liu; Satish Patel
Cc: ovs-discuss
Subject: Re: [ovs-discuss] [External] : Re: SR-IOV OVN OpenStack
Mellanox
I want the libvert/KVM/QEMU VMs that I currently create that
"hook" into
an OVN
overlay network using the libvert 'openvswitch' network to work
in the same
OVN overlay Network with the various Logical Switches, routers
Gateways,
NAT rules
ACLs etc to work as is except that they bypass br-int and go to the
SmartNIC. So
somehow the OVS flows that OVN creates in br-int, will need to be
implemented on
the smart NIC.
It is not just enough to have the smart NIC do the encapsulation, the
OVN Logical Gateways
do various NAT and routing operations, the OVN Logical switches have
various ACLs
rules for security. etc. If traffic by-passes br-int and goes directly
to the SmartNIC All these
still need to be "honored" by the smart NIC . Like I said I though
I saw
a Mellanox presentation
that talks about doing something like this with representor ports.
But I
don't know
the details.
On 15/02/2022 17:27, Tony Liu wrote:
> OVN is virtual networking implementation.
> With OpenStack Neutron, follow the doc, VM will directly connect
to NIC VF.
> I don't see OVN integration is needed.
>
> It's another story if you want connectivity between OVN-based VM and
> SRIOV-based VM.
>
> Tony
>
> From: Brendan Doyle
> Sent: February 15, 2022 08:53 AM
> To: Tony Liu; Satish Patel
> Cc: ovs-discuss
> Subject: Re: [ovs-discuss] [External] : Re: SR-IOV OVN OpenStack
Mellanox
>
>
> Yes but how is that integrated into OVN?
>
> On 15/02/2022 16:41, Tony Liu wrote:
>> SRIOV connects VM directly to NIC VF and bypasses virtual
networking stack.
>> SmartNIC is another story where virtual networking stack can be
installed on the NIC.
>> SRIOV is supported by Neutron.
>>
https://urldefense.com/v3/__https://docs.openstack.org/neutron/xena/admin/config-sriov.html__;!!ACWV5N9M2RV99hQ!eHoUiPE3yK4G7wru3NHOe4xZEY8JZCTBMzsMhhoNnkr7oZQDSf-4QrFyS4Y5t7g5lTU$
>>
>> Tony
>>
>> From: discuss on behalf
of Brendan Doyle
>> Sent: February 15, 2022 07:10 AM
>> To: Satish Patel
>> Cc: ovs-discuss
>> Subject: Re: [ovs-discuss] [External] : Re: SR-IOV OVN
OpenStack Mellanox
>>
>> Kinda looking for at a high level yes it is possible and is
integrated into
>> ovn control plane, or not not there yet. At a high level first.
>> And is anyone doing this.
>>
>>
>> On 15/02/2022 14:03, Satish Patel wrote:
>>> Not sure if this is what you are looking for
>>>
https://urldefense.com/v3/__https://docs.nvidia.com/networking/display/TAN10/ASAP*OVS*Offload__;Kys!!ACWV5N9M2RV99hQ!dFnukkvR8ggDcHotwyqXNVu8B3dWlc7LBGXbc5fECYqHdok6NIeoKkz5aSlp-RCKC10$
>>>
>>> On Tue, Feb 15, 2022 at 5:47 AM Brendan Doyle
wrote:
>>>> Hi,
>>>>
>>>> I'm trying to understand if OVN supports SR-IOV. I found some
OpentStack
>>>> documentation:
>>>>
https://urldefense.com/v3/__https://docs.openstack.org/project-deploy-guide/charm-deployment-guide/wallaby/app-ovn.html*configuration__;Iw!!ACWV5N9M2RV99hQ!dFnukkvR8ggDcHotwyqXNVu8B3dWlc7LBGXbc5fECYqHdok6NIeoKkz5aSlpwu75q1Q$
>>>> that suggests it might, but it is short on details, with
specifics
>>>> abstracted via the OpenStack CMS.
>>>>
>>>> Also in the OVN Architecture documentation there are hints to
support:
>>>>
>>>> "For instances connected through representor ports,
typically used
>>>> with hardware
>>>> offload, the ovn-controller may on CMS direction
consult a VIF
>>>> plug pro
Re: [ovs-discuss] [External] : Re: SR-IOV OVN OpenStack Mellanox
Hi Tony, AFAIK Mellanox ConnectX-5 and newer supports OVS DP offloading but it may be not fully supported (I mean not all match ops). On Tue, 15 Feb 2022 at 20:02 Tony Liu wrote: > You want virtual networking data plane on SmartNIC. > This idea came up a few years ago, but I am not aware of any SmartNIC > supporting > any virtual networking implementation. I think, it's because that, it's > lots easier to > achieve the same with physical networking device. > > Tony > > From: Brendan Doyle > Sent: February 15, 2022 10:06 AM > To: Tony Liu; Satish Patel > Cc: ovs-discuss > Subject: Re: [ovs-discuss] [External] : Re: SR-IOV OVN OpenStack Mellanox > > > I want the libvert/KVM/QEMU VMs that I currently create that "hook" into > an OVN > overlay network using the libvert 'openvswitch' network to work in the > same > OVN overlay Network with the various Logical Switches, routers Gateways, > NAT rules > ACLs etc to work as is except that they bypass br-int and go to the > SmartNIC. So > somehow the OVS flows that OVN creates in br-int, will need to be > implemented on > the smart NIC. > > It is not just enough to have the smart NIC do the encapsulation, the > OVN Logical Gateways > do various NAT and routing operations, the OVN Logical switches have > various ACLs > rules for security. etc. If traffic by-passes br-int and goes directly > to the SmartNIC All these > still need to be "honored" by the smart NIC . Like I said I though I saw > a Mellanox presentation > that talks about doing something like this with representor ports. But I > don't know > the details. > > > On 15/02/2022 17:27, Tony Liu wrote: > > OVN is virtual networking implementation. > > With OpenStack Neutron, follow the doc, VM will directly connect to NIC > VF. > > I don't see OVN integration is needed. > > > > It's another story if you want connectivity between OVN-based VM and > > SRIOV-based VM. > > > > Tony > > ____________ > > From: Brendan Doyle > > Sent: February 15, 2022 08:53 AM > > To: Tony Liu; Satish Patel > > Cc: ovs-discuss > > Subject: Re: [ovs-discuss] [External] : Re: SR-IOV OVN OpenStack Mellanox > > > > > > Yes but how is that integrated into OVN? > > > > On 15/02/2022 16:41, Tony Liu wrote: > >> SRIOV connects VM directly to NIC VF and bypasses virtual networking > stack. > >> SmartNIC is another story where virtual networking stack can be > installed on the NIC. > >> SRIOV is supported by Neutron. > >> > https://urldefense.com/v3/__https://docs.openstack.org/neutron/xena/admin/config-sriov.html__;!!ACWV5N9M2RV99hQ!eHoUiPE3yK4G7wru3NHOe4xZEY8JZCTBMzsMhhoNnkr7oZQDSf-4QrFyS4Y5t7g5lTU$ > >> > >> Tony > >> > >> From: discuss on behalf of > Brendan Doyle > >> Sent: February 15, 2022 07:10 AM > >> To: Satish Patel > >> Cc: ovs-discuss > >> Subject: Re: [ovs-discuss] [External] : Re: SR-IOV OVN OpenStack > Mellanox > >> > >> Kinda looking for at a high level yes it is possible and is integrated > into > >> ovn control plane, or not not there yet. At a high level first. > >> And is anyone doing this. > >> > >> > >> On 15/02/2022 14:03, Satish Patel wrote: > >>> Not sure if this is what you are looking for > >>> > https://urldefense.com/v3/__https://docs.nvidia.com/networking/display/TAN10/ASAP*OVS*Offload__;Kys!!ACWV5N9M2RV99hQ!dFnukkvR8ggDcHotwyqXNVu8B3dWlc7LBGXbc5fECYqHdok6NIeoKkz5aSlp-RCKC10$ > >>> > >>> On Tue, Feb 15, 2022 at 5:47 AM Brendan Doyle < > brendan.do...@oracle.com> wrote: > >>>> Hi, > >>>> > >>>> I'm trying to understand if OVN supports SR-IOV. I found some > OpentStack > >>>> documentation: > >>>> > https://urldefense.com/v3/__https://docs.openstack.org/project-deploy-guide/charm-deployment-guide/wallaby/app-ovn.html*configuration__;Iw!!ACWV5N9M2RV99hQ!dFnukkvR8ggDcHotwyqXNVu8B3dWlc7LBGXbc5fECYqHdok6NIeoKkz5aSlpwu75q1Q$ > >>>> that suggests it might, but it is short on details, with specifics > >>>> abstracted via the OpenStack CMS. > >>>> > >>>> Also in the OVN Architecture documentation there are hints to support: > >>>> > >>>> "For instances connected through representor ports, typically used > >>>> with hardware > >>>> offload, the ovn-controller may on CMS directio
Re: [ovs-discuss] [External] : Re: SR-IOV OVN OpenStack Mellanox
You want virtual networking data plane on SmartNIC. This idea came up a few years ago, but I am not aware of any SmartNIC supporting any virtual networking implementation. I think, it's because that, it's lots easier to achieve the same with physical networking device. Tony From: Brendan Doyle Sent: February 15, 2022 10:06 AM To: Tony Liu; Satish Patel Cc: ovs-discuss Subject: Re: [ovs-discuss] [External] : Re: SR-IOV OVN OpenStack Mellanox I want the libvert/KVM/QEMU VMs that I currently create that "hook" into an OVN overlay network using the libvert 'openvswitch' network to work in the same OVN overlay Network with the various Logical Switches, routers Gateways, NAT rules ACLs etc to work as is except that they bypass br-int and go to the SmartNIC. So somehow the OVS flows that OVN creates in br-int, will need to be implemented on the smart NIC. It is not just enough to have the smart NIC do the encapsulation, the OVN Logical Gateways do various NAT and routing operations, the OVN Logical switches have various ACLs rules for security. etc. If traffic by-passes br-int and goes directly to the SmartNIC All these still need to be "honored" by the smart NIC . Like I said I though I saw a Mellanox presentation that talks about doing something like this with representor ports. But I don't know the details. On 15/02/2022 17:27, Tony Liu wrote: > OVN is virtual networking implementation. > With OpenStack Neutron, follow the doc, VM will directly connect to NIC VF. > I don't see OVN integration is needed. > > It's another story if you want connectivity between OVN-based VM and > SRIOV-based VM. > > Tony > > From: Brendan Doyle > Sent: February 15, 2022 08:53 AM > To: Tony Liu; Satish Patel > Cc: ovs-discuss > Subject: Re: [ovs-discuss] [External] : Re: SR-IOV OVN OpenStack Mellanox > > > Yes but how is that integrated into OVN? > > On 15/02/2022 16:41, Tony Liu wrote: >> SRIOV connects VM directly to NIC VF and bypasses virtual networking stack. >> SmartNIC is another story where virtual networking stack can be installed on >> the NIC. >> SRIOV is supported by Neutron. >> https://urldefense.com/v3/__https://docs.openstack.org/neutron/xena/admin/config-sriov.html__;!!ACWV5N9M2RV99hQ!eHoUiPE3yK4G7wru3NHOe4xZEY8JZCTBMzsMhhoNnkr7oZQDSf-4QrFyS4Y5t7g5lTU$ >> >> Tony >> >> From: discuss on behalf of Brendan >> Doyle >> Sent: February 15, 2022 07:10 AM >> To: Satish Patel >> Cc: ovs-discuss >> Subject: Re: [ovs-discuss] [External] : Re: SR-IOV OVN OpenStack Mellanox >> >> Kinda looking for at a high level yes it is possible and is integrated into >> ovn control plane, or not not there yet. At a high level first. >> And is anyone doing this. >> >> >> On 15/02/2022 14:03, Satish Patel wrote: >>> Not sure if this is what you are looking for >>> https://urldefense.com/v3/__https://docs.nvidia.com/networking/display/TAN10/ASAP*OVS*Offload__;Kys!!ACWV5N9M2RV99hQ!dFnukkvR8ggDcHotwyqXNVu8B3dWlc7LBGXbc5fECYqHdok6NIeoKkz5aSlp-RCKC10$ >>> >>> On Tue, Feb 15, 2022 at 5:47 AM Brendan Doyle >>> wrote: >>>> Hi, >>>> >>>> I'm trying to understand if OVN supports SR-IOV. I found some OpentStack >>>> documentation: >>>> https://urldefense.com/v3/__https://docs.openstack.org/project-deploy-guide/charm-deployment-guide/wallaby/app-ovn.html*configuration__;Iw!!ACWV5N9M2RV99hQ!dFnukkvR8ggDcHotwyqXNVu8B3dWlc7LBGXbc5fECYqHdok6NIeoKkz5aSlpwu75q1Q$ >>>> that suggests it might, but it is short on details, with specifics >>>> abstracted via the OpenStack CMS. >>>> >>>> Also in the OVN Architecture documentation there are hints to support: >>>> >>>> "For instances connected through representor ports, typically used >>>> with hardware >>>> offload, the ovn-controller may on CMS direction consult a VIF >>>> plug provider for >>>> representor port lookup and plug them into the integration bridge >>>> (please refer to >>>> Docu mentation/topics/vif-plug-providers/vif-plug-providers.rst for >>>> more information)." >>>> >>>> But again short on details. >>>> >>>> So I believe something like a CX-5/6/7 would have the capability to do >>>> this, but here would have to be some >>>> sort of OVN hook for the OVS flows created by OVN to be "copied/moved" >>>> to the H/W so that encapsulation, >>>> NAT, distributed routin
Re: [ovs-discuss] [External] : Re: SR-IOV OVN OpenStack Mellanox
Hi Brendan, Smart NIC offloads DataPath flows from OVS (because OVN uses OVS underneath). If you like to integrate OVN with Smart NIC, it have to support all flow operations used by OVN. Especially conntrack offloading, because OVN flows extensively use ct() operations in flow matching and flow actions. You also need Nova os-vif plugin support for SmartNIC VF port representors. First packet always is analyzed in the slow path by OVS and in the result DP flow is created and offloaded to SmartNIC. On Tue, 15 Feb 2022 at 19:06 Brendan Doyle wrote: > > I want the libvert/KVM/QEMU VMs that I currently create that "hook" into > an OVN > overlay network using the libvert 'openvswitch' network to work in the > same > OVN overlay Network with the various Logical Switches, routers Gateways, > NAT rules > ACLs etc to work as is except that they bypass br-int and go to the > SmartNIC. So > somehow the OVS flows that OVN creates in br-int, will need to be > implemented on > the smart NIC. > > It is not just enough to have the smart NIC do the encapsulation, the > OVN Logical Gateways > do various NAT and routing operations, the OVN Logical switches have > various ACLs > rules for security. etc. If traffic by-passes br-int and goes directly > to the SmartNIC All these > still need to be "honored" by the smart NIC . Like I said I though I saw > a Mellanox presentation > that talks about doing something like this with representor ports. But I > don't know > the details. > > > On 15/02/2022 17:27, Tony Liu wrote: > > OVN is virtual networking implementation. > > With OpenStack Neutron, follow the doc, VM will directly connect to NIC > VF. > > I don't see OVN integration is needed. > > > > It's another story if you want connectivity between OVN-based VM and > > SRIOV-based VM. > > > > Tony > > ____________ > > From: Brendan Doyle > > Sent: February 15, 2022 08:53 AM > > To: Tony Liu; Satish Patel > > Cc: ovs-discuss > > Subject: Re: [ovs-discuss] [External] : Re: SR-IOV OVN OpenStack Mellanox > > > > > > Yes but how is that integrated into OVN? > > > > On 15/02/2022 16:41, Tony Liu wrote: > >> SRIOV connects VM directly to NIC VF and bypasses virtual networking > stack. > >> SmartNIC is another story where virtual networking stack can be > installed on the NIC. > >> SRIOV is supported by Neutron. > >> > https://urldefense.com/v3/__https://docs.openstack.org/neutron/xena/admin/config-sriov.html__;!!ACWV5N9M2RV99hQ!eHoUiPE3yK4G7wru3NHOe4xZEY8JZCTBMzsMhhoNnkr7oZQDSf-4QrFyS4Y5t7g5lTU$ > >> > >> Tony > >> > >> From: discuss on behalf of > Brendan Doyle > >> Sent: February 15, 2022 07:10 AM > >> To: Satish Patel > >> Cc: ovs-discuss > >> Subject: Re: [ovs-discuss] [External] : Re: SR-IOV OVN OpenStack > Mellanox > >> > >> Kinda looking for at a high level yes it is possible and is integrated > into > >> ovn control plane, or not not there yet. At a high level first. > >> And is anyone doing this. > >> > >> > >> On 15/02/2022 14:03, Satish Patel wrote: > >>> Not sure if this is what you are looking for > >>> > https://urldefense.com/v3/__https://docs.nvidia.com/networking/display/TAN10/ASAP*OVS*Offload__;Kys!!ACWV5N9M2RV99hQ!dFnukkvR8ggDcHotwyqXNVu8B3dWlc7LBGXbc5fECYqHdok6NIeoKkz5aSlp-RCKC10$ > >>> > >>> On Tue, Feb 15, 2022 at 5:47 AM Brendan Doyle < > brendan.do...@oracle.com> wrote: > >>>> Hi, > >>>> > >>>> I'm trying to understand if OVN supports SR-IOV. I found some > OpentStack > >>>> documentation: > >>>> > https://urldefense.com/v3/__https://docs.openstack.org/project-deploy-guide/charm-deployment-guide/wallaby/app-ovn.html*configuration__;Iw!!ACWV5N9M2RV99hQ!dFnukkvR8ggDcHotwyqXNVu8B3dWlc7LBGXbc5fECYqHdok6NIeoKkz5aSlpwu75q1Q$ > >>>> that suggests it might, but it is short on details, with specifics > >>>> abstracted via the OpenStack CMS. > >>>> > >>>> Also in the OVN Architecture documentation there are hints to support: > >>>> > >>>> "For instances connected through representor ports, typically used > >>>> with hardware > >>>> offload, the ovn-controller may on CMS direction consult a > VIF > >>>> plug provider for > >>>> representor port lookup and plug them into the integration > bridge > >>>> (please r
Re: [ovs-discuss] [External] : Re: SR-IOV OVN OpenStack Mellanox
I want the libvert/KVM/QEMU VMs that I currently create that "hook" into an OVN overlay network using the libvert 'openvswitch' network to work in the same OVN overlay Network with the various Logical Switches, routers Gateways, NAT rules ACLs etc to work as is except that they bypass br-int and go to the SmartNIC. So somehow the OVS flows that OVN creates in br-int, will need to be implemented on the smart NIC. It is not just enough to have the smart NIC do the encapsulation, the OVN Logical Gateways do various NAT and routing operations, the OVN Logical switches have various ACLs rules for security. etc. If traffic by-passes br-int and goes directly to the SmartNIC All these still need to be "honored" by the smart NIC . Like I said I though I saw a Mellanox presentation that talks about doing something like this with representor ports. But I don't know the details. On 15/02/2022 17:27, Tony Liu wrote: OVN is virtual networking implementation. With OpenStack Neutron, follow the doc, VM will directly connect to NIC VF. I don't see OVN integration is needed. It's another story if you want connectivity between OVN-based VM and SRIOV-based VM. Tony From: Brendan Doyle Sent: February 15, 2022 08:53 AM To: Tony Liu; Satish Patel Cc: ovs-discuss Subject: Re: [ovs-discuss] [External] : Re: SR-IOV OVN OpenStack Mellanox Yes but how is that integrated into OVN? On 15/02/2022 16:41, Tony Liu wrote: SRIOV connects VM directly to NIC VF and bypasses virtual networking stack. SmartNIC is another story where virtual networking stack can be installed on the NIC. SRIOV is supported by Neutron. https://urldefense.com/v3/__https://docs.openstack.org/neutron/xena/admin/config-sriov.html__;!!ACWV5N9M2RV99hQ!eHoUiPE3yK4G7wru3NHOe4xZEY8JZCTBMzsMhhoNnkr7oZQDSf-4QrFyS4Y5t7g5lTU$ Tony From: discuss on behalf of Brendan Doyle Sent: February 15, 2022 07:10 AM To: Satish Patel Cc: ovs-discuss Subject: Re: [ovs-discuss] [External] : Re: SR-IOV OVN OpenStack Mellanox Kinda looking for at a high level yes it is possible and is integrated into ovn control plane, or not not there yet. At a high level first. And is anyone doing this. On 15/02/2022 14:03, Satish Patel wrote: Not sure if this is what you are looking for https://urldefense.com/v3/__https://docs.nvidia.com/networking/display/TAN10/ASAP*OVS*Offload__;Kys!!ACWV5N9M2RV99hQ!dFnukkvR8ggDcHotwyqXNVu8B3dWlc7LBGXbc5fECYqHdok6NIeoKkz5aSlp-RCKC10$ On Tue, Feb 15, 2022 at 5:47 AM Brendan Doyle wrote: Hi, I'm trying to understand if OVN supports SR-IOV. I found some OpentStack documentation: https://urldefense.com/v3/__https://docs.openstack.org/project-deploy-guide/charm-deployment-guide/wallaby/app-ovn.html*configuration__;Iw!!ACWV5N9M2RV99hQ!dFnukkvR8ggDcHotwyqXNVu8B3dWlc7LBGXbc5fECYqHdok6NIeoKkz5aSlpwu75q1Q$ that suggests it might, but it is short on details, with specifics abstracted via the OpenStack CMS. Also in the OVN Architecture documentation there are hints to support: "For instances connected through representor ports, typically used with hardware offload, the ovn-controller may on CMS direction consult a VIF plug provider for representor port lookup and plug them into the integration bridge (please refer to Docu mentation/topics/vif-plug-providers/vif-plug-providers.rst for more information)." But again short on details. So I believe something like a CX-5/6/7 would have the capability to do this, but here would have to be some sort of OVN hook for the OVS flows created by OVN to be "copied/moved" to the H/W so that encapsulation, NAT, distributed routing ACLs etc is done in the hardware. I can't find any details on this nor what would the control plane for programing the hardware be to do that, ovn-nbctl?, ovsdbapp? some other out of band control plane?. Also from what I gather from the OpenStack docs this seems experimental and limited to VXLAN encapsulation? At present I use a libvirt OVN hook that hooks KVM/QEMU VMs into OVN br-int but these are using software VIFs. I'm trying to ascertain if can have these VM use SR-IOV, and still have them integrated into the OVN logical networks. Any pointers would welcome. Thanks Brendan ___ discuss mailing list disc...@openvswitch.org https://urldefense.com/v3/__https://mail.openvswitch.org/mailman/listinfo/ovs-discuss__;!!ACWV5N9M2RV99hQ!dFnukkvR8ggDcHotwyqXNVu8B3dWlc7LBGXbc5fECYqHdok6NIeoKkz5aSlpNIZip9k$ ___ discuss mailing list disc...@openvswitch.org https://urldefense.com/v3/__https://mail.openvswitch.org/mailman/listinfo/ovs-discuss__;!!ACWV5N9M2RV99hQ!eHoUiPE3yK4G7wru3NHOe4xZEY8JZCTBMzsMhhoNnkr7oZQDSf-4QrFyS4Y5EkKIAjo$ ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] [External] : Re: SR-IOV OVN OpenStack Mellanox
OVN is virtual networking implementation. With OpenStack Neutron, follow the doc, VM will directly connect to NIC VF. I don't see OVN integration is needed. It's another story if you want connectivity between OVN-based VM and SRIOV-based VM. Tony From: Brendan Doyle Sent: February 15, 2022 08:53 AM To: Tony Liu; Satish Patel Cc: ovs-discuss Subject: Re: [ovs-discuss] [External] : Re: SR-IOV OVN OpenStack Mellanox Yes but how is that integrated into OVN? On 15/02/2022 16:41, Tony Liu wrote: > SRIOV connects VM directly to NIC VF and bypasses virtual networking stack. > SmartNIC is another story where virtual networking stack can be installed on > the NIC. > SRIOV is supported by Neutron. > https://urldefense.com/v3/__https://docs.openstack.org/neutron/xena/admin/config-sriov.html__;!!ACWV5N9M2RV99hQ!eHoUiPE3yK4G7wru3NHOe4xZEY8JZCTBMzsMhhoNnkr7oZQDSf-4QrFyS4Y5t7g5lTU$ > > Tony > > From: discuss on behalf of Brendan > Doyle > Sent: February 15, 2022 07:10 AM > To: Satish Patel > Cc: ovs-discuss > Subject: Re: [ovs-discuss] [External] : Re: SR-IOV OVN OpenStack Mellanox > > Kinda looking for at a high level yes it is possible and is integrated into > ovn control plane, or not not there yet. At a high level first. > And is anyone doing this. > > > On 15/02/2022 14:03, Satish Patel wrote: >> Not sure if this is what you are looking for >> https://urldefense.com/v3/__https://docs.nvidia.com/networking/display/TAN10/ASAP*OVS*Offload__;Kys!!ACWV5N9M2RV99hQ!dFnukkvR8ggDcHotwyqXNVu8B3dWlc7LBGXbc5fECYqHdok6NIeoKkz5aSlp-RCKC10$ >> >> On Tue, Feb 15, 2022 at 5:47 AM Brendan Doyle >> wrote: >>> Hi, >>> >>> I'm trying to understand if OVN supports SR-IOV. I found some OpentStack >>> documentation: >>> https://urldefense.com/v3/__https://docs.openstack.org/project-deploy-guide/charm-deployment-guide/wallaby/app-ovn.html*configuration__;Iw!!ACWV5N9M2RV99hQ!dFnukkvR8ggDcHotwyqXNVu8B3dWlc7LBGXbc5fECYqHdok6NIeoKkz5aSlpwu75q1Q$ >>> that suggests it might, but it is short on details, with specifics >>> abstracted via the OpenStack CMS. >>> >>> Also in the OVN Architecture documentation there are hints to support: >>> >>> "For instances connected through representor ports, typically used >>> with hardware >>> offload, the ovn-controller may on CMS direction consult a VIF >>> plug provider for >>> representor port lookup and plug them into the integration bridge >>> (please refer to >>> Docu mentation/topics/vif-plug-providers/vif-plug-providers.rst for >>> more information)." >>> >>> But again short on details. >>> >>> So I believe something like a CX-5/6/7 would have the capability to do >>> this, but here would have to be some >>> sort of OVN hook for the OVS flows created by OVN to be "copied/moved" >>> to the H/W so that encapsulation, >>> NAT, distributed routing ACLs etc is done in the hardware. I can't find >>> any details on this nor what would the >>> control plane for programing the hardware be to do that, ovn-nbctl?, >>> ovsdbapp? some other out of band control plane?. >>> Also from what I gather from the OpenStack docs this seems experimental >>> and limited to VXLAN encapsulation? >>> >>> At present I use a libvirt OVN hook that hooks KVM/QEMU VMs into OVN >>> br-int but these are using >>> software VIFs. I'm trying to ascertain if can have these VM use SR-IOV, >>> and still have them integrated >>> into the OVN logical networks. >>> >>> >>> Any pointers would welcome. >>> >>> Thanks >>> >>> >>> Brendan >>> >>> ___ >>> discuss mailing list >>> disc...@openvswitch.org >>> https://urldefense.com/v3/__https://mail.openvswitch.org/mailman/listinfo/ovs-discuss__;!!ACWV5N9M2RV99hQ!dFnukkvR8ggDcHotwyqXNVu8B3dWlc7LBGXbc5fECYqHdok6NIeoKkz5aSlpNIZip9k$ > ___ > discuss mailing list > disc...@openvswitch.org > https://urldefense.com/v3/__https://mail.openvswitch.org/mailman/listinfo/ovs-discuss__;!!ACWV5N9M2RV99hQ!eHoUiPE3yK4G7wru3NHOe4xZEY8JZCTBMzsMhhoNnkr7oZQDSf-4QrFyS4Y5EkKIAjo$ ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] [External] : Re: SR-IOV OVN OpenStack Mellanox
Yes but how is that integrated into OVN? On 15/02/2022 16:41, Tony Liu wrote: SRIOV connects VM directly to NIC VF and bypasses virtual networking stack. SmartNIC is another story where virtual networking stack can be installed on the NIC. SRIOV is supported by Neutron. https://urldefense.com/v3/__https://docs.openstack.org/neutron/xena/admin/config-sriov.html__;!!ACWV5N9M2RV99hQ!eHoUiPE3yK4G7wru3NHOe4xZEY8JZCTBMzsMhhoNnkr7oZQDSf-4QrFyS4Y5t7g5lTU$ Tony From: discuss on behalf of Brendan Doyle Sent: February 15, 2022 07:10 AM To: Satish Patel Cc: ovs-discuss Subject: Re: [ovs-discuss] [External] : Re: SR-IOV OVN OpenStack Mellanox Kinda looking for at a high level yes it is possible and is integrated into ovn control plane, or not not there yet. At a high level first. And is anyone doing this. On 15/02/2022 14:03, Satish Patel wrote: Not sure if this is what you are looking for https://urldefense.com/v3/__https://docs.nvidia.com/networking/display/TAN10/ASAP*OVS*Offload__;Kys!!ACWV5N9M2RV99hQ!dFnukkvR8ggDcHotwyqXNVu8B3dWlc7LBGXbc5fECYqHdok6NIeoKkz5aSlp-RCKC10$ On Tue, Feb 15, 2022 at 5:47 AM Brendan Doyle wrote: Hi, I'm trying to understand if OVN supports SR-IOV. I found some OpentStack documentation: https://urldefense.com/v3/__https://docs.openstack.org/project-deploy-guide/charm-deployment-guide/wallaby/app-ovn.html*configuration__;Iw!!ACWV5N9M2RV99hQ!dFnukkvR8ggDcHotwyqXNVu8B3dWlc7LBGXbc5fECYqHdok6NIeoKkz5aSlpwu75q1Q$ that suggests it might, but it is short on details, with specifics abstracted via the OpenStack CMS. Also in the OVN Architecture documentation there are hints to support: "For instances connected through representor ports, typically used with hardware offload, the ovn-controller may on CMS direction consult a VIF plug provider for representor port lookup and plug them into the integration bridge (please refer to Docu mentation/topics/vif-plug-providers/vif-plug-providers.rst for more information)." But again short on details. So I believe something like a CX-5/6/7 would have the capability to do this, but here would have to be some sort of OVN hook for the OVS flows created by OVN to be "copied/moved" to the H/W so that encapsulation, NAT, distributed routing ACLs etc is done in the hardware. I can't find any details on this nor what would the control plane for programing the hardware be to do that, ovn-nbctl?, ovsdbapp? some other out of band control plane?. Also from what I gather from the OpenStack docs this seems experimental and limited to VXLAN encapsulation? At present I use a libvirt OVN hook that hooks KVM/QEMU VMs into OVN br-int but these are using software VIFs. I'm trying to ascertain if can have these VM use SR-IOV, and still have them integrated into the OVN logical networks. Any pointers would welcome. Thanks Brendan ___ discuss mailing list disc...@openvswitch.org https://urldefense.com/v3/__https://mail.openvswitch.org/mailman/listinfo/ovs-discuss__;!!ACWV5N9M2RV99hQ!dFnukkvR8ggDcHotwyqXNVu8B3dWlc7LBGXbc5fECYqHdok6NIeoKkz5aSlpNIZip9k$ ___ discuss mailing list disc...@openvswitch.org https://urldefense.com/v3/__https://mail.openvswitch.org/mailman/listinfo/ovs-discuss__;!!ACWV5N9M2RV99hQ!eHoUiPE3yK4G7wru3NHOe4xZEY8JZCTBMzsMhhoNnkr7oZQDSf-4QrFyS4Y5EkKIAjo$ ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] [External] : Re: SR-IOV OVN OpenStack Mellanox
SRIOV connects VM directly to NIC VF and bypasses virtual networking stack. SmartNIC is another story where virtual networking stack can be installed on the NIC. SRIOV is supported by Neutron. https://docs.openstack.org/neutron/xena/admin/config-sriov.html Tony From: discuss on behalf of Brendan Doyle Sent: February 15, 2022 07:10 AM To: Satish Patel Cc: ovs-discuss Subject: Re: [ovs-discuss] [External] : Re: SR-IOV OVN OpenStack Mellanox Kinda looking for at a high level yes it is possible and is integrated into ovn control plane, or not not there yet. At a high level first. And is anyone doing this. On 15/02/2022 14:03, Satish Patel wrote: > Not sure if this is what you are looking for > https://urldefense.com/v3/__https://docs.nvidia.com/networking/display/TAN10/ASAP*OVS*Offload__;Kys!!ACWV5N9M2RV99hQ!dFnukkvR8ggDcHotwyqXNVu8B3dWlc7LBGXbc5fECYqHdok6NIeoKkz5aSlp-RCKC10$ > > On Tue, Feb 15, 2022 at 5:47 AM Brendan Doyle > wrote: >> Hi, >> >> I'm trying to understand if OVN supports SR-IOV. I found some OpentStack >> documentation: >> https://urldefense.com/v3/__https://docs.openstack.org/project-deploy-guide/charm-deployment-guide/wallaby/app-ovn.html*configuration__;Iw!!ACWV5N9M2RV99hQ!dFnukkvR8ggDcHotwyqXNVu8B3dWlc7LBGXbc5fECYqHdok6NIeoKkz5aSlpwu75q1Q$ >> that suggests it might, but it is short on details, with specifics >> abstracted via the OpenStack CMS. >> >> Also in the OVN Architecture documentation there are hints to support: >> >> "For instances connected through representor ports, typically used >> with hardware >> offload, the ovn-controller may on CMS direction consult a VIF >> plug provider for >> representor port lookup and plug them into the integration bridge >> (please refer to >>Docu mentation/topics/vif-plug-providers/vif-plug-providers.rst for >> more information)." >> >> But again short on details. >> >> So I believe something like a CX-5/6/7 would have the capability to do >> this, but here would have to be some >> sort of OVN hook for the OVS flows created by OVN to be "copied/moved" >> to the H/W so that encapsulation, >> NAT, distributed routing ACLs etc is done in the hardware. I can't find >> any details on this nor what would the >> control plane for programing the hardware be to do that, ovn-nbctl?, >> ovsdbapp? some other out of band control plane?. >> Also from what I gather from the OpenStack docs this seems experimental >> and limited to VXLAN encapsulation? >> >> At present I use a libvirt OVN hook that hooks KVM/QEMU VMs into OVN >> br-int but these are using >> software VIFs. I'm trying to ascertain if can have these VM use SR-IOV, >> and still have them integrated >> into the OVN logical networks. >> >> >> Any pointers would welcome. >> >> Thanks >> >> >> Brendan >> >> ___ >> discuss mailing list >> disc...@openvswitch.org >> https://urldefense.com/v3/__https://mail.openvswitch.org/mailman/listinfo/ovs-discuss__;!!ACWV5N9M2RV99hQ!dFnukkvR8ggDcHotwyqXNVu8B3dWlc7LBGXbc5fECYqHdok6NIeoKkz5aSlpNIZip9k$ ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] [External] : Re: SR-IOV OVN OpenStack Mellanox
Kinda looking for at a high level yes it is possible and is integrated into ovn control plane, or not not there yet. At a high level first. And is anyone doing this. On 15/02/2022 14:03, Satish Patel wrote: Not sure if this is what you are looking for https://urldefense.com/v3/__https://docs.nvidia.com/networking/display/TAN10/ASAP*OVS*Offload__;Kys!!ACWV5N9M2RV99hQ!dFnukkvR8ggDcHotwyqXNVu8B3dWlc7LBGXbc5fECYqHdok6NIeoKkz5aSlp-RCKC10$ On Tue, Feb 15, 2022 at 5:47 AM Brendan Doyle wrote: Hi, I'm trying to understand if OVN supports SR-IOV. I found some OpentStack documentation: https://urldefense.com/v3/__https://docs.openstack.org/project-deploy-guide/charm-deployment-guide/wallaby/app-ovn.html*configuration__;Iw!!ACWV5N9M2RV99hQ!dFnukkvR8ggDcHotwyqXNVu8B3dWlc7LBGXbc5fECYqHdok6NIeoKkz5aSlpwu75q1Q$ that suggests it might, but it is short on details, with specifics abstracted via the OpenStack CMS. Also in the OVN Architecture documentation there are hints to support: "For instances connected through representor ports, typically used with hardware offload, the ovn-controller may on CMS direction consult a VIF plug provider for representor port lookup and plug them into the integration bridge (please refer to Docu mentation/topics/vif-plug-providers/vif-plug-providers.rst for more information)." But again short on details. So I believe something like a CX-5/6/7 would have the capability to do this, but here would have to be some sort of OVN hook for the OVS flows created by OVN to be "copied/moved" to the H/W so that encapsulation, NAT, distributed routing ACLs etc is done in the hardware. I can't find any details on this nor what would the control plane for programing the hardware be to do that, ovn-nbctl?, ovsdbapp? some other out of band control plane?. Also from what I gather from the OpenStack docs this seems experimental and limited to VXLAN encapsulation? At present I use a libvirt OVN hook that hooks KVM/QEMU VMs into OVN br-int but these are using software VIFs. I'm trying to ascertain if can have these VM use SR-IOV, and still have them integrated into the OVN logical networks. Any pointers would welcome. Thanks Brendan ___ discuss mailing list disc...@openvswitch.org https://urldefense.com/v3/__https://mail.openvswitch.org/mailman/listinfo/ovs-discuss__;!!ACWV5N9M2RV99hQ!dFnukkvR8ggDcHotwyqXNVu8B3dWlc7LBGXbc5fECYqHdok6NIeoKkz5aSlpNIZip9k$ ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
