Hi Frode, Thanks for the fast reply!
Replies in-line as well. On Wed, May 24, 2023 at 11:41 PM Frode Nordahl <frode.nord...@canonical.com> wrote: > Hello, Robert, > > See my response in-line below. > > On Thu, May 25, 2023 at 8:20 AM Robert Navarro via discuss > <ovs-discuss@openvswitch.org> wrote: > > > > Hello, > > > > I've followed the directions here: > > > https://docs.nvidia.com/networking/pages/viewpage.action?pageId=119763689 > > > > But I can't seem to get HW offload to work on my system. > > > > I'm using the latest OFED drivers with a ConnectX-5 SFP28 card running > on kernel 5.15.107-2-pve > > Note that if you plan to use this feature with OVN you may find that > the ConnectX-5 does not provide all the features required. Among other > things it does not support the `dec_ttl` action which is a > prerequisite for processing L3 routing, I'm also not sure whether it > fully supports connection tracking offload. I'd go with CX-6 DX or > above if this is one of your use cases. Good to know, I'll keep that in mind as I progress with testing. > > > I have two hosts directly connected to each other, running a simple ping > between hosts shows the following flows: > > > > root@pvet1:~# ovs-appctl dpctl/dump-flows -m type=tc > > ufid:be1670c1-b36b-4f0f-8aba-e9415b9d0fb1, > skb_priority(0/0),skb_mark(0/0),ct_state(0/0),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),recirc_id(0),dp_hash(0/0),in_port(ens15f0np0),packet_type(ns=0/0,id=0/0),eth(src=6e:56:fd:40:6e:22,dst=f6:36:11:c6:04:f0),eth_type(0x0800),ipv4(src= > 0.0.0.0/0.0.0.0,dst=0.0.0.0/0.0.0.0,proto=0/0,tos=0/0,ttl=0/0,frag=no), > packets:639, bytes:53676, used:0.710s, dp:tc, actions:tap103i1 > > > > root@pvet2:~# ovs-appctl dpctl/dump-flows -m type=tc > > ufid:f4d0ebd2-7ba9-4e21-9bf8-090f90bac072, > skb_priority(0/0),skb_mark(0/0),ct_state(0/0),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),recirc_id(0),dp_hash(0/0),in_port(ens15f0np0),packet_type(ns=0/0,id=0/0),eth(src=f6:36:11:c6:04:f0,dst=6e:56:fd:40:6e:22),eth_type(0x0800),ipv4(src= > 0.0.0.0/0.0.0.0,dst=0.0.0.0/0.0.0.0,proto=0/0,tos=0/0,ttl=0/0,frag=no), > packets:656, bytes:55104, used:0.390s, dp:tc, actions:tap100i1 > > The two flows listed above have an action towards what looks like a > virtual Ethernet tap interface. That is not a supported configuration > for hardware offload. I'm using Proxmox as the hypervisor. It seems like Proxmox is attaching the VM (using a virtio nic) as a tap (tap100i1) to the OVS: root@pvet2:~# ovs-dpctl show system@ovs-system: lookups: hit:143640674 missed:1222 lost:45 flows: 2 masks: hit:143642152 total:2 hit/pkt:1.00 port 0: ovs-system (internal) port 1: vmbr1 (internal) port 2: ens15f0np0 port 3: vlan44 (internal) port 4: vlan66 (internal) port 5: ens15f0npf0vf0 port 6: ens15f0npf0vf1 port 7: tap100i1 Given that proxmox uses KVM for virtualization, what's the correct way to link a KVM VM to OVS? > The instance needs to be connected to an > When you say instance here, do you mean the KVM virtual machine or the instance of OVS? > interface wired directly to the embedded switch in the card by > attaching a VF or SF to the instance. > OVS is attached to the physical nic using the PF and 2 VFs as shown in the ovs-dpctl output above > > -- > Frode Nordahl > > > Neither of which shows offloaded > > > > The commands I'm using to setup the interfaces are: > > > > echo 2 | tee /sys/class/net/ens15f0np0/device/sriov_numvfs > > > > lspci -nn | grep Mellanox > > > > echo 0000:03:00.2 | tee /sys/bus/pci/drivers/mlx5_core/unbind > > echo 0000:03:00.3 | tee /sys/bus/pci/drivers/mlx5_core/unbind > > > > devlink dev eswitch set pci/0000:03:00.0 mode switchdev > > > > echo 0000:03:00.2 | tee /sys/bus/pci/drivers/mlx5_core/bind > > echo 0000:03:00.3 | tee /sys/bus/pci/drivers/mlx5_core/bind > > > > ovs-vsctl set Open_vSwitch . other_config:hw-offload=true > > > > systemctl restart openvswitch-switch.service > > > > ovs-vsctl add-port vmbr1 ens15f0np0 > > ovs-vsctl add-port vmbr1 ens15f0npf0vf0 > > ovs-vsctl add-port vmbr1 ens15f0npf0vf1 > > > > ethtool -K ens15f0np0 hw-tc-offload on > > ethtool -K ens15f0npf0vf0 hw-tc-offload on > > ethtool -K ens15f0npf0vf1 hw-tc-offload on > > > > ip link set dev ens15f0np0 up > > ip link set dev ens15f0npf0vf0 up > > ip link set dev ens15f0npf0vf1 up > > > > ovs-dpctl show > > > > Any ideas on what else to check? > > > > -- > > Robert Navarro > > _______________________________________________ > > discuss mailing list > > disc...@openvswitch.org > > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss > -- Robert Navarro
_______________________________________________ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
