On 2/6/2021 9:00 AM, William Tu wrote:
On Fri, Feb 5, 2021 at 1:08 PM Gregory Rose <gvrose8...@gmail.com> wrote:
On 2/4/2021 7:08 PM, William Tu wrote:
On Thu, Feb 4, 2021 at 3:17 PM Gregory Rose <gvrose8...@gmail.com> wrote:
On 2/3/2021 1:21 PM, William Tu wrote:
Mellanox card has different XSK design. It requires users to create
dedicated queues for XSK. Unlike Intel's NIC which loads XDP program
to all queues, Mellanox only loads XDP program to a subset of its queue.
When OVS uses AF_XDP with mlx5, it doesn't replace the existing RX and TX
queues in the channel with XSK RX and XSK TX queues, but it creates an
additional pair of queues for XSK in that channel. To distinguish
regular and XSK queues, mlx5 uses a different range of qids.
That means, if the card has 24 queues, queues 0..11 correspond to
regular queues, and queues 12..23 are XSK queues.
In this case, we should attach the netdev-afxdp with 'start-qid=12'.
I tested using Mellanox Connect-X 6Dx, by setting 'start-qid=1', and:
$ ethtool -L enp2s0f0np0 combined 1
# queue 0 is for non-XDP traffic, queue 1 is for XSK
$ ethtool -N enp2s0f0np0 flow-type udp4 action 1
note: we need additionally add flow-redirect rule to queue 1
Seems awfully hardware dependent. Is this just for Mellanox or does
it have general usefulness?
It is just Mellanox's design which requires pre-configure the flow-director.
I only have cards from Intel and Mellanox so I don't know about other vendors.
Thanks,
William
I think we need to abstract the HW layer a little bit. This start-qid
option is specific to a single piece of HW, at least at this point.
We should expect that further HW specific requirements for
different NIC vendors will come up in the future. I suggest
adding a hw_options:mellanox:start-qid type hierarchy so that
as new HW requirements come up we can easily scale. It will
also make adding new vendors easier in the future.
Even with NIC vendors you can't always count on each new generation
design to always keep old requirements and methods for feature
enablement.
What do you think?
Thanks for the feedback.
So far I don't know whether other vendors will need this option or not.
I think adding another "hw_options" is a little confusing because this
is already an option on the device.
Looking at AF_XDP driver at DPDK, it also has similar option:
see start_queue
https://doc.dpdk.org/guides/nics/af_xdp.html
OK, makes sense in this context then.
Thanks,
- Greg
_______________________________________________
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
