On Fri, Dec 22, 2017 at 11:28:45AM +0000, Finn Christensen wrote: > Hi, > > The addition of the offload thread is an great addition, furthermore, RSS > action usage for target queues, works now well with our NICs, I just had to > do some additional drive work, to make it all work. > > The support for RSS removed the problem with rte-flow queue selection, but > RSS in general added an additional challenge as now multiple pmd's may > request the same flow offload (as Yuanhan pointed out). The solution has > worked without issues for me. I have run tests with 1 and 2 queues in a PV > and PVP setup going over virtio. > Yuanhan mentioned tests in a P2P setup with high throughput acceleration. I > have focused on a PVP scenario, which, in addition, will crosses virtio, and > most importantly, has acceleration in Rx direction only (seen from VM). It > still gives fairly good performance improvements.
Yes, indeed. Those nubmer below are promising for the PVP case. And thanks lot of for the testing. --yliu > > Here are my initial test results. > > Percentage improvements: > 1 RX/Tx queue: > 1 megaflow - 10K flowsPV: 73% > 10 megaflows - 100K flowsPVP: 50%PV: 56% > 100 megaflows - 1M flowsPVP: 42%PV: 49% > 1000 megaflows - 10M flowsPVP: 40%PV: 42% > > With RSS: 2 Rx/Tx queue pairs: > 1 megaflow - 10K flowsPV: 55% > 10 megaflows - 100K flowsPVP: 36%PV: 38% > 100 megaflows - 1M flowsPVP: 27%PV: 24% > 1000 megaflows - 10M flowsPVP: 21%PV: 24% > > PVP for 1 megaflow offload is omitted because my VM-app imposed a bottle-neck. > > Regards, > Finn > > -----Original Message----- > From: Yuanhan Liu [mailto:y...@fridaylinux.org] > Sent: 20. december 2017 15:45 > To: d...@openvswitch.org > Cc: Finn Christensen <f...@napatech.com>; Darrell Ball > <db...@vmware.com>; Chandran Sugesh <sugesh.chand...@intel.com>; > Simon Horman <simon.hor...@netronome.com>; Ian Stokes > <ian.sto...@intel.com>; Yuanhan Liu <y...@fridaylinux.org> > Subject: [PATCH v5 0/5] OVS-DPDK flow offload with rte_flow > > Hi, > > Here is a joint work from Mellanox and Napatech, to enable the flow hw > offload with the DPDK generic flow interface (rte_flow). > > The basic idea is to associate the flow with a mark id (a unit32_t > number). > Later, we then get the flow directly from the mark id, which could bypass > some heavy CPU operations, including but not limiting to mini flow > extract, > emc lookup, dpcls lookup, etc. > > The association is done with CMAP in patch 1. The CPU workload bypassing > is done in patch 2. The flow offload is done in patch 3, which mainly does > two things: > > - translate the ovs match to DPDK rte flow patterns > - bind those patterns with a RSS + MARK action. > > Patch 5 makes the offload work happen in another thread, for leaving the > datapath as light as possible. > > A PHY-PHY forwarding with 1000 mega flows (udp,tp_src=1000-1999) and 1 > million streams (tp_src=1000-1999, tp_dst=2000-2999) show more than > 260% performance boost. > > Note that it's disabled by default, which can be enabled by: > > $ ovs-vsctl set Open_vSwitch . other_config:hw-offload=true > > > v5: - fixed an issue that it took too long if we do flow add/remove > repeatedly. > - removed an unused mutex lock > - turned most of the log level to DBG > - rebased on top of the latest code > > v4: - use RSS action instead of QUEUE action with MARK > - make it work with multiple queue (see patch 1) > - rebased on top of latest code > > v3: - The mark and id association is done with array instead of CMAP. > - Added a thread to do hw offload operations > - Removed macros completely > - dropped the patch to set FDIR_CONF, which is a workround some > Intel NICs. > - Added a debug patch to show all flow patterns we have created. > - Misc fixes > > v2: - workaround the queue action issue > - fixed the tcp_flags being skipped issue, which also fixed the > build warnings > - fixed l2 patterns for Intel nic > - Converted some macros to functions > - did not hardcode the max number of flow/action > - rebased on top of the latest code > > Thanks. > > --yliu > > > --- > Finn Christensen (1): > netdev-dpdk: implement flow offload with rte flow > > Yuanhan Liu (4): > dpif-netdev: associate flow with a mark id > dpif-netdev: retrieve flow directly from the flow mark > netdev-dpdk: add debug for rte flow patterns > dpif-netdev: do hw flow offload in a thread > > lib/dp-packet.h | 13 + > lib/dpif-netdev.c | 473 ++++++++++++++++++++++++++++++++++- > lib/flow.c | 155 +++++++++--- > lib/flow.h | 1 + > lib/netdev-dpdk.c | 732 > +++++++++++++++++++++++++++++++++++++++++++++++++++++- > lib/netdev.h | 6 + > 6 files changed, 1341 insertions(+), 39 deletions(-) > > -- > 2.7.4 > > Disclaimer: This email and any files transmitted with it may contain > confidential information intended for the addressee(s) only. The information > is not to be surrendered or copied to unauthorized persons. If you have > received this communication in error, please notify the sender immediately > and delete this e-mail from your system. _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev