I was told more tests and reviews are needed from yesterday's meeting.
For testing, as you can see below, Finn already have done some performance
testing covering quite few testcases. I have also done some very basic
performance testing, basically just with rxonly.
Besides that, I have actually spent a lot of effort for doing functional
testing, to make sure it doesn't break anything. The limited testing
including:
- 1 queue
- 2 queues
- 1 flow
- 1000 flows
- adding/deleting flows without traffic
- adding/deleting flows with traffic
- keep adding and then deleting flows with traffic
For review, I think Finn from Napatech should also have done it. He
has provided some valuable comments after all :)
Thanks.
--yliu
On Fri, Dec 22, 2017 at 11:28:45AM +0000, Finn Christensen wrote:
> Hi,
>
> The addition of the offload thread is an great addition, furthermore, RSS
> action usage for target queues, works now well with our NICs, I just had to
> do some additional drive work, to make it all work.
>
> The support for RSS removed the problem with rte-flow queue selection, but
> RSS in general added an additional challenge as now multiple pmd's may
> request the same flow offload (as Yuanhan pointed out). The solution has
> worked without issues for me. I have run tests with 1 and 2 queues in a PV
> and PVP setup going over virtio.
> Yuanhan mentioned tests in a P2P setup with high throughput acceleration. I
> have focused on a PVP scenario, which, in addition, will crosses virtio, and
> most importantly, has acceleration in Rx direction only (seen from VM). It
> still gives fairly good performance improvements.
>
> Here are my initial test results.
>
> Percentage improvements:
> 1 RX/Tx queue:
> 1 megaflow - 10K flowsPV: 73%
> 10 megaflows - 100K flowsPVP: 50%PV: 56%
> 100 megaflows - 1M flowsPVP: 42%PV: 49%
> 1000 megaflows - 10M flowsPVP: 40%PV: 42%
>
> With RSS: 2 Rx/Tx queue pairs:
> 1 megaflow - 10K flowsPV: 55%
> 10 megaflows - 100K flowsPVP: 36%PV: 38%
> 100 megaflows - 1M flowsPVP: 27%PV: 24%
> 1000 megaflows - 10M flowsPVP: 21%PV: 24%
>
> PVP for 1 megaflow offload is omitted because my VM-app imposed a bottle-neck.
>
> Regards,
> Finn
>
> -----Original Message-----
> From: Yuanhan Liu [mailto:[email protected]]
> Sent: 20. december 2017 15:45
> To: [email protected]
> Cc: Finn Christensen <[email protected]>; Darrell Ball
> <[email protected]>; Chandran Sugesh <[email protected]>;
> Simon Horman <[email protected]>; Ian Stokes
> <[email protected]>; Yuanhan Liu <[email protected]>
> Subject: [PATCH v5 0/5] OVS-DPDK flow offload with rte_flow
>
> Hi,
>
> Here is a joint work from Mellanox and Napatech, to enable the flow hw
> offload with the DPDK generic flow interface (rte_flow).
>
> The basic idea is to associate the flow with a mark id (a unit32_t
> number).
> Later, we then get the flow directly from the mark id, which could bypass
> some heavy CPU operations, including but not limiting to mini flow
> extract,
> emc lookup, dpcls lookup, etc.
>
> The association is done with CMAP in patch 1. The CPU workload bypassing
> is done in patch 2. The flow offload is done in patch 3, which mainly does
> two things:
>
> - translate the ovs match to DPDK rte flow patterns
> - bind those patterns with a RSS + MARK action.
>
> Patch 5 makes the offload work happen in another thread, for leaving the
> datapath as light as possible.
>
> A PHY-PHY forwarding with 1000 mega flows (udp,tp_src=1000-1999) and 1
> million streams (tp_src=1000-1999, tp_dst=2000-2999) show more than
> 260% performance boost.
>
> Note that it's disabled by default, which can be enabled by:
>
> $ ovs-vsctl set Open_vSwitch . other_config:hw-offload=true
>
>
> v5: - fixed an issue that it took too long if we do flow add/remove
> repeatedly.
> - removed an unused mutex lock
> - turned most of the log level to DBG
> - rebased on top of the latest code
>
> v4: - use RSS action instead of QUEUE action with MARK
> - make it work with multiple queue (see patch 1)
> - rebased on top of latest code
>
> v3: - The mark and id association is done with array instead of CMAP.
> - Added a thread to do hw offload operations
> - Removed macros completely
> - dropped the patch to set FDIR_CONF, which is a workround some
> Intel NICs.
> - Added a debug patch to show all flow patterns we have created.
> - Misc fixes
>
> v2: - workaround the queue action issue
> - fixed the tcp_flags being skipped issue, which also fixed the
> build warnings
> - fixed l2 patterns for Intel nic
> - Converted some macros to functions
> - did not hardcode the max number of flow/action
> - rebased on top of the latest code
>
> Thanks.
>
> --yliu
>
>
> ---
> Finn Christensen (1):
> netdev-dpdk: implement flow offload with rte flow
>
> Yuanhan Liu (4):
> dpif-netdev: associate flow with a mark id
> dpif-netdev: retrieve flow directly from the flow mark
> netdev-dpdk: add debug for rte flow patterns
> dpif-netdev: do hw flow offload in a thread
>
> lib/dp-packet.h | 13 +
> lib/dpif-netdev.c | 473 ++++++++++++++++++++++++++++++++++-
> lib/flow.c | 155 +++++++++---
> lib/flow.h | 1 +
> lib/netdev-dpdk.c | 732
> +++++++++++++++++++++++++++++++++++++++++++++++++++++-
> lib/netdev.h | 6 +
> 6 files changed, 1341 insertions(+), 39 deletions(-)
>
> --
> 2.7.4
>
> Disclaimer: This email and any files transmitted with it may contain
> confidential information intended for the addressee(s) only. The information
> is not to be surrendered or copied to unauthorized persons. If you have
> received this communication in error, please notify the sender immediately
> and delete this e-mail from your system.
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
