Nothing significantly changed since the previous versions. This patch set effectively breaks following cases if multi-segment mbufs enabled:
1. Host <-> VM communication broken. With this patch applied VMs has offloading enabled by default. This makes all the packets that goes out from the VM to have incorrect checksums (beside possible multi-segmenting). As a result any packet that reaches host interfaces dropped by the host kernel as corrupted. Could be easily reproduced by trying to use ssh from host to VM, which is the common case for usual OpenStack installations. If checksums will be fixed, segmented packets will be dropped anyway in netdev-linux/bsd/dummy. 2. Broken VM to VM communication in case of not negotiated offloading in one of VMs. If one of VMs does not negotiate offloading, all the packets will be dropped for the same reason as in Host to VM case. This is a really bad case because virtio driver could change the feature set in runtime and this should be handled in OVS. Note that linux kernel virtio driver always has rx checksum offloading enabled, i.e. it doesn't check/recalculates the checksums in software if needed. So, you need the dpdk driver or a FreeBSD machine to reproduce checksumming issue. 3. Broken support of all the virtual and HW NICs that doen't support TSO. It's actually big number of NICs. More than a half of PMDs in DPDK does not support TSO. All the packets just dropped by netdev-dpdk before send. This is most probably not a full list of issues. To fix most of them fallback to full software TSO implementation required. Until this is done I prefer not to merge the feature because it breaks the basic functionality. I understand that you're going to make it 'experimental', but, IMHO, this doesn't worth even that. Patches are fairly small and there is nothing actually we need to test before the full support. The main part is not implemented yet. NACK for the series. Sorry. ---- One more thing I wanted to mention is that I was surprised to not see the performance comparison of TSO with usual Jumbo frames in your slides on a recent OVS Conference [1]. You had a comparison slide with 1500 MTU, TSO and the kernel vhost, but not the jumbo frames. In my testing I have more than 22 Gbps for VM<->VM jumbo frames performance with current OVS and iperf. And this is almost equal to vhost kernel with TSO performance on my system (I have probably a bit less powerful CPU, so the results are a bit lower than in your slides). Also, in real cases there will be significant performance drop because of fallback to software TSO in many cases. Sorry, but I don't really see the benefits of using the Multi-segment mbufs and TSO. All of this work positioned as a feature to cover the gap between the kernel and userspace datapaths, but there is no any gap from the performance point of view and overcomplicating the OVS by multisegmenting and TSO is really not necessary. [1] http://www.openvswitch.org/support/ovscon2018/5/0935-lam.pptx On 10.01.2019 19:58, Tiago Lam wrote: > Enabling TSO offload allows a host stack to delegate the segmentation of > oversized TCP packets to the underlying physical NIC, if supported. In the > case > of a VM this means that the segmentation of the packets is not performed by > the > guest kernel, but by the host NIC itself. In turn, since the TSO calculations > and checksums are being performed in hardware, this alleviates the CPU load on > the host system. In inter VM communication this might account to significant > savings, and higher throughput, even more so if the VMs are running on the > same > host. > > Thus, although inter VM communication is already possible as is, there's a > sacrifice in terms of CPU, which may affect the overall throughput. > > This series adds support for TSO in OvS-DPDK, by making use of the TSO > offloading feature already supported by DPDK vhost backend, having the > following scenarios in mind: > - Inter VM communication on the same host; > - Inter VM communication on different hosts; > - The same two use cases above, but on a VLAN network. > > The work is based on [1]; It has been rebased to run on top of the > multi-segment mbufs work (v13) [2] and re-worked to use DPDK v18.11. > > [1] https://patchwork.ozlabs.org/patch/749564/ > [2] https://mail.openvswitch.org/pipermail/ovs-dev/2019-January/354950.html > > Considerations: > - As mentioned above, this series depends on the multi-segment mbuf series > (v13) and can't be applied on master as is; > - The `rte_eth_tx_prepare()` API in DPDK is marked experimental, and although > I'm not getting any errors / warnings while compiling, do shout if get into > trouble while testing; > - I'm due to send v3 in the next few days, but sending v2 now to enable early > testing; > > Tiago Lam (3): > netdev-dpdk: Validate packets burst before Tx. > netdev-dpdk: Consider packets marked for TSO. > netdev-dpdk: Enable TSO when using multi-seg mbufs > > Documentation/automake.mk | 1 + > Documentation/topics/dpdk/index.rst | 1 + > Documentation/topics/dpdk/tso.rst | 111 ++++++++++++++++++++ > NEWS | 1 + > lib/dp-packet.h | 14 +++ > lib/netdev-bsd.c | 11 +- > lib/netdev-dpdk.c | 203 > ++++++++++++++++++++++++++++++------ > lib/netdev-dummy.c | 11 +- > lib/netdev-linux.c | 15 +++ > 9 files changed, 332 insertions(+), 36 deletions(-) > create mode 100644 Documentation/topics/dpdk/tso.rst > _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev