On Wed, Jan 22, 2020 at 12:26 AM Flavio Leitner <f...@sysclose.org> wrote:
>
> On Tue, Jan 21, 2020 at 10:39:14AM -0800, William Tu wrote:
> > On Thu, Jan 16, 2020 at 9:01 AM Flavio Leitner <f...@sysclose.org> wrote:
> > >
> > > Abbreviated as TSO, TCP Segmentation Offload is a feature which enables
> > > the network stack to delegate the TCP segmentation to the NIC reducing
> > > the per packet CPU overhead.
> > >
> > > A guest using vhost-user interface with TSO enabled can send TCP packets
> > > much bigger than the MTU, which saves CPU cycles normally used to break
> > > the packets down to MTU size and to calculate checksums.
> > >
> > > It also saves CPU cycles used to parse multiple packets/headers during
> > > the packet processing inside virtual switch.
> > >
> > > If the destination of the packet is another guest in the same host, then
> > > the same big packet can be sent through a vhost-user interface skipping
> > > the segmentation completely. However, if the destination is not local,
> > > the NIC hardware is instructed to do the TCP segmentation and checksum
> > > calculation.
> > >
> > > The first 2 patches are not really part of TSO support, but they are
> > > required to make sure everything works.
> > >
> > > There are good improvements sending to or receiving from veth pairs or
> > > tap devices as well. See the iperf3 results below:
> > >
> > > [*] veth with ethtool tx off.
> > >
> >
> > Hi Flavio,
> >
> > I want to test performance of namespace to namespace using veth, hoping to
> > see TSO packets. Using below setup:
> > iperf -c (ns0) -> veth peer -> OVS -> veth peer -> iperf -s (ns1)
> >
> > With current master I'm not able to see large packet size being sent.
> > I compile ovs with --with-dpdk and,
> > $ ovs-vsctl set Open_vSwitch . other_config:userspace-tso-enable=true
> >
> > At ns0 and ns1, enable tso by ethtool sg and tso on
> > The veth driver shows
> > # ip netns exec at_ns0 ethtool -k p0
> > Features for p0:
> > Cannot get device udp-fragmentation-offload settings: Operation not
> > supported
> > rx-checksumming: on
> > tx-checksumming: off
> ^^^^^^^^^^^^^^^^^^^^^
>
> That disables TSO. We had to do '[*] veth with ethtool tx off'
> before TSO is supported exactly to avoid the large packets to
> be able to run iperf3 tests, etc...
>
> You must leave tx on (default) to enable TSO in veth pairs.
Hi Flavio,
Thanks! With this setup:
iperf3 -c (ns0) -> veth peer -> OVS -> veth peer -> iperf3 -s (ns1)
I got it working now. I can see TCP packet with 64k length.
without TSO: 1.3Gbps
with TSO: 6Gbps
Do you know that for af_packet (netdev-linux.c), if we switch to use
packet mmap[1]
instead of recvmmsg, will it improve performance?
[1] https://www.mjmwired.net/kernel/Documentation/networking/packet_mmap.txt
Regards,
William
_______________________________________________
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
