Hi Jesse,
Thanks a lot for your information, seems the commit have been merged to
v4.6+ kernel which may not be easily for us to upgrade.
We will try to upgrade to lastest OVS and do test again.
2016-07-13 22:48 GMT+08:00 Jesse Gross <je...@kernel.org>:
> On Wed, Jul 13, 2016 at 4:05 AM, 张东亚 <fortitude.zh...@gmail.com> wrote:
> > Hi,
> >
> > Recently we upgraded to ovs 2.5 and use the datapath in the ovs
> code(means
> > we do not use kernel datapath), however, when we test vxlan performance
> > between two VMs on different compute node, we observed the bandwidth can
> > only achieve 5Gbps on the 10Gbps intel nic pair, and we are still using
> > kernel v3.14.57.
> >
> > We found lots of following warning (see stack in the PS) in dmesg and
> thinks
> > it causes the bad performance.
> >
> > After trying to read the source code of the kernel and ovs 2.5 datapath
> > code, it seems gso_type 513(0x201) cause the problem, since our VM tap
> > interface does not have
> tx-udp_tnl-segmentation(NETIF_F_GSO_UDP_TUNNEL_BIT)
> > flag enabled by default, net_gso_ok return false which cause
> > __skb_gso_segment being called, and since tcpv4_gso_complete have set the
> > skb->ip_summed to CHECKSUM_UNCESSARY, which cause skb_warn_bad_offload
> being
> > called.
> >
> > My wonder is whether this is a desired behavior, or OVS should clear the
> > NETIF_F_GSO_UDP_TUNNEL_BIT bit when send the inner packet (may be GSOed)
> to
> > tap interface on the host?
>
> This is an upstream kernel problem, fixed by:
>
> commit a09a4c8dd1ec7f830e1fb9e59eb72bddc965d168
> Author: Jesse Gross <je...@kernel.org>
>
> tunnels: Remove encapsulation offloads on decap.
>
> If a packet is either locally encapsulated or processed through GRO
> it is marked with the offloads that it requires. However, when it is
> decapsulated these tunnel offload indications are not removed. This
> means that if we receive an encapsulated TCP packet, aggregate it with
> GRO, decapsulate, and retransmit the resulting frame on a NIC that does
> not support encapsulation, we won't be able to take advantage of
> hardware
> offloads even though it is just a simple TCP packet at this point.
>
> This fixes the problem by stripping off encapsulation offload
> indications
> when packets are decapsulated.
>
> The performance impacts of this bug are significant. In a test where a
> Geneve encapsulated TCP stream is sent to a hypervisor, GRO'ed,
> decapsulated,
> and bridged to a VM performance is improved by 60% (5Gbps->8Gbps) as a
> result of avoiding unnecessary segmentation at the VM tap interface.
>
> Reported-by: Ramu Ramamurthy <srama...@linux.vnet.ibm.com>
> Fixes: 68c33163 ("v4 GRE: Add TCP segmentation offload for GRE")
> Signed-off-by: Jesse Gross <je...@kernel.org>
> Signed-off-by: David S. Miller <da...@davemloft.net>
>
> The problem is not really related to OVS and so the right solution is
> really to upgrade your base kernel. However, since it does affect OVS
> users we have backported code into the OVS tree to avoid it.
> Unfortunately, the fix is too large to bring to OVS 2.5, so you'll
> need to use the current master branch.
>
_______________________________________________
discuss mailing list
discuss@openvswitch.org
http://openvswitch.org/mailman/listinfo/discuss
