With userspace TSO enabled, packets that need TCP segmentation were dropped when sent through an ESP tunnel, as for every tunnel type without offload support. For ESP, no NIC can segment the packets after encryption, so segment them in software before.
This requires two fixes to dp_packet_gso(), which until now was only used for packets that start with an Ethernet header, while ESP carries L3 packets whose Ethernet header was popped: - The length of the headers copied into each segment is now measured from the start of the packet rather than from its Ethernet header, which dp_packet_eth() reports as missing for L3 packets. For Ethernet packets this is the same. - Segments now keep the packet type of the original packet instead of being reset to Ethernet. The test checks that a 2000-byte TCP packet is sent as four encrypted 500-byte segments. Decrypting them with an independent implementation showed correct TCP sequence numbers, IP IDs and checksums. Assisted-by: claude-opus-5-5, Claude Code Signed-off-by: Thilak Raj Surendra Babu <[email protected]> --- Documentation/faq/configuration.rst | 3 -- lib/dp-packet-gso.c | 7 ++-- lib/netdev.c | 11 ++++++- tests/tunnel-push-pop.at | 51 +++++++++++++++++++++++++++++ 4 files changed, 65 insertions(+), 7 deletions(-) diff --git a/Documentation/faq/configuration.rst b/Documentation/faq/configuration.rst index f751c6723..0369877b4 100644 --- a/Documentation/faq/configuration.rst +++ b/Documentation/faq/configuration.rst @@ -301,9 +301,6 @@ Q: Does the userspace datapath, for example with DPDK, support IPsec? must leave room for up to 57 bytes of overhead with an IPv4 underlay and 77 with IPv6. - * With userspace TSO enabled, packets that need TCP segmentation are - dropped. - Q: How do I connect two bridges? A: First, why do you want to do this? Two connected bridges are not much diff --git a/lib/dp-packet-gso.c b/lib/dp-packet-gso.c index 8c8817352..ff4ea4fb5 100644 --- a/lib/dp-packet-gso.c +++ b/lib/dp-packet-gso.c @@ -47,7 +47,8 @@ dp_packet_gso_seg_new(const struct dp_packet *p, size_t hdr_len, dp_packet_put(seg, dp_packet_data(p), hdr_len); dp_packet_put(seg, (char *) dp_packet_data(p) + data_off, data_len); - /* The new segment should have the same offsets. */ + /* The new segment should have the same type and offsets. */ + seg->packet_type = p->packet_type; seg->l2_5_ofs = p->l2_5_ofs; seg->l3_ofs = p->l3_ofs; seg->l4_ofs = p->l4_ofs; @@ -207,11 +208,11 @@ dp_packet_gso__(struct dp_packet *p, struct dp_packet_batch *batch, if (dp_packet_tunnel(p)) { hdr_len = (char *) dp_packet_get_inner_tcp_payload(p) - - (char *) dp_packet_eth(p); + - (char *) dp_packet_data(p); data_len = dp_packet_get_inner_tcp_payload_length(p); } else { hdr_len = (char *) dp_packet_get_tcp_payload(p) - - (char *) dp_packet_eth(p); + - (char *) dp_packet_data(p); data_len = dp_packet_get_tcp_payload_length(p); } diff --git a/lib/netdev.c b/lib/netdev.c index ba7505122..215a30ff6 100644 --- a/lib/netdev.c +++ b/lib/netdev.c @@ -920,7 +920,16 @@ netdev_push_header(const struct netdev *netdev, struct dp_packet *packet; if (userspace_tso_enabled()) { - if (OVS_UNLIKELY(!supported_offloads)) { + if (data->tnl_type == OVS_VPORT_TYPE_ESP) { + /* Segmentation must happen before encryption, and no NIC can + * segment the encrypted packets, so segment them here. */ + DP_PACKET_BATCH_FOR_EACH (j, packet, batch) { + if (dp_packet_get_tso_segsz(packet)) { + dp_packet_gso_batch(batch); + break; + } + } + } else if (OVS_UNLIKELY(!supported_offloads)) { DP_PACKET_BATCH_REFILL_FOR_EACH (i, size, packet, batch) { if (OVS_UNLIKELY(dp_packet_get_tso_segsz(packet))) { COVERAGE_INC(netdev_push_header_drops); diff --git a/tests/tunnel-push-pop.at b/tests/tunnel-push-pop.at index 3a9648851..aa887340b 100644 --- a/tests/tunnel-push-pop.at +++ b/tests/tunnel-push-pop.at @@ -1732,3 +1732,54 @@ recirc_id(X),tunnel(tun_id=0x1000,src=1.1.2.92,dst=1.1.2.88,flags(+key)),in_port OVS_VSWITCHD_STOP AT_CLEANUP + +AT_SETUP([tunnel_push_pop - esp with userspace TSO]) +AT_KEYWORDS([ipsec userspace offload]) +AT_SKIP_IF([test "$HAVE_OPENSSL" = no]) + +dnl ESP packets cannot be segmented after encryption, so TCP segmentation +dnl must happen in software before. +OVS_VSWITCHD_START([set Open_vSwitch . other_config:userspace-tso-enable=true dnl + -- add-port br0 p0 -- set Interface p0 type=dummy dnl + ofport_request=1 other-config:hwaddr=aa:55:aa:55:00:00 dnl + options:tx_pcap=p0.pcap]) +AT_CHECK([ovs-vsctl add-br int-br -- set bridge int-br datapath_type=dummy]) +AT_CHECK([ovs-vsctl add-port int-br t1 -- set Interface t1 type=esp dnl + ofport_request=2 options:remote_ip=1.1.2.92 dnl + options:esp_in_spi=4096 options:esp_out_spi=8192 dnl + options:esp_in_key=0x101112131415161718191a1b1c1d1e1fcafebabe dnl + options:esp_out_key=0x202122232425262728292a2b2c2d2e2fdeadbeef dnl + -- add-port int-br p1 -- set Interface p1 type=dummy dnl + ofport_request=3 options:ol_tso_segsz=500]) + +AT_CHECK([ovs-appctl netdev-dummy/ip4addr br0 1.1.2.88/24], [0], [OK +]) +AT_CHECK([ovs-appctl tnl/neigh/set br0 1.1.2.92 aa:55:aa:55:00:01], [0], [OK +]) +AT_CHECK([ovs-ofctl add-flow br0 action=normal]) +AT_CHECK([ovs-ofctl add-flow int-br in_port=p1,action=t1]) + +dnl A TCP packet with 2000 bytes of payload becomes 4 segments of 500 bytes, +dnl each encrypted separately. +AT_CHECK([ovs-appctl netdev-dummy/receive p1 dnl + 'eth(src=f8:bc:12:44:34:b6,dst=aa:55:aa:55:00:00),eth_type(0x0800),ipv4(src=10.0.0.1,dst=10.0.0.2,proto=6,tos=0,ttl=64,frag=no),tcp(src=54392,dst=5201),tcp_flags(ack)' dnl + --len 2054]) +OVS_WAIT_UNTIL([test $(ovs-pcap p0.pcap | wc -l) -ge 4]) + +dnl Each 540-byte segment becomes 16 + 540 + 2 (padding) + 2 + 16 bytes of +dnl ESP. Only check the headers before the random IV. +AT_CHECK([ovs-pcap p0.pcap | cut -c1-84], [0], [dnl +aa55aa550001aa55aa5500000800450002540000400040323[]1c3010102580101025c0000200000000001 +aa55aa550001aa55aa5500000800450002540000400040323[]1c3010102580101025c0000200000000002 +aa55aa550001aa55aa5500000800450002540000400040323[]1c3010102580101025c0000200000000003 +aa55aa550001aa55aa5500000800450002540000400040323[]1c3010102580101025c0000200000000004 +]) +AT_CHECK([ovs-appctl esp/show t1 | sed -n 2,3p], [0], [dnl + outbound: spi 0x00002000, aes128-gcm16, esn off + packets 4, bytes 2160, last seq 4, errors 0 +]) +AT_CHECK([ovs-appctl coverage/read-counter netdev_push_header_drops], [0], [0 +]) + +OVS_VSWITCHD_STOP +AT_CLEANUP -- 2.43.0 _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
