RTE_ETH_RX_OFFLOAD_VLAN_STRIP is advertised, but no hardware VLAN strip backs it: when enabled, the Rx burst calls rte_vlan_strip() on every frame, a software op masquerading as a hardware offload.
It saves a forwarding application nothing: the datapath reads the L2 header anyway to classify or strip. The offload does not remove that read, it relocates it into the driver Rx burst, where it is far more expensive. The cost is a matter of timing. rte_vlan_strip() reaches the L2 header through rte_pktmbuf_mtod(), which dereferences mbuf->buf_addr. On a freshly recycled buffer that mbuf cacheline is cold. eth_fd_to_mbuf() has just written other fields of it (data_off, ol_flags), but buf_addr is a persistent field it does not rewrite. A write does not stall: it posts to the store buffer while the line fills in the background, and the rewritten fields are forwarded straight from there. buf_addr has nothing to forward, so it must be read from the line, whose fill is still in flight, and the read stalls. The ethertype read that follows, on the cold payload line, stalls again. Read later by the application, when the fill has completed, the same read hits. The offload just performs it at the worst possible moment. Measured on a single-core port-to-port forwarding test over two 10G ports (one core at 2 GHz, 64-byte untagged frames): - throughput 4.22 -> 5.00 Mpps (+18 percent) - IPC 0.93 -> 1.25: the cost was memory stall, not compute - L3/DRAM-bound L2 refills 319M -> 200M over 10s (-37 percent) perf confirms it: with the offload, the buf_addr load (the cold mbuf field) and the payload load account for about 84 percent of the Rx burst's L2 refills; removing it, those vanish and only the inherent DQRR dequeue misses remain. Stop advertising VLAN_STRIP and remove the rte_vlan_strip() calls from every Rx path. This is a behavioural change: the tag is left in the frame, so an application must strip it itself, on the L2 header it already reads. Signed-off-by: Maxime Leroy <[email protected]> Acked-by: Morten Brørup <[email protected]> Acked-by: Stephen Hemminger <[email protected]> Acked-by: Hemant Agrawal <[email protected]> --- doc/guides/rel_notes/release_26_07.rst | 3 +++ drivers/net/dpaa2/dpaa2_ethdev.c | 1 - drivers/net/dpaa2/dpaa2_rxtx.c | 9 --------- 3 files changed, 3 insertions(+), 10 deletions(-) diff --git a/doc/guides/rel_notes/release_26_07.rst b/doc/guides/rel_notes/release_26_07.rst index 9c724e7cc5..c40d3d73a2 100644 --- a/doc/guides/rel_notes/release_26_07.rst +++ b/doc/guides/rel_notes/release_26_07.rst @@ -152,6 +152,9 @@ New Features * Added inner RSS level support for tunnelled traffic. * Added RSS RETA query and update support. + * Removed the software VLAN strip offload: ``RTE_ETH_RX_OFFLOAD_VLAN_STRIP`` + is no longer advertised, as no hardware strip backs it. An application + that needs the tag removed must now strip it itself. * **Updated PCAP ethernet driver.** diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c index 56682717cf..a68404ee5e 100644 --- a/drivers/net/dpaa2/dpaa2_ethdev.c +++ b/drivers/net/dpaa2/dpaa2_ethdev.c @@ -45,7 +45,6 @@ static uint64_t dev_rx_offloads_sup = RTE_ETH_RX_OFFLOAD_SCTP_CKSUM | RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM | RTE_ETH_RX_OFFLOAD_OUTER_UDP_CKSUM | - RTE_ETH_RX_OFFLOAD_VLAN_STRIP | RTE_ETH_RX_OFFLOAD_VLAN_FILTER | RTE_ETH_RX_OFFLOAD_TIMESTAMP; diff --git a/drivers/net/dpaa2/dpaa2_rxtx.c b/drivers/net/dpaa2/dpaa2_rxtx.c index b316e23e87..884cea43c9 100644 --- a/drivers/net/dpaa2/dpaa2_rxtx.c +++ b/drivers/net/dpaa2/dpaa2_rxtx.c @@ -890,10 +890,6 @@ dpaa2_dev_prefetch_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) } #endif - if (eth_data->dev_conf.rxmode.offloads & - RTE_ETH_RX_OFFLOAD_VLAN_STRIP) - rte_vlan_strip(bufs[num_rx]); - dq_storage++; num_rx++; } while (pending); @@ -1100,11 +1096,6 @@ dpaa2_dev_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) } #endif - if (eth_data->dev_conf.rxmode.offloads & - RTE_ETH_RX_OFFLOAD_VLAN_STRIP) { - rte_vlan_strip(bufs[num_rx]); - } - dq_storage++; num_rx++; num_pulled++; -- 2.43.0
