RTE_ETH_RX_OFFLOAD_VLAN_STRIP is advertised, but no hardware VLAN strip backs it: when enabled, the Rx burst calls rte_vlan_strip() on every frame, a software op masquerading as a hardware offload.
It saves a forwarding application nothing: the datapath reads the L2 header anyway to classify or strip. The offload does not remove that read, it relocates it into the driver Rx burst, where it is far more expensive. The cost is a matter of timing. rte_vlan_strip() reaches the L2 header through rte_pktmbuf_mtod(), which dereferences mbuf->buf_addr. On a freshly recycled buffer that mbuf cacheline is cold. eth_fd_to_mbuf() has just written other fields of it (data_off, ol_flags), but buf_addr is a persistent field it does not rewrite. A write does not stall: it posts to the store buffer while the line fills in the background, and the rewritten fields are forwarded straight from there. buf_addr has nothing to forward, so it must be read from the line, whose fill is still in flight, and the read stalls. The ethertype read that follows, on the cold payload line, stalls again. Read later by the application, when the fill has completed, the same read hits. The offload just performs it at the worst possible moment. Measured on a single-core port-to-port forwarding test over two 10G ports (one core at 2 GHz, 64-byte untagged frames): - throughput 4.22 -> 5.00 Mpps (+18 percent) - IPC 0.93 -> 1.25: the cost was memory stall, not compute - L3/DRAM-bound L2 refills 319M -> 200M over 10s (-37 percent) perf confirms it: with the offload, the buf_addr load (the cold mbuf field) and the payload load account for about 84 percent of the Rx burst's L2 refills; removing it, those vanish and only the inherent DQRR dequeue misses remain. Stop advertising VLAN_STRIP and remove the rte_vlan_strip() calls from every Rx path. This is a behavioural change: the tag is left in the frame, so an application must strip it itself, on the L2 header it already reads. Signed-off-by: Maxime Leroy <[email protected]> --- doc/guides/rel_notes/release_26_07.rst | 3 +++ drivers/net/dpaa2/dpaa2_ethdev.c | 1 - drivers/net/dpaa2/dpaa2_rxtx.c | 23 +++-------------------- 3 files changed, 6 insertions(+), 21 deletions(-) diff --git a/doc/guides/rel_notes/release_26_07.rst b/doc/guides/rel_notes/release_26_07.rst index 87c7c57bcc..9d01099dad 100644 --- a/doc/guides/rel_notes/release_26_07.rst +++ b/doc/guides/rel_notes/release_26_07.rst @@ -130,6 +130,9 @@ New Features * Added RSS RETA query and update support. * Added Rx queue interrupt support. + * Removed the software VLAN strip offload: ``RTE_ETH_RX_OFFLOAD_VLAN_STRIP`` + is no longer advertised, as no hardware strip backs it. An application + that needs the tag removed must now strip it itself. * **Updated PCAP ethernet driver.** diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c index fb117e761f..b3ea826db9 100644 --- a/drivers/net/dpaa2/dpaa2_ethdev.c +++ b/drivers/net/dpaa2/dpaa2_ethdev.c @@ -48,7 +48,6 @@ static uint64_t dev_rx_offloads_sup = RTE_ETH_RX_OFFLOAD_SCTP_CKSUM | RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM | RTE_ETH_RX_OFFLOAD_OUTER_UDP_CKSUM | - RTE_ETH_RX_OFFLOAD_VLAN_STRIP | RTE_ETH_RX_OFFLOAD_VLAN_FILTER | RTE_ETH_RX_OFFLOAD_TIMESTAMP; diff --git a/drivers/net/dpaa2/dpaa2_rxtx.c b/drivers/net/dpaa2/dpaa2_rxtx.c index 189accc1de..d16e4f8f35 100644 --- a/drivers/net/dpaa2/dpaa2_rxtx.c +++ b/drivers/net/dpaa2/dpaa2_rxtx.c @@ -890,10 +890,6 @@ dpaa2_dev_prefetch_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) } #endif - if (eth_data->dev_conf.rxmode.offloads & - RTE_ETH_RX_OFFLOAD_VLAN_STRIP) - rte_vlan_strip(bufs[num_rx]); - dq_storage++; num_rx++; } while (pending); @@ -922,22 +918,14 @@ dpaa2_dev_prefetch_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) return num_rx; } -/* Convert a DQRR'd FD (single or scatter-gather) to an mbuf and apply software - * VLAN strip, like the poll path. - */ +/* Convert a DQRR'd FD (single or scatter-gather) to an mbuf. */ static inline struct rte_mbuf * dpaa2_dqrr_fd_to_mbuf(const struct qbman_fd *fd, struct rte_eth_dev_data *eth_data) { - struct rte_mbuf *m; - if (unlikely(DPAA2_FD_GET_FORMAT(fd) == qbman_fd_sg)) - m = eth_sg_fd_to_mbuf(fd, eth_data->port_id); - else - m = eth_fd_to_mbuf(fd, eth_data->port_id); - if (eth_data->dev_conf.rxmode.offloads & RTE_ETH_RX_OFFLOAD_VLAN_STRIP) - rte_vlan_strip(m); - return m; + return eth_sg_fd_to_mbuf(fd, eth_data->port_id); + return eth_fd_to_mbuf(fd, eth_data->port_id); } /* prefetch a DQRR'd FD's HW annotation (parse area) ahead of conversion */ @@ -1222,11 +1210,6 @@ dpaa2_dev_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) } #endif - if (eth_data->dev_conf.rxmode.offloads & - RTE_ETH_RX_OFFLOAD_VLAN_STRIP) { - rte_vlan_strip(bufs[num_rx]); - } - dq_storage++; num_rx++; num_pulled++; -- 2.43.0
