The RSS key only extracted the outer IP header. Tunnelled traffic whose outer headers are fixed then carries no entropy for the hash, so every flow lands on a single Rx queue.
Extract both the outer IP (header index 0) and the innermost IP instance (HDR_INDEX_LAST). Plain frames keep being hashed on their only IP header; the inner extract resolves to nothing and adds no entropy. Tunnelled frames are also hashed on their inner IP and spread across the Rx queues. This is the PMD default hash: dpaa2 does not expose the ethdev RSS level selector, so it applies to every RSS request. The hardware cannot hash the inner IP alone, as HDR_INDEX_LAST only resolves when several IP headers are stacked and a plain frame would hash to a constant. Folding the outer IP into the key is therefore unavoidable, and two tunnelled flows that share an inner IP but differ in their outer IP may hash to different queues. This is documented in the dpaa2 guide. Signed-off-by: Maxime Leroy <[email protected]> --- doc/guides/nics/dpaa2.rst | 3 ++ doc/guides/rel_notes/release_26_07.rst | 2 + drivers/net/dpaa2/base/dpaa2_hw_dpni.c | 63 +++++++++++++------------- 3 files changed, 37 insertions(+), 31 deletions(-) diff --git a/doc/guides/nics/dpaa2.rst b/doc/guides/nics/dpaa2.rst index ae8b32af2c..aaaf5f9713 100644 --- a/doc/guides/nics/dpaa2.rst +++ b/doc/guides/nics/dpaa2.rst @@ -588,6 +588,9 @@ Other Limitations - RSS hash key cannot be modified. - RSS RETA cannot be configured. +- RSS hashes on both the outer and the inner IP header. Tunnelled flows + that share the same inner IP but differ in their outer IP may therefore + be steered to different Rx queues. .. _dpaa2_dptmapi: diff --git a/doc/guides/rel_notes/release_26_07.rst b/doc/guides/rel_notes/release_26_07.rst index 6a528e4a0d..c952512023 100644 --- a/doc/guides/rel_notes/release_26_07.rst +++ b/doc/guides/rel_notes/release_26_07.rst @@ -163,6 +163,8 @@ New Features * **Updated NXP dpaa2 driver.** * Added RSS RETA query and update support. + * Added the inner IP header to the RSS hash so tunnelled traffic is + distributed across the Rx queues. * Removed the software VLAN strip offload: ``RTE_ETH_RX_OFFLOAD_VLAN_STRIP`` is no longer advertised, as no hardware strip backs it. diff --git a/drivers/net/dpaa2/base/dpaa2_hw_dpni.c b/drivers/net/dpaa2/base/dpaa2_hw_dpni.c index 26ad105c73..5bb9ad4591 100644 --- a/drivers/net/dpaa2/base/dpaa2_hw_dpni.c +++ b/drivers/net/dpaa2/base/dpaa2_hw_dpni.c @@ -236,6 +236,9 @@ dpaa2_remove_flow_dist(struct rte_eth_dev *eth_dev, return ret; } +/* hdr_index selecting the innermost IP instance in a dpkg extract */ +#define DPAA2_DIST_HDR_INDEX_LAST 0xff + int dpaa2_distset_to_dpkg_profile_cfg( uint64_t req_dist_set, @@ -381,42 +384,40 @@ dpaa2_distset_to_dpkg_profile_cfg( case RTE_ETH_RSS_IPV6: case RTE_ETH_RSS_FRAG_IPV6: case RTE_ETH_RSS_NONFRAG_IPV6_OTHER: - case RTE_ETH_RSS_IPV6_EX: + case RTE_ETH_RSS_IPV6_EX: { + static const uint32_t ip_fields[] = { + NH_FLD_IP_SRC, NH_FLD_IP_DST, + NH_FLD_IP_PROTO }; + static const uint8_t ip_hdr_index[] = { + 0, DPAA2_DIST_HDR_INDEX_LAST }; + unsigned int f, h; if (l3_configured) break; l3_configured = 1; - kg_cfg->extracts[i].extract.from_hdr.prot = - NET_PROT_IP; - kg_cfg->extracts[i].extract.from_hdr.field = - NH_FLD_IP_SRC; - kg_cfg->extracts[i].type = - DPKG_EXTRACT_FROM_HDR; - kg_cfg->extracts[i].extract.from_hdr.type = - DPKG_FULL_FIELD; - i++; - - kg_cfg->extracts[i].extract.from_hdr.prot = - NET_PROT_IP; - kg_cfg->extracts[i].extract.from_hdr.field = - NH_FLD_IP_DST; - kg_cfg->extracts[i].type = - DPKG_EXTRACT_FROM_HDR; - kg_cfg->extracts[i].extract.from_hdr.type = - DPKG_FULL_FIELD; - i++; - - kg_cfg->extracts[i].extract.from_hdr.prot = - NET_PROT_IP; - kg_cfg->extracts[i].extract.from_hdr.field = - NH_FLD_IP_PROTO; - kg_cfg->extracts[i].type = - DPKG_EXTRACT_FROM_HDR; - kg_cfg->extracts[i].extract.from_hdr.type = - DPKG_FULL_FIELD; - i++; - break; + /* Hash on the outer IP (index 0) and the innermost + * IP instance. A plain frame has a single IP header, + * so only the outer extract resolves; a tunnelled + * frame resolves both and is also spread on its inner + * IP. + */ + for (h = 0; h < RTE_DIM(ip_hdr_index); h++) + for (f = 0; f < RTE_DIM(ip_fields); f++) { + kg_cfg->extracts[i].extract.from_hdr.prot = + NET_PROT_IP; + kg_cfg->extracts[i].extract.from_hdr.hdr_index = + ip_hdr_index[h]; + kg_cfg->extracts[i].extract.from_hdr.field = + ip_fields[f]; + kg_cfg->extracts[i].type = + DPKG_EXTRACT_FROM_HDR; + kg_cfg->extracts[i].extract.from_hdr.type = + DPKG_FULL_FIELD; + i++; + } + break; + } case RTE_ETH_RSS_NONFRAG_IPV4_TCP: case RTE_ETH_RSS_NONFRAG_IPV6_TCP: -- 2.43.0
