On Thu, 8 Oct 2026 16:15:26 +0000 "Randy Tice (rtice)" <[email protected]> wrote:
> I did a brief look at this and still not convinced about the exact use case. > What is the problem this is trying to solve and why can't it be done > by using existing API’s. > > #RT: > Our implementation currently requires an additional 256 bytes of metadata > per > mbuf. That does not fit in the existing dynamic-field storage, so today we > carry a private patch to extend struct rte_mbuf. The goal of this work is to > replace that private struct change with a supported upstream mechanism. > > We did look at using mbuf private data first. It can work when the > application > owns all mbuf pool creation, but it is not sufficient for our case without > another global/base reservation mechanism. Some mbuf pools are created by > drivers or libraries rather than directly by the application; the CNXK > inline > IPsec/OOP meta pool is one example (NIX_INL_META_POOL, created through > cnxk_nix_inl_meta_pool_cb()). To make private data work there, we had to add > an EAL argument that reserved a base private size for all pktmbufs, > including > PMD-created pools. > > Private data also lacks a central layout registry. If multiple modules use > private data, they must coordinate offsets out of band to avoid overlaying > each other. The dynamic-field registry solves that coordination problem, but > the existing copied dynamic-field area is too small and has copy/clone > semantics that are wrong for this metadata. > > That is why this version uses a globally configured per-mbuf metadata area > managed by the dynamic-field registry, with explicit metadata fields that > are > not copied by generic copy/clone/attach paths. This direction came out of > the > prior discussion with you, Morten, and me: avoid a Cisco-private mbuf struct > patch, avoid per-pool private-data layout coordination, and keep > sizeof(struct > rte_mbuf) fixed. You uncovered a design flaw in the CNXK driver and the proposed solution is wider than it needs to be. All mbufs visible to application must come from mempools controlled by the application. The design of CNXK driver is wrong, it shouldn't be using a private hidden pool. It is ok for drivers to have hidden mempools that are used for non-visible things, an example is the packet capture mempool where the mbufs only go into capture stream. More long winded AI description: On Thu, 8 Oct 2026 16:15:26 +0000 "Randy Tice (rtice)" <[email protected]> wrote: > We did look at using mbuf private data first. It can work when the > application owns all mbuf pool creation, but it is not sufficient for > our case without another global/base reservation mechanism. Some mbuf > pools are created by drivers or libraries rather than directly by the > application; the CNXK inline IPsec/OOP meta pool is one example That is the real bug. A driver should not hand the application mbufs from a pool the application did not create. The Rx queue API already takes the pool from the application, and rte_eth_rxconf can carry more than one (rx_mempools). If cnxk needs a meta pool whose mbufs reach the application, it should get it from the application, or at minimum create it with the same priv_size as the Rx queue pool. Internal pools are fine when the mbufs never cross the API, e.g. dumpcap or the bonding LACP pool. With that fixed, the existing private area does what you need. It is per pool, sized by the application, and is not copied by copy, clone or attach. Offset coordination inside it is the application's job, and if a registry is wanted it can be layered on top of priv without changing the mbuf layout. So my answer to option 1 vs 2 is neither. I don't want a global layout knob that puts a load in every inline helper and needs per-driver range checks, to work around one driver. What I would take: - cnxk: validate wqe_skip/later_skip/first_skip against priv_size and headroom. This is a bug today, independent of this series. - cnxk: meta pool comes from, or matches, the application pool. - ethdev: under RTE_ETHDEV_DEBUG_RX, check that received mbufs belong to a pool configured on that queue. Nithin, Harman: do mbufs from NIX_INL_META_POOL get returned to the application by rx_burst, or are they only consumed internally?
