Public <snipped>
> > > > > > > +/** > > > > + * Pre-reserve backing memory. > > > > + * > > > > + * Ensures that at least @p size bytes of memzone-backed memory > > are > > > > + * available to the allocator on @p socket_id, reserving > > additional > > > > + * memzones from EAL as needed to reach that total. Subsequent > > > > + * allocations served from the pre-reserved memory do not incur > > > > + * memzone-reservation cost. > > > > + * > > > > + * The reservation is cumulative: repeated calls to > > > > + * rte_fastmem_reserve() with the same @p socket_id grow the > > > > + * reservation monotonically. Reserved memory is never returned > > > > + to > > > > + * the system during the allocator's lifetime. > > > > + * > > > > + * A typical use is to call rte_fastmem_reserve() once at > > > > + * application startup, with a size chosen to cover the expected > > > > + * steady-state working set. Allocations and frees during > > > > + * steady-state operation then avoid memzone reservations > > entirely. > > > > + * > > > > + * @param size > > > > + * The minimum amount of backing memory, in bytes, to make > > > > + * available on @p socket_id. The allocator may reserve more > > > > + than > > > > + * the requested amount due to internal rounding (e.g., to > > memzone > > > > + * or block granularity). > > > > + * > > > > + * @param socket_id > > > > + * The NUMA socket on which to reserve memory, or SOCKET_ID_ANY > > > > + * to leave the choice to the allocator. With SOCKET_ID_ANY, the > > > > + * allocator starts on the calling lcore's socket (or the first > > > > + * configured socket if the caller is not bound to one) and > > > > + falls > > > > + * back to other sockets if the preferred socket cannot satisfy > > > > + * the reservation. > > > > + * > > > > + * @return > > > > + * - 0: Success. > > > > + * - -ENOMEM: Insufficient huge-page memory to satisfy the > > request. > > > > + * - -EINVAL: Invalid @p socket_id. > > > > + */ > > > > +__rte_experimental > > > > +int > > > > +rte_fastmem_reserve(size_t size, int socket_id); > > > > > > @Bruce, > > > I vaguely recall that we discussed something about busses and > > > sockets > > a long time > > > ago, but I cannot remember the details. > > > Is socket_id the right type (and parameter name) to identify a > > > memory > > bus? > > > > > > @Vipin, > > > You have been working on topology awareness. Same question to you: > > > Is socket_id the right type (and parameter name) to identify a > > > memory > > bus? > > > > Short answer: socket_id is no longer a precise or sufficient > > abstraction to represent a memory bus. > > Based on the topology work with libhwloc, we’ve observed the following > > across Ampere, Intel, and AMD platforms: > > > > Features like SNC (Sub-NUMA Clustering) on Intel and NPS (NUMA Per > > Socket) on AMD change how socket_id maps to hardware. > > In these modes: > > > > 1) A single physical socket can expose multiple NUMA domains. > > 2)These NUMA domains align more closely with memory controller > > groupings (i.e., memory buses) rather than the full socket. > > > > > > Depending on the architecture: > > a) Memory controllers may be collocated with compute cores or placed > > on separate tiles. > > b) As a result, socket_id can represent different scopes (full socket > > vs. sub-socket domains), making it inconsistent. > > > > > > > > Hence practically: In some configurations, socket_id ≈ memory domain. > > In others, it is coarser than the actual memory bus topology. > > > > To address this ambiguity, in the topology patches (v5/v6), we are > > moving toward clearer separation: > > > > a. Cache domains (L1/L2/L3/L4) for compute locality b. NUMA domains > > (memory + IO) as the unit for allocation locality > > > > This direction better reflects real hardware and avoids overloading > > socket_id with multiple meanings. > > > > Happy to align this with the topology model we’re introducing so the > > abstraction remains consistent going forward. > > Thanks, > > Vipin > > Thank you for the quick and detailed response, Vipin! > > I haven't looked deeply into the v5/v6 topology patches yet (it's on my TODO > list). > > The rte_fastmem library builds on top of the rte_memzone library. > > So, if the rte_memzone library is updated to replace the meaning of its > "socket_id" > parameter with some NUMA domain identifier (we better rename the "socket_id" > to a > new name "numa_domain_id"), then the rte_fastmem library could remain > unaffected, and its "socket_id" parameter would be passed on directly to the > rte_memzone library's "numa_domain_id"? Valid point, I finishing the last changes for v6. Will be pushing it out by weekend. +1 changing terminology > > This is my conclusion: At this point, proper support for allocating memory in > specific > NUMA domains is an rte_memzone library issue, and nothing to worry about for > the > rte_fastmem library - it will be automatically supported in rte_fastmem when > supported by rte_memzone. >
