Currently, the VA space reservation is governed by a combination of a few values:
- Total max VA space (512G for most platforms, 1T for some, 2G for 32-bit) - Max memory per memory type - Max pages per memory type The "memory" type is defined as unique combination of NUMA node and page size. The reason why there are two limits is because for large pages, having large segment limit causes runaway multi-terabyte reservations, while for smaller pages, having large memory limit causes hundreds of thousands of huge page slots. The total maximum memory size was originally intended as a safeguard against discontiguous NUMA nodes, but this has since been fixed by EAL API explicitly supporting discontiguous NUMA nodes, so this is no longer a problem. In addition to that, each memory type was split into multiple segment lists, with the idea that it should be easier for a secondary process to reserve multiple smaller chunks at discontiguous addresses than it is to reserve a large single chunk of memory. It is unknown whether this actually makes a difference, but what *is* known is that it's a source of additional complexity with memory reservation, as well as a source of gratuitous memory reservation limits placed on DPDK. This patchset attempts to simplify and improve this situation in a few key areas: - Get rid of global memory limits Total memory usage can, and should, scale with NUMA sockets, and so now it does. - Get rid of multiple segment lists per memory type This removes two config options, and makes the address space reservations a lot simpler. - Allocate all memory segment lists as one big blob of memory This further simplifies address space reservations. - Use memory size limits instead of segments limits Despite smaller page sizes still needing limits on number of segments, they are directly translated into memory size limits at init time, so that all limits the VA space reservation ever sees are expressed in bytes, not segments. This reduces complexity in how we manage the VA space reservations and work with our limits. - Do not use config constants directly We switch to only invoking these constants once - at startup, when we are discovering hugepage sizes available to the system. This allows us to be more flexible in how we manage these limits. - Add EAL command-line option to set per-page size limits The final piece of the puzzle - the "more flexible in how we manage these limits" part. This new parameter affords us more flexible VA space management, including disabling specific page sizes entirely (by specifying 0 as the limit). This allows increasing/decreasing VA space reservation limits without recompiling DPDK. Anatoly Burakov (5): eal/memory: always use one segment per memory type eal/memory: allocate all VA space in one go eal/memory: get rid of global VA space limits eal/memory: store default segment limits in config eal/memory: add page size VA limits EAL parameter app/test/test.c | 1 + app/test/test_eal_flags.c | 113 ++++++++++++ config/arm/meson.build | 1 - config/meson.build | 5 - config/rte_config.h | 2 - doc/guides/linux_gsg/linux_eal_parameters.rst | 13 ++ .../prog_guide/env_abstraction_layer.rst | 33 +++- lib/eal/common/eal_common_dynmem.c | 160 ++++++++--------- lib/eal/common/eal_common_memory.c | 29 ++- lib/eal/common/eal_common_options.c | 120 +++++++++++++ lib/eal/common/eal_internal_cfg.h | 8 + lib/eal/common/eal_memcfg.h | 6 + lib/eal/common/eal_option_list.h | 1 + lib/eal/common/eal_options.h | 1 + lib/eal/common/eal_private.h | 15 +- lib/eal/freebsd/eal.c | 6 + lib/eal/freebsd/eal_memory.c | 98 +++-------- lib/eal/linux/eal.c | 6 + lib/eal/linux/eal_memalloc.c | 2 +- lib/eal/linux/eal_memory.c | 165 +++++++++++------- lib/eal/windows/eal.c | 6 + 21 files changed, 542 insertions(+), 249 deletions(-) -- 2.47.3
