On Wed, 8 Oct 2025 21:42:23 +0100 Bruce Richardson <[email protected]> wrote:
> The ultimate of this patchset is to make it easier to run on systems > with large numbers of cores, by simplifying the process of using core > numbers >RTE_MAX_LCORE. The new EAL arg ``-remap-lcore-ids``, also > shortened to ``-R``, is added to DPDK to support this. > > However, in order to add this new flag easily, the first dozen or more > patches rework the argument handling in EAL to simplify things, using > the argparse library for argument handling. > > When processing cmdline arguments in DPDK, we always do so with very > little context. So, for example, when processing the "-l" flag, we have > no idea whether there will be later a --proc-type=secondary flag. We > have all sorts of post-arg-processing checks in place to try and catch > these scenarios. > > To improve this situation, this patchset tries to simplify the handling > of argument processing, by explicitly doing an initial pass to collate > all arguments into a structure. Thereafter, the actual arg parsing is > done in a fixed order, meaning that e.g. when processing the > --main-lcore flag, we have already processed the service core flags. We > also can far quicker and easier check for conflicting options, since > they can all be checked for NULL/non-NULL in the arg structure > immediately after the struct has been populated. > > An additional benefit of this work is that the argument parsing for EAL > is much more centralised into common options and the options list file. > This single list with ifdefs makes it clear to the viewer what options > are common across OS's, vs what are unix-only or linux-only. > > Once the cleanup and rework is done, adding the new options for > remapping cores becomes a lot simpler, since we can very easily check > for scenarios like multi-process and handle those appropriately. > > > V10: > * add patch 2 to fix func_reentrancy test bug reported mid-way through > the series. > * added patches 7 & 8 to resolve arg parsing compat issues: > - support reordering args to have non-flag parameters at the end so > we can e.g. mix autotest names in with EAL args > - support having optional params with single-letter args without > having to use "=" syntax, i.e. -l4 rather than -l=4 > * fixed issues with the final patch when the autodetected range of cores > was >RTE_MAX_LCORE > > V9: rebase to latest main. CI complains cannot apply v8 patches. > > V8: > * dropped the final two patches from the series, dropping the new -L > option in favour of the -R modifier. > * reordered patch 11 to be with the other argparse patches (now patch 5) > * added patch 12, which uses macros to initialize the args structure > from the arguments header file, avoiding potential issues when we add > new args. > * simplified and consolidated lcore mask and core list parsing to always > work off cpusets rather than arrays of uint8 > * enhanced debug printouts to also work better with cpusets and handle > core values in those sets >= RTE_MAX_LCORE > * for completeness, ensure the new -R option works for coremasks, and > for cases where no explicit core-list or coremask is specified. > > V7: > * expand the scope of the patchset beyond just cleanup to add in the > extra 3 patches for -L and --lcoreid-base option. > > V6: > * Rebase to apply cleanly on latest main > > V5: > * Resubmit of V4, which didn't get sent correctly, or picked up correctly > in patchwork. > > V4: > * Updated patch 5 to auto-generate the arg struct definition from the same > list of defines used to construct the argument list. > > V3: > * Added 3 new initial patches, one for minor build-system addition, and two > for functionality in argparse to allow the user-callback help function to > be maintained as we move to argparse. > * Added doc updates in the first EAL patch adding the long options > * Fixed ASAN issues by adding a patch to properly clean up EAL init - both > memory allocations and fixing the run-once flag > * Put ifdefs around the linux-only or unix-only options in EAL patch 2 > * Updated args to handle numa-mem and numa-limit as equivalent socket-mem > and socket-limit > > Bruce Richardson (21): > build: add define for the OS environment name > test/func_reentrancy: fix args to EAL init call > argparse: export function to print help text for object > argparse: allow user-override of help printing > argparse: add documentation on supported value types > argparse: add support for parsing core lists > argparse: allow optional flag reordering > argparse: support parameters to short options without "=" > eal: add long options for each short option > eal: define the EAL parameters in argparse format > eal: gather EAL args before processing > eal: ensure proper cleanup on EAL init failure > eal: combine parameter validation checks > eal: simplify handling of conflicting cmdline options > eal: automatically init arg list options > eal: add internal fn for converting cpuset to string > eal: use common cpuset to string function > eal: introduce lcore remapping option for coremasks > eal: rework internal coremask parsing to use cpu sets > eal: allow lcore ID remapping with core lists > eal: allow lcore remapping with autodetected core affinity > > app/test/test_argparse.c | 695 ++++++++++ > app/test/test_func_reentrancy.c | 5 +- > config/meson.build | 1 + > doc/guides/contributing/design.rst | 2 +- > doc/guides/linux_gsg/eal_args.include.rst | 42 +- > doc/guides/prog_guide/argparse_lib.rst | 138 +- > doc/guides/rel_notes/release_25_11.rst | 18 + > drivers/event/dlb2/dlb2_priv.h | 2 +- > drivers/event/dlb2/pf/base/dlb2_resource.c | 11 +- > lib/argparse/rte_argparse.c | 181 ++- > lib/argparse/rte_argparse.h | 29 +- > lib/eal/common/eal_common_lcore.c | 13 +- > lib/eal/common/eal_common_memory.c | 3 +- > lib/eal/common/eal_common_options.c | 1439 +++++++++++--------- > lib/eal/common/eal_common_thread.c | 43 +- > lib/eal/common/eal_option_list.h | 98 ++ > lib/eal/common/eal_options.h | 104 +- > lib/eal/common/eal_private.h | 21 + > lib/eal/common/eal_thread.h | 13 +- > lib/eal/freebsd/eal.c | 245 +--- > lib/eal/include/rte_eal.h | 20 - > lib/eal/linux/eal.c | 470 +------ > lib/eal/linux/eal_memory.c | 2 +- > lib/eal/meson.build | 2 +- > lib/eal/windows/eal.c | 156 +-- > lib/meson.build | 1 + > 26 files changed, 2148 insertions(+), 1606 deletions(-) > create mode 100644 lib/eal/common/eal_option_list.h > > -- > 2.48.1 > There is one other thing about eal_init. The arguments in argv can not be const right now, since there are cases where args are modified. Could we fix that. Code like dpdk-dumpcap has to strdup() the args.
