On 3/31/21 2:06 AM, Juraj Linkeš wrote:
Add an option to automatically discover the host's numa and cpu counts and use those values for a non cross-build. Give users the option to override the per-arch default values or values from cross files by specifying them on the command line with -Dmax_lcores and -Dmax_numa_nodes. Signed-off-by: Juraj Linkeš <[email protected]> Reviewed-by: Honnappa Nagarahalli <[email protected]> --- MAINTAINERS | 2 ++ buildtools/get-cpu-count.py | 7 ++++++ buildtools/get-numa-count.py | 22 +++++++++++++++++ buildtools/meson.build | 2 ++ config/meson.build | 47 ++++++++++++++++++++++++++++++++++-- config/x86/meson.build | 2 ++ meson_options.txt | 8 +++--- 7 files changed, 84 insertions(+), 6 deletions(-) create mode 100644 buildtools/get-cpu-count.py create mode 100644 buildtools/get-numa-count.py diff --git a/MAINTAINERS b/MAINTAINERS index 0ec5588540..7270f33cf5 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -99,6 +99,8 @@ F: meson_options.txt F: config/ F: buildtools/chkincs/ F: buildtools/call-sphinx-build.py +F: buildtools/get-cpu-count.py +F: buildtools/get-numa-count.py F: buildtools/list-dir-globs.py F: buildtools/pkg-config/ F: buildtools/symlink-drivers-solibs.sh diff --git a/buildtools/get-cpu-count.py b/buildtools/get-cpu-count.py new file mode 100644 index 0000000000..317b32088f --- /dev/null +++ b/buildtools/get-cpu-count.py @@ -0,0 +1,7 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright (c) 2021 PANTHEON.tech s.r.o. + +import os + +print(os.cpu_count())
Returns the expected value on a P9 system.
diff --git a/buildtools/get-numa-count.py b/buildtools/get-numa-count.py
new file mode 100644
index 0000000000..77ef2b9f24
--- /dev/null
+++ b/buildtools/get-numa-count.py
@@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright (c) 2021 PANTHEON.tech s.r.o.
+
+import ctypes
+import glob
+import os
+import subprocess
+
+if os.name == 'posix':
+ if os.path.isdir('/sys/devices/system/node'):
+ print(len(glob.glob('/sys/devices/system/node/node*')))
+ else:
+ subprocess.run(['sysctl', '-n', 'vm.ndomains'])
+
+elif os.name == 'nt':
+ libkernel32 = ctypes.windll.kernel32
+
+ count = ctypes.c_ulong()
+
+ libkernel32.GetNumaHighestNodeNumber(ctypes.pointer(count))
+ print(count.value + 1)
Does not return the expected value on my P9 system (NUMA nodes are not contiguous). Got 6, expect to see at least 8 otherwise I can't use lcores on NUMA node 8.
$ python3 ./get-numa-count.py 6 $ lscpu Architecture: ppc64le Byte Order: Little Endian CPU(s): 128 On-line CPU(s) list: 0-127 Thread(s) per core: 4 Core(s) per socket: 16 Socket(s): 2 NUMA node(s): 6 Model: 2.3 (pvr 004e 1203) Model name: POWER9, altivec supported CPU max MHz: 3800.0000 CPU min MHz: 2300.0000 L1d cache: 32K L1i cache: 32K L2 cache: 512K L3 cache: 10240K NUMA node0 CPU(s): 0-63 NUMA node8 CPU(s): 64-127 NUMA node252 CPU(s): NUMA node253 CPU(s): NUMA node254 CPU(s): NUMA node255 CPU(s): See attached ls-node.txt.gz for full directory structure on a P9 system. Dave
ls-node.txt.gz
Description: GNU Zip compressed data
