Re: [PATCH v2 4/7] util: Add write-only "node-affinity" property for ThreadContext
On 12.10.22 10:03, Markus Armbruster wrote: Acked-by: Markus Armbruster Thanks, I'll similarly adjust to use "host nodes" and "host CPUs". -- Thanks, David / dhildenb
Re: [PATCH v2 4/7] util: Add write-only "node-affinity" property for ThreadContext
David Hildenbrand writes: > On 11.10.22 08:03, Markus Armbruster wrote: >> David Hildenbrand writes: >> >>> Let's make it easier to pin threads created via a ThreadContext to >>> all CPUs currently belonging to a given set of NUMA nodes -- which is the >>> common case. >>> >>> "node-affinity" is simply a shortcut for setting "cpu-affinity" manually >>> to the list of CPUs belonging to the set of nodes. This property can only >>> be written. >>> >>> A simple QEMU example to set the CPU affinity to Node 1 on a system with >>> two NUMA nodes, 24 CPUs each: >>> qemu-system-x86_64 -S \ >>>-object thread-context,id=tc1,node-affinity=1 >>> >>> And we can query the cpu-affinity via HMP/QMP: >>> (qemu) qom-get tc1 cpu-affinity >>> [ >>> 1, >>> 3, >>> 5, >>> 7, >>> 9, >>> 11, >>> 13, >>> 15, >>> 17, >>> 19, >>> 21, >>> 23, >>> 25, >>> 27, >>> 29, >>> 31, >>> 33, >>> 35, >>> 37, >>> 39, >>> 41, >>> 43, >>> 45, >>> 47 >>> ] >> >> Double-checking my understanding: on this system, the even CPUs belong >> to NUMA node 0, and the odd ones to node 1. Setting node-affinity=1 is >> therefore sugar for setting cpu-affinity to the set of even CPUs. >> Correct? > > Yes! > > # lscpu > ... > NUMA: > NUMA node(s): 2 > NUMA node0 CPU(s): > 0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46 > NUMA node1 CPU(s): > 1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,33,35,37,39,41,43,45,47 Thanks! >>> We cannot query the node-affinity: >>> (qemu) qom-get tc1 node-affinity >>> Error: Insufficient permission to perform this operation >> >> The error message is somewhat misleading. "Insufficient permission" >> suggests this could work if I had more "permission". Not the case. The >> message comes from object_property_get(), i.e. it's not this patch's >> fault. I'll post a patch to improve it. > > I agree, thanks! QAPI schema Acked-by: Markus Armbruster
Re: [PATCH v2 4/7] util: Add write-only "node-affinity" property for ThreadContext
On 11.10.22 08:03, Markus Armbruster wrote: David Hildenbrand writes: Let's make it easier to pin threads created via a ThreadContext to all CPUs currently belonging to a given set of NUMA nodes -- which is the common case. "node-affinity" is simply a shortcut for setting "cpu-affinity" manually to the list of CPUs belonging to the set of nodes. This property can only be written. A simple QEMU example to set the CPU affinity to Node 1 on a system with two NUMA nodes, 24 CPUs each: qemu-system-x86_64 -S \ -object thread-context,id=tc1,node-affinity=1 And we can query the cpu-affinity via HMP/QMP: (qemu) qom-get tc1 cpu-affinity [ 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29, 31, 33, 35, 37, 39, 41, 43, 45, 47 ] Double-checking my understanding: on this system, the even CPUs belong to NUMA node 0, and the odd ones to node 1. Setting node-affinity=1 is therefore sugar for setting cpu-affinity to the set of even CPUs. Correct? Yes! # lscpu ... NUMA: NUMA node(s): 2 NUMA node0 CPU(s): 0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46 NUMA node1 CPU(s): 1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,33,35,37,39,41,43,45,47 We cannot query the node-affinity: (qemu) qom-get tc1 node-affinity Error: Insufficient permission to perform this operation The error message is somewhat misleading. "Insufficient permission" suggests this could work if I had more "permission". Not the case. The message comes from object_property_get(), i.e. it's not this patch's fault. I'll post a patch to improve it. I agree, thanks! -- Thanks, David / dhildenb
Re: [PATCH v2 4/7] util: Add write-only "node-affinity" property for ThreadContext
David Hildenbrand writes:
> Let's make it easier to pin threads created via a ThreadContext to
> all CPUs currently belonging to a given set of NUMA nodes -- which is the
> common case.
>
> "node-affinity" is simply a shortcut for setting "cpu-affinity" manually
> to the list of CPUs belonging to the set of nodes. This property can only
> be written.
>
> A simple QEMU example to set the CPU affinity to Node 1 on a system with
> two NUMA nodes, 24 CPUs each:
> qemu-system-x86_64 -S \
> -object thread-context,id=tc1,node-affinity=1
>
> And we can query the cpu-affinity via HMP/QMP:
> (qemu) qom-get tc1 cpu-affinity
> [
> 1,
> 3,
> 5,
> 7,
> 9,
> 11,
> 13,
> 15,
> 17,
> 19,
> 21,
> 23,
> 25,
> 27,
> 29,
> 31,
> 33,
> 35,
> 37,
> 39,
> 41,
> 43,
> 45,
> 47
> ]
Double-checking my understanding: on this system, the even CPUs belong
to NUMA node 0, and the odd ones to node 1. Setting node-affinity=1 is
therefore sugar for setting cpu-affinity to the set of even CPUs.
Correct?
> We cannot query the node-affinity:
> (qemu) qom-get tc1 node-affinity
> Error: Insufficient permission to perform this operation
The error message is somewhat misleading. "Insufficient permission"
suggests this could work if I had more "permission". Not the case. The
message comes from object_property_get(), i.e. it's not this patch's
fault. I'll post a patch to improve it.
> But note that due to dynamic library loading this example will not work
> before we actually make use of thread_context_create_thread() in QEMU
> code, because the type will otherwise not get registered.
>
> Note that if the CPUs for a node change due do physical CPU hotplug or
> hotunplug (e.g., lscpu output changes) after the ThreadContext was started,
> the CPU affinity will not get updated.
>
> Reviewed-by: Michal Privoznik
> Signed-off-by: David Hildenbrand
> ---
> qapi/qom.json | 9 -
> util/meson.build | 2 +-
> util/thread-context.c | 84 +++
> 3 files changed, 93 insertions(+), 2 deletions(-)
>
> diff --git a/qapi/qom.json b/qapi/qom.json
> index 67d47f4051..cb6f700c0c 100644
> --- a/qapi/qom.json
> +++ b/qapi/qom.json
> @@ -839,10 +839,17 @@
> #created in the thread context (default: QEMU main thread
> #affinity)
> #
> +# @node-affinity: the list of node numbers that will be resolved to a list
> +# of CPU numbers used as CPU affinity. This is a shortcut for
> +# specifying the list of CPU numbers belonging to the nodes
> +# manually by setting @cpu-affinity. (default: QEMU main
> thread
> +# affinity)
> +#
> # Since: 7.2
> ##
> { 'struct': 'ThreadContextProperties',
> - 'data': { '*cpu-affinity': ['uint16'] } }
> + 'data': { '*cpu-affinity': ['uint16'],
> +'*node-affinity': ['uint16'] } }
>
>
> ##
[...]
