Re: [libvirt] [RFC][PATCHv2 00/11] add numatune command
On Thu, Nov 17, 2011 at 05:44:10PM +0800, Hu Tao wrote: This series does mainly two things: 1. use cgroup cpuset to manage numa parameters 2. add a virsh command numatune to allow user to change numa parameters from command line Current numa parameters include nodeset and mode, but these cgroup cpuset provides don't completely match with them, details: params cpuset -- nodeset cpuset provides cpuset.mems mode strict cpuset provides cpuset.mem_hardwall mode interleave cpuset provices cpuset.memory_spread_* mode preferred no equivalent. !spread to preferred? This isn't right - there are only 3 existing configs in the XML currently, current 'strict' does not map to mem_hardwall, nor does interleave map to memory_spread AFAICT Currently we have have three different configurations possible for memory with the following semantics mode=strict- allocation is from designated nodes, or fails mode=preferred - allocation is from designated nodes, or falls back to other nodes mode=interleave- allocation is interleaved across designated nodes In cgroups cpuset controller you can set cpuset.mems - memory is allocated from designated nodes, or fails cpuset.mem_exclusive - no other cgroups, except parents, or children can allocation from nos listed in cpuset.mems cpuset.mem_hardwall - no other cgroups are allowed to allocate from the nodes listed in cpuset.mems cpuset.memory_spread* - control allocations of internal kernel data structures IMHO, the last three are not really required for libvirt per VM usage - the management application can trivially decide whether to allow overlapping allocation between VMs without needing to set this kernel tunable. So, if using the cgroups cpuset controller for NUMA, the *only* policy we can implement is mode=strict. We cannot implement mode=preferred or mode=interleave, given the currently available cpuset controls. IMHO, we should thus continue to use libnuma for specifying *all* the policies, however, if mode=strict, then we should *also* apply the policy in the cgroups using cpuset.mems since this will at least allow later tuning of nodemask on the fly. We will have to refuse any attempt to switch between different modes on the fly. Only the nodemask, with mode=strict will be dynamically changable. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [RFC][PATCHv2 00/11] add numatune command
This series does mainly two things:
1. use cgroup cpuset to manage numa parameters
2. add a virsh command numatune to allow user to change numa parameters
from command line
Current numa parameters include nodeset and mode, but these cgroup cpuset
provides don't completely match with them, details:
params cpuset
--
nodeset cpuset provides cpuset.mems
mode strict cpuset provides cpuset.mem_hardwall
mode interleave cpuset provices cpuset.memory_spread_*
mode preferred no equivalent. !spread to preferred?
Besides, only one of the mode can be set currently at a time, but
for cpuset, the parameters are independent. From the perspective
of cpuset, we can set all the modes values independently, but it
seems not be consistent with the current numatune definition in xml.
Maybe we can improve the xml definition to fit cpuset better?(there
are more cpuset parameters than listed above)
Hu Tao (11):
don't modify CPU set string in virDomainCpuSetParse
enable cgroup cpuset by default
Add functions to set/get cgroup cpuset parameters
introduce numa backend
use cpuset to manage numa
add VIR_DOMAIN_NUMATUNE_MEM_NONE
add new API virDomain{G,S}etNumaParameters
Implement main entries of virDomain{G,S}etNumaParameters
Add virDomain{G,S}etNumaParameters support to the remote driver
Implement virDomain{G,S}etNumaParameters for the qemu driver
add new command numatune to virsh
daemon/remote.c | 64 +++
include/libvirt/libvirt.h.in | 23 +++
python/generator.py |2 +
src/conf/domain_conf.c |2 +-
src/conf/domain_conf.h |9 +
src/driver.h | 15 ++
src/libvirt.c| 113
src/libvirt_private.syms | 10 +
src/libvirt_public.syms |6 +
src/qemu/qemu.conf |5 +-
src/qemu/qemu_cgroup.c | 73
src/qemu/qemu_conf.c |3 +-
src/qemu/qemu_driver.c | 399 ++
src/qemu/qemu_process.c | 11 +-
src/remote/remote_driver.c | 50 ++
src/remote/remote_protocol.x | 25 +++-
src/remote_protocol-structs | 16 ++
src/util/cgroup.c| 112
src/util/cgroup.h| 11 ++
tools/virsh.c| 180 +++
20 files changed, 1121 insertions(+), 8 deletions(-)
--
1.7.3.1
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [RFC][PATCHv2 00/11] add numatune command
于 2011年11月17日 17:44, Hu Tao 写道:
This series does mainly two things:
1. use cgroup cpuset to manage numa parameters
2. add a virsh command numatune to allow user to change numa parameters
from command line
Current numa parameters include nodeset and mode, but these cgroup cpuset
provides don't completely match with them, details:
params cpuset
--
nodeset cpuset provides cpuset.mems
mode strict cpuset provides cpuset.mem_hardwall
mode interleave cpuset provices cpuset.memory_spread_*
mode preferred no equivalent. !spread to preferred?
Besides, only one of the mode can be set currently at a time, but
for cpuset, the parameters are independent. From the perspective
of cpuset, we can set all the modes values independently,
Which of the mode will work actually? Per they are independant
with each other.
but it
seems not be consistent with the current numatune definition in xml.
Maybe we can improve the xml definition to fit cpuset better?(there
are more cpuset parameters than listed above)
As long as the XML is there, it can't be changed for backwards
compatibility, make addtiontions on the existed XML may work?
e.g.
numatune type='libnuma'
memory mode='strict' nodeset='0-10, 15,20'/
/numatune
numatune type='cgroup'
memory mode='strict' nodeset='0,1'/
memory mode='interleave' nodeset='2,3'/
memory mode='preferred' nodeset='4,5'/
...
more for the lots of cpuset.mem*?
/numatune
The type can be libnuma by default for backwards campatible.
Hu Tao (11):
don't modify CPU set string in virDomainCpuSetParse
enable cgroup cpuset by default
Add functions to set/get cgroup cpuset parameters
introduce numa backend
use cpuset to manage numa
add VIR_DOMAIN_NUMATUNE_MEM_NONE
add new API virDomain{G,S}etNumaParameters
Implement main entries of virDomain{G,S}etNumaParameters
Add virDomain{G,S}etNumaParameters support to the remote driver
Implement virDomain{G,S}etNumaParameters for the qemu driver
add new command numatune to virsh
daemon/remote.c | 64 +++
include/libvirt/libvirt.h.in | 23 +++
python/generator.py |2 +
src/conf/domain_conf.c |2 +-
src/conf/domain_conf.h |9 +
src/driver.h | 15 ++
src/libvirt.c| 113
src/libvirt_private.syms | 10 +
src/libvirt_public.syms |6 +
src/qemu/qemu.conf |5 +-
src/qemu/qemu_cgroup.c | 73
src/qemu/qemu_conf.c |3 +-
src/qemu/qemu_driver.c | 399 ++
src/qemu/qemu_process.c | 11 +-
src/remote/remote_driver.c | 50 ++
src/remote/remote_protocol.x | 25 +++-
src/remote_protocol-structs | 16 ++
src/util/cgroup.c| 112
src/util/cgroup.h| 11 ++
tools/virsh.c| 180 +++
20 files changed, 1121 insertions(+), 8 deletions(-)
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
