Hi folks, Any more comments?
Thanks, Wanlong Gao > As you know, QEMU can't direct it's memory allocation now, this may cause > guest cross node access performance regression. > And, the worse thing is that if PCI-passthrough is used, > direct-attached-device uses DMA transfer between device and qemu process. > All pages of the guest will be pinned by get_user_pages(). > > KVM_ASSIGN_PCI_DEVICE ioctl > kvm_vm_ioctl_assign_device() > =>kvm_assign_device() > => kvm_iommu_map_memslots() > => kvm_iommu_map_pages() > => kvm_pin_pages() > > So, with direct-attached-device, all guest page's page count will be +1 and > any page migration will not work. AutoNUMA won't too. > > So, we should set the guest nodes memory allocation policy before > the pages are really mapped. > > According to this patch set, we are able to set guest nodes memory policy > like following: > > -numa node,nodeid=0,cpus=0, \ > -numa mem,size=1024M,policy=membind,host-nodes=0-1 \ > -numa node,nodeid=1,cpus=1 \ > -numa mem,size=1024M,policy=interleave,host-nodes=1 > > This supports > "policy={default|membind|interleave|preferred},relative=true,host-nodes=N-N" > like format. > > And add a QMP command "query-numa" to show numa info through > this API. > > And convert the "info numa" monitor command to use this > QMP command "query-numa". > > This version removes "set-mem-policy" qmp and hmp commands temporarily > as Marcelo and Paolo suggested. > > V1->V2: > change to use QemuOpts in numa options (Paolo) > handle Error in mpol parser (Paolo) > change qmp command format to mem-policy=membind,mem-hostnode=0-1 like > (Paolo) > V2->V3: > also handle Error in cpus parser (5/10) > split out common parser from cpus and hostnode parser (Bandan 6/10) > V3-V4: > rebase to request for comments > V4->V5: > use OptVisitor and split -numa option (Paolo) > - s/set-mpol/set-mem-policy (Andreas) > - s/mem-policy/policy > - s/mem-hostnode/host-nodes > fix hmp command process after error (Luiz) > add qmp command query-numa and convert info numa to it (Luiz) > V5->V6: > remove tabs in json file (Laszlo, Paolo) > add back "-numa node,mem=xxx" as legacy (Paolo) > change cpus and host-nodes to array (Laszlo, Eric) > change "nodeid" to "uint16" > add NumaMemPolicy enum type (Eric) > rebased on Laszlo's "OptsVisitor: support / flatten integer ranges for > repeating options" patch set, thanks for Laszlo's help > V6-V7: > change UInt16 to uint16 (Laszlo) > fix a typo in adding qmp command set-mem-policy > V7-V8: > rebase to current master with Laszlo's V2 of OptsVisitor patch set > fix an adding white space line error > V8->V9: > rebase to current master > check if total numa memory size is equal to ram_size (Paolo) > add comments to the OptsVisitor stuff in qapi-schema.json (Eric, Laszlo) > replace the use of numa_num_configured_nodes() (Andrew) > avoid abusing the fact i==nodeid (Andrew) > V9->V10: > rebase to current master > remove libnuma (Andrew) > MAX_NODES=64 -> MAX_NODES=128 since libnuma selected 128 (Andrew) > use MAX_NODES instead of MAX_CPUMASK_BITS for host_mem bitmap (Andrew) > remove a useless clear_bit() operation (Andrew) > V10->V11: > rebase to current master > fix "maxnode" argument of mbind(2) > V11->V12: > rebase to current master > split patch 02/11 of V11 (Eduardo) > add some max value check (Eduardo) > split MAX_NODES change patch (Eduardo) > V12->V13: > rebase to current master > thanks for Luiz's review (Luiz) > doc hmp command set-mem-policy (Luiz) > rename: NUMAInfo -> NUMANode (Luiz) > V13->V14: > remove "set-mem-policy" qmp and hmp commands (Marcelo, Paolo) > > > *I hope this can catch up the train of 1.7.* > > Thanks, > Wanlong Gao > > Wanlong Gao (11): > NUMA: move numa related code to new file numa.c > NUMA: check if the total numa memory size is equal to ram_size > NUMA: Add numa_info structure to contain numa nodes info > NUMA: convert -numa option to use OptsVisitor > NUMA: introduce NumaMemOptions > NUMA: add "-numa mem," options > NUMA: expand MAX_NODES from 64 to 128 > NUMA: parse guest numa nodes memory policy > NUMA: set guest numa nodes memory policy > NUMA: add qmp command query-numa > NUMA: convert hmp command info_numa to use qmp command query_numa > > Makefile.target | 2 +- > cpus.c | 14 -- > hmp.c | 57 +++++++ > hmp.h | 1 + > hw/i386/pc.c | 4 +- > include/sysemu/cpus.h | 1 - > include/sysemu/sysemu.h | 18 ++- > monitor.c | 21 +-- > numa.c | 395 > ++++++++++++++++++++++++++++++++++++++++++++++++ > qapi-schema.json | 112 ++++++++++++++ > qemu-options.hx | 6 +- > qmp-commands.hx | 48 ++++++ > vl.c | 160 +++----------------- > 13 files changed, 654 insertions(+), 185 deletions(-) > create mode 100644 numa.c >