* Alexey Kardashevskiy (a...@ozlabs.ru) wrote: > On 07/09/17 19:51, Dr. David Alan Gilbert wrote: > > * Alexey Kardashevskiy (a...@ozlabs.ru) wrote: > >> This was inspired by https://bugzilla.redhat.com/show_bug.cgi?id=1481593 > >> > >> What happens ithere is that every virtio block device creates 2 address > >> spaces - for modern config space (called "virtio-pci-cfg-as") and > >> for busmaster (common pci thing, called after the device name, > >> in my case "virtio-blk-pci"). > >> > >> Each address_space_init() updates topology for every address space. > >> Every topology update (address_space_update_topology()) creates a new > >> dispatch tree - AddressSpaceDispatch with nodes (1KB) and > >> sections (48KB) and destroys the old one. > >> > >> However the dispatch destructor is postponed via RCU which does not > >> get a chance to execute until the machine is initialized but before > >> we get there, memory is not returned to the pool, and this is a lot > >> of memory which grows n^2. > >> > >> These patches are trying to address the memory use and boot time > >> issues but tbh only the first one provides visible outcome. > > > > Do you have a feel for how much memory is saved? > > > The 1/4 saves ~33GB (~44GB -> 11GB) for a 2GB guest and 400 virtio-pci > devices. These GB figures are the peak values (but it does not matter for > OOM killer), memory gets released in one go when RCU kicks in, it just > happens too late.
Nice saving! Still, why is it using 11GB? What's it like for more sane configurations, say 2-3 virtio devices - is there anything noticable or is it just the huge setups? Dave > The 3/4 saves less, I'd say 50KB per VCPU (more if you count peaks but so > much). Strangely, I do not see the difference in valgrind output when I run > a guest with 1024 or just 8 CPUs, probably "massif" is not the right tool > to catch this. > > > > > Dave > > > >> There are still things to polish and double check the use of RCU, > >> I'd like to get any feedback before proceeding - is this going > >> the right way or way too ugly? > >> > >> > >> This is based on sha1 > >> 1ab5eb4efb Peter Maydell "Update version for v2.10.0 release". > >> > >> Please comment. Thanks. > >> > >> > >> > >> Alexey Kardashevskiy (4): > >> memory: Postpone flatview and dispatch tree building till all devices > >> are added > >> memory: Prepare for shared flat views > >> memory: Share flat views and dispatch trees between address spaces > >> memory: Add flat views to HMP "info mtree" > >> > >> include/exec/memory-internal.h | 6 +- > >> include/exec/memory.h | 93 +++++++++---- > >> exec.c | 242 +++++++++++++++++++-------------- > >> hw/alpha/typhoon.c | 2 +- > >> hw/dma/rc4030.c | 4 +- > >> hw/i386/amd_iommu.c | 2 +- > >> hw/i386/intel_iommu.c | 9 +- > >> hw/intc/openpic_kvm.c | 2 +- > >> hw/pci-host/apb.c | 2 +- > >> hw/pci/pci.c | 3 +- > >> hw/ppc/spapr_iommu.c | 4 +- > >> hw/s390x/s390-pci-bus.c | 2 +- > >> hw/vfio/common.c | 6 +- > >> hw/virtio/vhost.c | 6 +- > >> memory.c | 299 > >> +++++++++++++++++++++++++++-------------- > >> monitor.c | 3 +- > >> vl.c | 4 + > >> hmp-commands-info.hx | 7 +- > >> 18 files changed, 448 insertions(+), 248 deletions(-) > >> > >> -- > >> 2.11.0 > >> > >> > > -- > > Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK > > > > > -- > Alexey -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK