Based-on: 20210406080126.24010-1-da...@redhat.com Some cleanups previously sent in other context (resizeable allocations), followed by RAM_NORESERVE, implementing it under Linux using MAP_NORESERVE, and letting users configure it for memory backens using the "reserve" property (default: true).
MAP_NORESERVE under Linux has in the context of QEMU an effect on 1) Private/shared anonymous memory -> memory-backend-ram,id=mem0,size=10G 2) Private fd-based mappings -> memory-backend-file,id=mem0,size=10G,mem-path=/dev/shm/0 -> memory-backend-memfd,id=mem0,size=10G 3) Private/shared hugetlb mappings -> memory-backend-memfd,id=mem0,size=10G,hugetlb=on,hugetlbsize=2M With MAP_NORESERVE/"reserve=off", we won't be reserving swap space (1/2) or huge pages (3) for the whole memory region. The target use case is virtio-mem, which dynamically exposes memory inside a large, sparse memory area to the VM. MAP_NORESERVE tells the OS "this mapping might be very sparse". This essentially allows avoiding having to set "/proc/sys/vm/overcommit_memory == 1") when using virtio-mem and also supporting hugetlbfs in the future. v4 -> v5: - Sent out shared anonymous RAM fixes separately - Rebased - "hostmem: Wire up RAM_NORESERVE via "reserve" property" -- Adjusted/simplified description of new "reserve" property -- Properly add it to qapi/qom.json - "qmp: Clarify memory backend properties returned via query-memdev" -- Added - "qmp: Include "share" property of memory backends" -- Added - "hmp: Print "share" property of memory backends with "info memdev"" - Added - "qmp: Include "reserve" property of memory backends" -- Adjust description of new "reserve" property v3 -> v4: - Minor comment/description updates - "softmmu/physmem: Fix ram_block_discard_range() to handle shared ..." -- Extended description - "util/mmap-alloc: Pass flags instead of separate bools to ..." -- Move flags to include/qemu/osdep.h and rename to "QEMU_MAP_*" - "memory: Introduce RAM_NORESERVE and wire it up in qemu_ram_mmap()" -- Adjust to new flags. Handle errors in mmap_activate() for now. - "util/mmap-alloc: Support RAM_NORESERVE via MAP_NORESERVE under Linux" -- Restrict support to Linux only for now - "qmp: Include "reserve" property of memory backends" -- Added - "hmp: Print "reserve" property of memory backends with ..." -- Added v2 -> v3: - Renamed "softmmu/physmem: Drop "shared" parameter from ram_block_add()" to "softmmu/physmem: Mark shared anonymous memory RAM_SHARED" and adjusted the description - Added "softmmu/physmem: Fix ram_block_discard_range() to handle shared anonymous memory" - Added "softmmu/physmem: Fix qemu_ram_remap() to handle shared anonymous memory" - Added "util/mmap-alloc: Pass flags instead of separate bools to qemu_ram_mmap()" - "util/mmap-alloc: Support RAM_NORESERVE via MAP_NORESERVE" -- Further tweak code comments -- Handle shared anonymous memory v1 -> v2: - Rebased to upstream and phs_mem_alloc simplifications -- Upsteam added the "map_offset" parameter to many RAM allocation interfaces. - "softmmu/physmem: Drop "shared" parameter from ram_block_add()" -- Use local variable "shared" - "memory: introduce RAM_NORESERVE and wire it up in qemu_ram_mmap()" -- Simplify due to phs_mem_alloc changes - "util/mmap-alloc: Support RAM_NORESERVE via MAP_NORESERVE" -- Add a whole bunch of comments. -- Exclude shared anonymous memory that QEMU doesn't use -- Special-case readonly mappings Cc: Peter Xu <pet...@redhat.com> Cc: "Michael S. Tsirkin" <m...@redhat.com> Cc: Eduardo Habkost <ehabk...@redhat.com> Cc: "Dr. David Alan Gilbert" <dgilb...@redhat.com> Cc: Richard Henderson <richard.hender...@linaro.org> Cc: Paolo Bonzini <pbonz...@redhat.com> Cc: Igor Mammedov <imamm...@redhat.com> Cc: "Philippe Mathieu-Daudé" <phi...@redhat.com> Cc: Stefan Hajnoczi <stefa...@redhat.com> Cc: Murilo Opsfelder Araujo <muri...@linux.ibm.com> Cc: Greg Kurz <gr...@kaod.org> Cc: Liam Merwick <liam.merw...@oracle.com> Cc: Marcel Apfelbaum <mapfe...@redhat.com> David Hildenbrand (14): util/mmap-alloc: Factor out calculation of the pagesize for the guard page util/mmap-alloc: Factor out reserving of a memory region to mmap_reserve() util/mmap-alloc: Factor out activating of memory to mmap_activate() softmmu/memory: Pass ram_flags to qemu_ram_alloc_from_fd() softmmu/memory: Pass ram_flags to memory_region_init_ram_shared_nomigrate() util/mmap-alloc: Pass flags instead of separate bools to qemu_ram_mmap() memory: Introduce RAM_NORESERVE and wire it up in qemu_ram_mmap() util/mmap-alloc: Support RAM_NORESERVE via MAP_NORESERVE under Linux hostmem: Wire up RAM_NORESERVE via "reserve" property qmp: Clarify memory backend properties returned via query-memdev qmp: Include "share" property of memory backends hmp: Print "share" property of memory backends with "info memdev" qmp: Include "reserve" property of memory backends hmp: Print "reserve" property of memory backends with "info memdev" backends/hostmem-file.c | 11 +- backends/hostmem-memfd.c | 8 +- backends/hostmem-ram.c | 7 +- backends/hostmem.c | 32 +++ hw/core/machine-hmp-cmds.c | 4 + hw/core/machine-qmp-cmds.c | 2 + hw/m68k/next-cube.c | 4 +- hw/misc/ivshmem.c | 5 +- include/exec/cpu-common.h | 1 + include/exec/memory.h | 42 ++-- include/exec/ram_addr.h | 9 +- include/qemu/mmap-alloc.h | 16 +- include/qemu/osdep.h | 30 ++- include/sysemu/hostmem.h | 2 +- migration/ram.c | 3 +- qapi/machine.json | 13 +- qapi/qom.json | 4 + .../memory-region-housekeeping.cocci | 8 +- softmmu/memory.c | 27 ++- softmmu/physmem.c | 44 ++-- util/mmap-alloc.c | 212 +++++++++++++----- util/oslib-posix.c | 7 +- util/oslib-win32.c | 13 +- 23 files changed, 349 insertions(+), 155 deletions(-) -- 2.30.2