This series enables virtio-balloon free page reporting and memory balloon inflation/deflation in Confidential Computing (CoCo) guests, starting with architectural hooks implemented for AMD SEV variants, in order to support dynamic CoCo guest resizing.
Today, memory ballooning and free page reporting are essentially unusable in CoCo guests due to three main roadblocks: 1. SWIOTLB Bounce-Buffering on Free Page Reporting: Under VIRTIO_F_ACCESS_PLATFORM, virtqueue_add_inbuf() maps every scatterlist entry through the DMA API. For reporting_vq, these are multi-megabyte ranges of free buddy pages. The host never DMA-reads or writes the page contents; it only inspects the GPA to punch holes in backing memory. Running these ranges through SWIOTLB exhausts bounce buffer pools and wastes cycles copying gigabytes of unencrypted free memory. 2. Dynamic Memory Unacceptance & Re-acceptance: When the host reclaims guest physical pages via hole punching, the guest must transition those GPAs back to unaccepted/shared before the host reclaims them (rescinding PVALIDATE and issuing a GHCB Page State Change on SEV-SNP). Otherwise, any subsequent access triggers a fatal #NPF or #VC. While the EFI unaccepted memory subsystem handles boot-time lazy acceptance, it lacks an API or state tracking to dynamically unaccept memory at runtime or route unaccepted freed pages in the page allocator. 3. Acceptance Granularity (2 MiB) vs. Balloon Pages (4 KiB): On x86, the EFI unaccepted memory bitmap tracks state at 2 MiB (PMD_ORDER) granularity both to keep the bitmap compact and for performance: accepting and unaccepting in 2 MiB blocks preserves 2 MiB host hugepage mappings (NPT/RMP on SEV-SNP and Secure EPT on TDX), avoids 512x per-4 KiB page state transition and validation overhead, and prevents RMP/SEPT size-mismatch splits. Because a 2 MiB bitmap unit cannot be partially unaccepted, CoCo free page reporting and balloon inflation require 2 MiB blocks and do not fall back to 4 KiB allocations (though low-level PVALIDATE still falls back to 4 KiB instructions if the host RMP entry was already split). Standard virtio-balloon allocates 4 KiB pages and caps its PFN array at 256 PFNs (1 KiB descriptor), so CoCo guests must allocate 2 MiB compound pages while streaming PFNs in 256-PFN chunks to respect existing VMM descriptor limits (such as QEMU's 1024-byte cap). As a trade-off of requiring 2 MiB blocks rather than 4 KiB pages, the effectiveness of balloon inflation/deflation in CoCo guests depends on guest memory not being overly fragmented (which guest kernel command-line options like movablecore / ZONE_MOVABLE or background compaction can help mitigate). Even when unmovable allocations fragment some pageblocks, free page reporting already operates at this same 2 MiB (pageblock_order) granularity on x86_64 and continues to reclaim contiguous free blocks opportunistically. Key Design Elements in This Series ================================== - Per-virtqueue DMA map ops (Patches 1-2): Extends virtio_ring to allow individual virtqueues to install custom struct virtio_map_ops. For reporting_vq, virtio_balloon installs a no-op GPA identity mapper so free page ranges are passed directly without SWIOTLB bounce buffers, while keeping the descriptor ring and any indirect descriptor tables securely mapped via the device-level DMA API. - Dynamic unacceptance & SEV-SNP hooks (Patches 3-4): Adds arch_unaccept_memory() / snp_unaccept_memory() for SEV-SNP. Also adds exception fixup in vc_raw_handle_exception() for SVM_EXIT_PAGE_NOT_VALIDATED (0x404) and SVM_EXIT_NPF (0x400) at EX_TYPE_ZEROPAD instructions so speculative reads from load_unaligned_zeropad() that touch the edge of dynamically unaccepted memory are handled cleanly without kernel panics. - Dynamic unaccepted memory tracking & MM routing (Patches 5-9): Anchors the EFI unaccepted table at base 0 when unaccepted memory is present, introduces mark_memory_unaccepted() with error reporting and proper locking, generalizes zone->unaccepted_pages to variable orders >= unaccepted_memory_order(), and updates mm/page_alloc, mm/page_isolation, mm/page_owner, and mm/page_reporting to route unaccepted pages back to zone->unaccepted_pages without corrupting them with page poisoning, KASAN, or init_on_free before re-acceptance. Generalizing zone->unaccepted_pages from MAX_PAGE_ORDER (4 MiB on x86) down to >= unaccepted_memory_order() (2 MiB) aligns allocator tracking with the underlying hardware validation/RMP/SEPT and EFI bitmap unit: even when boot-time lazy acceptance processes a MAX_PAGE_ORDER block, the hardware and bitmap validate it in 2 MiB chunks, and runtime allocations, pageblocks, and mm/page_reporting split accepted memory below MAX_PAGE_ORDER anyway. Note for reviewers: The three mm/ patches (Patches 7-9) are the most intricate part of this series due to the interactions between lazy acceptance, page isolation, and page reporting. Extra scrutiny from mm reviewers to ensure no page-allocator or pageblock corner cases were missed would be especially appreciated. - 2 MiB compound ballooning with 256-PFN chunking (Patch 10): Allocates compound pages at PMD_ORDER in CoCo guests that require page state transitions, breaking them into 256-PFN chunks when queuing to the host so virtqueue descriptors remain at 1 KiB, preserving out-of-the-box compatibility with existing hypervisors. - Balloon state transitions & CoCo ACCESS_PLATFORM (Patches 11-13): Calls mark_memory_unaccepted() on reported ranges and inflated pages before adding buffers to reporting_vq and kicking the virtqueue, routes deflated pages to unaccepted freelists, preserves VIRTIO_F_ACCESS_PLATFORM, disables VIRTIO_BALLOON_F_FREE_PAGE_HINT and unsupported VIRTIO_BALLOON_F_PAGE_POISON in CoCo guests, and rejects probing on CoCo platforms where balloon page state transitions are unsupported. Why Both Free Page Reporting and Inflation/Deflation ==================================================== CoCo guests using lazy memory acceptance boot with their maximum RAM declared as EFI_UNACCEPTED_MEMORY in the initial EFI memory map, avoiding runtime ACPI/virtio memory hotplug. Prior to this series, lazy acceptance was a one-way ratchet: once a workload spike caused the guest to accept pages from zone->unaccepted_pages, those pages could never be unaccepted or returned to the host. Turning EFI unaccepted memory into a bidirectional lifecycle enables two complementary mechanisms on initial boot RAM: - Free Page Reporting (reporting_vq): Autonomously reports idle 2 MiB free buddy pages to the host for hole-punching, and lazily re-accepts them locally from zone->unaccepted_pages on demand without a host virtqueue round-trip. - Balloon Inflation/Deflation (inflate_vq / deflate_vq): Allows VMMs that size guests without hotplugging to actively apply memory pressure (triggering guest reclaim) and enforce dynamic memory targets on boot RAM. Architecture Support & Intel TDX Follow-Up ========================================== The efi/unaccepted, mm/, and virtio_balloon changes in this series are architecture-agnostic, with architecture hooks (arch_unaccept_memory()) wired up first for AMD SEV-SNP via snp_unaccept_memory() (alongside VIRTIO_F_ACCESS_PLATFORM support for SEV-ES). On Intel TDX, I think dynamic unacceptance should use TDG.MEM.PAGE.RELEASE to transition private pages from MAPPED back to PENDING so the host VMM can reclaim them (and the guest can later re-accept them via the existing TDG.MEM.PAGE.ACCEPT path in tdx_accept_memory()). Once TDG.MEM.PAGE.RELEASE support and load_unaligned_zeropad() handling for dynamically unaccepted pages (e.g., via direct-map unmapping under SEPT_VE_DISABLE) land upstream in arch/x86/coco/tdx/, tdx_unaccept_memory() can slot directly into arch_unaccept_memory(). Testing ======= - AMD SEV-SNP Milan hardware (CoCo): Passed end-to-end memory pressure soak testing with live guest_memfd hole punching (FALLOC_FL_PUNCH_HOLE) and dynamic lazy re-acceptance under memory pressure on an internal hypervisor. - Cloud Hypervisor & QEMU: Verified standard non-confidential ballooning remains compatible. End-to-end SEV-SNP balloon testing on upstream Cloud Hypervisor and QEMU has been limited so far (while both punch holes in guest_memfd on shared page-state transitions, QEMU's virtio-balloon still issues MADV_WILLNEED on shared HVA mappings during deflate); any testing help or suggestions on these hypervisors are greatly appreciated. Note: An LLM coding assistant was used during the development of this series to assist with code and commit message drafting and refactoring. I have reviewed and verified all changes. Kevin Loughlin (13): virtio_ring: support per-virtqueue DMA mapping ops virtio_balloon: add reporting_vq DMA map ops x86/coco/sev: add snp_unaccept_memory() x86/coco/sev: fix zeropad #VC on unaccepted memory efi/libstub: start unaccepted bitmap at address 0 efi/unaccepted: add mark_memory_unaccepted() mm/page_alloc: allow lower-order unaccepted_pages mm/page_alloc: route unaccepted isolated pages mm/page_reporting: account for unaccepted memory virtio_balloon: add higher-order page inflation virtio_balloon: unaccept free reporting ranges virtio_balloon: unaccept inflated balloon pages virtio_balloon: enforce ACCESS_PLATFORM in CoCo Documentation/mm/physical_memory.rst | 3 +- arch/x86/coco/sev/core.c | 13 + arch/x86/coco/sev/vc-handle.c | 29 ++ arch/x86/include/asm/sev.h | 2 + arch/x86/include/asm/unaccepted_memory.h | 14 + arch/x86/include/uapi/asm/svm.h | 2 + .../firmware/efi/libstub/unaccepted_memory.c | 45 +++- drivers/firmware/efi/unaccepted_memory.c | 107 +++++++- drivers/virtio/virtio_balloon.c | 214 +++++++++++++-- drivers/virtio/virtio_ring.c | 136 +++++++--- include/linux/balloon.h | 1 + include/linux/mm.h | 18 ++ include/linux/mmzone.h | 2 +- include/linux/virtio.h | 4 + mm/balloon.c | 67 +++-- mm/page_alloc.c | 248 ++++++++++++----- mm/page_isolation.c | 44 +++- mm/page_owner.c | 7 +- mm/page_reporting.c | 34 ++- 19 files changed, 824 insertions(+), 166 deletions(-) -- 2.56.0.307.g5fe7bd680c-goog
