From: Nikita Kalyazin <[email protected]> [ based on kvm/next ]
Unmapping virtual machine guest memory from the host kernel's direct map is a successful mitigation against Spectre-style transient execution issues: if the kernel page tables do not contain entries pointing to guest memory, then any attempted speculative read through the direct map will necessarily be blocked by the MMU before any observable microarchitectural side-effects happen. This means that Spectre-gadgets and similar cannot be used to target virtual machine memory. Roughly 60% of speculative execution issues fall into this category [1, Table 1]. This patch series extends guest_memfd with the ability to remove its memory from the host kernel's direct map, to be able to attain the above protection for KVM guests running inside guest_memfd. Additionally, a Firecracker branch with support for these VMs can be found on GitHub [2]. For more details, please refer to the v5 cover letter. No substantial changes in design have taken place since. See also related write() syscall support in guest_memfd [3] where the interoperation between the two features is described. Changes since v11: - Ackerley/Sashiko: fix previously missed __set_pages_* argument update in __kernel_map_pages (patch 1) - David: disallow large folios in folio_zap_direct_map (patch 2) - David/Sashiko: check for folio_is_zone_device if mapping is NULL in gup_fast_folio_allowed (patch 4) - Ackerley/Sashiko: kvm_arch_gmem_supports_no_direct_map to return false for SEV-SNP (patch 8). - David: replace a redundant check for GUEST_MEMFD_FLAG_NO_DIRECT_MAP with a WARN_ON_ONCE (patch 10) - David: assert the folio is locked when zapping direct map (patch 10) - Ackerley/Sashiko: reorder operations to "zap then prepare" and "invalidate then restore" (patch 10) v11: https://lore.kernel.org/kvm/[email protected] v10: https://lore.kernel.org/kvm/[email protected] v9: https://lore.kernel.org/kvm/[email protected] v8: https://lore.kernel.org/kvm/[email protected] v7: https://lore.kernel.org/kvm/[email protected] v6: https://lore.kernel.org/kvm/[email protected] v5: https://lore.kernel.org/kvm/[email protected] v4: https://lore.kernel.org/kvm/[email protected] RFCv3: https://lore.kernel.org/kvm/[email protected] RFCv2: https://lore.kernel.org/kvm/[email protected] RFCv1: https://lore.kernel.org/kvm/[email protected] [1] https://download.vusec.net/papers/quarantine_raid23.pdf [2] https://github.com/firecracker-microvm/firecracker/tree/feature/secret-hiding [3] https://lore.kernel.org/kvm/[email protected] Nikita Kalyazin (4): set_memory: set_direct_map_* to take address set_memory: add folio_{zap,restore}_direct_map helpers mm/secretmem: make use of folio_{zap,restore}_direct_map mm/gup: drop local variable in gup_fast_folio_allowed Patrick Roy (12): mm/gup: drop secretmem optimization from gup_fast_folio_allowed mm: introduce AS_NO_DIRECT_MAP KVM: guest_memfd: Add stub for kvm_arch_gmem_invalidate KVM: x86: define kvm_arch_gmem_supports_no_direct_map() KVM: arm64: define kvm_arch_gmem_supports_no_direct_map() KVM: guest_memfd: Add flag to remove from direct map KVM: selftests: load elf via bounce buffer KVM: selftests: set KVM_MEM_GUEST_MEMFD in vm_mem_add() if guest_memfd != -1 KVM: selftests: Add guest_memfd based vm_mem_backing_src_types KVM: selftests: cover GUEST_MEMFD_FLAG_NO_DIRECT_MAP in existing selftests KVM: selftests: stuff vm_mem_backing_src_type into vm_shape KVM: selftests: Test guest execution from direct map removed gmem Documentation/virt/kvm/api.rst | 21 +++--- arch/arm64/include/asm/kvm_host.h | 13 ++++ arch/arm64/include/asm/set_memory.h | 7 +- arch/arm64/mm/pageattr.c | 19 +++-- arch/loongarch/include/asm/set_memory.h | 7 +- arch/loongarch/mm/pageattr.c | 25 +++---- arch/riscv/include/asm/set_memory.h | 7 +- arch/riscv/mm/pageattr.c | 17 +++-- arch/s390/include/asm/set_memory.h | 7 +- arch/s390/mm/pageattr.c | 13 ++-- arch/x86/include/asm/kvm_host.h | 6 ++ arch/x86/include/asm/set_memory.h | 7 +- arch/x86/kvm/x86.c | 7 ++ arch/x86/mm/pat/set_memory.c | 27 +++---- include/linux/kvm_host.h | 14 ++++ include/linux/pagemap.h | 16 ++++ include/linux/secretmem.h | 18 ----- include/linux/set_memory.h | 22 +++++- include/uapi/linux/kvm.h | 1 + kernel/power/snapshot.c | 4 +- lib/buildid.c | 8 +- mm/execmem.c | 6 +- mm/gup.c | 47 ++++++------ mm/memory.c | 45 +++++++++++ mm/mlock.c | 2 +- mm/secretmem.c | 18 ++--- mm/vmalloc.c | 11 ++- .../testing/selftests/kvm/guest_memfd_test.c | 17 ++++- .../testing/selftests/kvm/include/kvm_util.h | 37 ++++++--- .../testing/selftests/kvm/include/test_util.h | 8 ++ tools/testing/selftests/kvm/lib/elf.c | 8 +- tools/testing/selftests/kvm/lib/io.c | 23 ++++++ tools/testing/selftests/kvm/lib/kvm_util.c | 59 ++++++++------- tools/testing/selftests/kvm/lib/test_util.c | 8 ++ tools/testing/selftests/kvm/lib/x86/sev.c | 1 + .../selftests/kvm/pre_fault_memory_test.c | 1 + .../selftests/kvm/set_memory_region_test.c | 52 ++++++++++++- .../kvm/x86/private_mem_conversions_test.c | 7 +- virt/kvm/guest_memfd.c | 75 +++++++++++++++++-- 39 files changed, 489 insertions(+), 202 deletions(-) base-commit: 24f9515de8778410e4b84c85b196c9850d2c1e18 -- 2.50.1
