Support iommufd devices with the cpr-exec live migration mode. No user-visible interfaces are added.
Pass the iommufd and vfio device descriptors from old to new QEMU. In new QEMU, during vfio_realize, skip the ioctls that configure the device, because it is already configured. In new QEMU, call ioctl(IOMMU_IOAS_CHANGE_PROCESS) to update the mm ownership, locked memory accounting, and virtual address of all DMA mappings. The old virtual address of each memory region is needed to identify the existing mapping, so pass the host address of each RAMBlock in the migration data stream. Block CPR if the iommufd container has any vfio mdevs (mediated devices). IOMMU_IOAS_CHANGE_PROCESS can be used as-is to support mdevs, but it requires extra work in userland at CPR time so that kernel threads have access to the old mappings until the mappings are updated in new QEMU. I have prototyped those changes but they need more work before posting. This series depends on the following qemu series [PATCH V1 00/08] Live update: vfio https://lore.kernel.org/qemu-devel/1720558737-451106-1-git-send-email-steven.sist...@oracle.com/ This series depends on the IOMMU_IOAS_CHANGE_PROCESS kernel interface which is a work in progress: iommufd live update https://lore.kernel.org/linux-iommu/1721501805-86928-1-git-send-email-steven.sist...@oracle.com Steve Sistare (12): vfio: move cpr_exec_notifier iommufd: no DMA to BARs iommufd: pass name to connect migration: cpr_find_fd_any iommufd: preserve device fd iommufd: export iommufd_cdev_get_info_iova_range iommufd: change_process kernel interface vfio/iommufd: register container for cpr vfio/iommufd: rebuild device migration/ram: old host address iommufd: update DMA virtual addresses vfio: mdev blocker backends/iommufd.c | 113 ++++++++++++++++++++++++++++++++-- hw/core/machine.c | 6 ++ hw/vfio/common.c | 3 +- hw/vfio/cpr-iommufd.c | 84 +++++++++++++++++++++++++ hw/vfio/cpr-legacy.c | 10 +-- hw/vfio/helpers.c | 1 + hw/vfio/iommufd.c | 43 ++++++++++--- hw/vfio/meson.build | 1 + hw/vfio/pci.c | 10 +++ include/exec/memory.h | 1 + include/exec/ramblock.h | 1 + include/hw/vfio/vfio-common.h | 7 ++- include/hw/vfio/vfio-container-base.h | 3 + include/migration/cpr.h | 1 + include/sysemu/iommufd.h | 7 ++- linux-headers/linux/iommufd.h | 19 ++++++ migration/cpr.c | 15 +++++ migration/migration.h | 2 + migration/options.c | 2 + migration/ram.c | 7 +++ 20 files changed, 316 insertions(+), 20 deletions(-) create mode 100644 hw/vfio/cpr-iommufd.c -- 1.8.3.1
