RE: [PATCH V5 00/38] Live update: vfio and iommufd
Hi Cédric, >-Original Message- >From: Cédric Le Goater >Subject: Re: [PATCH V5 00/38] Live update: vfio and iommufd > >Zhenzhong, Eric, > >On 6/12/25 09:23, Cédric Le Goater wrote: >> On 6/11/25 16:39, Steven Sistare wrote: >>> On 6/11/2025 10:25 AM, Cédric Le Goater wrote: >>>> On 6/10/25 19:39, Cédric Le Goater wrote: >>>>>> Steve, >>>>>> >>>>>> For the next vfio PR, I plan to take patches 1-17 when patch 10 is >>>>>> updated. The rest is for later in this cycle >>>>> >>>>> Applied 1-17 to vfio-next. Waiting for an Ack from Michael. >>>> >>>> I am planing to send a PR with this first part to get more visibility. >>>> There is a slight risk of merging useless changes since CPR is not >>>> fully reviewed. My optimistic nature tells me it should reach QEMU 10.1 >>>> and we have time to adjust. >>>> >>>> Please feel free to intervene if you prefer the series to be fully >>>> approved/reviewed before merging. >>> >>> A partial merge is fine with me. >> >> >> Now merged. I am waiting for some feedback on the IOMMUFD part. > >Would you have some time in June to take a look ? Sorry, short of bandwidth this week, I'll review it next week. Thanks Zhenzhong
Re: [PATCH V5 00/38] Live update: vfio and iommufd
Zhenzhong, Eric, On 6/12/25 09:23, Cédric Le Goater wrote: On 6/11/25 16:39, Steven Sistare wrote: On 6/11/2025 10:25 AM, Cédric Le Goater wrote: On 6/10/25 19:39, Cédric Le Goater wrote: Steve, For the next vfio PR, I plan to take patches 1-17 when patch 10 is updated. The rest is for later in this cycle Applied 1-17 to vfio-next. Waiting for an Ack from Michael. I am planing to send a PR with this first part to get more visibility. There is a slight risk of merging useless changes since CPR is not fully reviewed. My optimistic nature tells me it should reach QEMU 10.1 and we have time to adjust. Please feel free to intervene if you prefer the series to be fully approved/reviewed before merging. A partial merge is fine with me. Now merged. I am waiting for some feedback on the IOMMUFD part. Would you have some time in June to take a look ? Thanks, C.
Re: [PATCH V5 00/38] Live update: vfio and iommufd
On 6/11/25 16:39, Steven Sistare wrote: On 6/11/2025 10:25 AM, Cédric Le Goater wrote: On 6/10/25 19:39, Cédric Le Goater wrote: Steve, For the next vfio PR, I plan to take patches 1-17 when patch 10 is updated. The rest is for later in this cycle Applied 1-17 to vfio-next. Waiting for an Ack from Michael. I am planing to send a PR with this first part to get more visibility. There is a slight risk of merging useless changes since CPR is not fully reviewed. My optimistic nature tells me it should reach QEMU 10.1 and we have time to adjust. Please feel free to intervene if you prefer the series to be fully approved/reviewed before merging. A partial merge is fine with me. Now merged. I am waiting for some feedback on the IOMMUFD part. Thanks, C.
Re: [PATCH V5 00/38] Live update: vfio and iommufd
On Wed, Jun 11, 2025 at 04:25:13PM +0200, Cédric Le Goater wrote: > Peter, Fabiano, > > The first 2 patches are migration patches. Do you agree if I take them > through the VFIO queue ? Yep, please go ahead. -- Peter Xu
Re: [PATCH V5 00/38] Live update: vfio and iommufd
On 6/11/2025 10:25 AM, Cédric Le Goater wrote: On 6/10/25 19:39, Cédric Le Goater wrote: Steve, For the next vfio PR, I plan to take patches 1-17 when patch 10 is updated. The rest is for later in this cycle Applied 1-17 to vfio-next. Waiting for an Ack from Michael. I am planing to send a PR with this first part to get more visibility. There is a slight risk of merging useless changes since CPR is not fully reviewed. My optimistic nature tells me it should reach QEMU 10.1 and we have time to adjust. Please feel free to intervene if you prefer the series to be fully approved/reviewed before merging. A partial merge is fine with me. - Steve Peter, Fabiano, The first 2 patches are migration patches. Do you agree if I take them through the VFIO queue ? Thanks, C.
Re: [PATCH V5 00/38] Live update: vfio and iommufd
On 6/10/25 19:39, Cédric Le Goater wrote: Steve, For the next vfio PR, I plan to take patches 1-17 when patch 10 is updated. The rest is for later in this cycle Applied 1-17 to vfio-next. Waiting for an Ack from Michael. I am planing to send a PR with this first part to get more visibility. There is a slight risk of merging useless changes since CPR is not fully reviewed. My optimistic nature tells me it should reach QEMU 10.1 and we have time to adjust. Please feel free to intervene if you prefer the series to be fully approved/reviewed before merging. Peter, Fabiano, The first 2 patches are migration patches. Do you agree if I take them through the VFIO queue ? Thanks, C.
Re: [PATCH V5 00/38] Live update: vfio and iommufd
Steve, For the next vfio PR, I plan to take patches 1-17 when patch 10 is updated. The rest is for later in this cycle Applied 1-17 to vfio-next. Waiting for an Ack from Michael. Thanks, C.
Re: [PATCH V5 00/38] Live update: vfio and iommufd
On 6/10/25 17:39, Steve Sistare wrote: Support vfio and iommufd devices with the cpr-transfer live migration mode. Devices that do not support live migration can still support cpr-transfer, allowing live update to a new version of QEMU on the same host, with no loss of guest connectivity. No user-visible interfaces are added. For legacy containers: Pass vfio device descriptors to new QEMU. In new QEMU, during vfio_realize, skip the ioctls that configure the device, because it is already configured. Use VFIO_DMA_UNMAP_FLAG_VADDR to abandon the old VA's for DMA mapped regions, and use VFIO_DMA_MAP_FLAG_VADDR to register the new VA in new QEMU and update the locked memory accounting. The physical pages remain pinned, because the descriptor of the device that locked them remains open, so DMA to those pages continues without interruption. Mediated devices are not supported, however, because they require the VA to always be valid, and there is a brief window where no VA is registered. Save the MSI message area as part of vfio-pci vmstate, and pass the interrupt and notifier eventfd's to new QEMU. New QEMU loads the MSI data, then the vfio-pci post_load handler finds the eventfds in CPR state, rebuilds vector data structures, and attaches the interrupts to the new KVM instance. This logic also applies to iommufd containers. For iommufd containers: Use IOMMU_IOAS_MAP_FILE to register memory regions for DMA when they are backed by a file (including a memfd), so DMA mappings do not depend on VA, which can differ after live update. This allows mediated devices to be supported. Pass the iommufd and vfio device descriptors from old to new QEMU. In new QEMU, during vfio_realize, skip the ioctls that configure the device, because it is already configured. In new QEMU, call ioctl(IOMMU_IOAS_CHANGE_PROCESS) to update mm ownership and locked memory accounting. Patches 3 to 8 are specific to legacy containers. Patches 21 to 36 are specific to iommufd containers. The remainder apply to both. Changes from previous versions: * V1 of this series contains minor changes from the "Live update: vfio" and "Live update: iommufd" series, mainly bug fixes and refactored patches. Changes in V2: * refactored various vfio code snippets into new cpr helpers * refactored vfio struct members into cpr-specific structures * refactored various small changes into their own patches * split complex patches. Notably: - split "refactor for cpr" into 5 patches - split "reconstruct device" into 4 patches * refactored vfio_connect_container using helpers and made its error recovery more robust. * moved vfio pci msi/vector/intx cpr functions to cpr.c * renamed "reused" to cpr_reused and cpr.reused * squashed vfio_cpr_[un]register_container to their call sites * simplified iommu_type setting after cpr * added cpr_open_fd and cpr_is_incoming helpers * removed changes from vfio_legacy_dma_map, and instead temporarily override dma_map and dma_unmap ops. * deleted error_report and returned Error to callers where possible. * simplified the memory_get_xlat_addr interface * fixed flags passed to iommufd_backend_alloc_hwpt * defined MIG_PRI_UNINITIALIZED * added maintainers Changes in V3: * removed cleanup patches that were already pulled * rebased to latest master Changes in V4: * added SPDX-License-Identifier * patch "vfio/container: preserve descriptors" - rewrote search loop in vfio_container_connect - do not return pfd from vfio_cpr_container_match - add helper for VFIO_GROUP_GET_DEVICE_FD * deleted patch "export vfio_legacy_dma_map" * patch "vfio/container: restore DMA vaddr" - deleted redundant error_report from vfio_legacy_cpr_dma_map - save old dma_map function * patch "vfio-pci: skip reset during cpr" - use cpr_is_incoming instead of cpr_reused * renamed err -> local_err in all new code * patch "export MSI functions" - renamed with vfio_pci prefix, and defined wrappers for low level routines instead of exporting them. * patch "close kvm after cpr" - fixed build error for !CONFIG_KVM * added the cpr_resave_fd helper * dropped patch "pass ramblock to vfio_container_dma_map", relying on "pass MemoryRegion" from the vfio-user series instead. * deleted "reused" variables, replaced with cpr_is_incoming() * renamed cpr_needed_for_reuse -> cpr_incoming_needed * rewrote patch "pci: skip reset during cpr" * rebased to latest master for iommufd: * deleted redundant error_report from iommufd_backend_map_file_dma * added interface doc for dma_map_file * check return value of cpr_open_fd * deleted "export iommufd_cdev_get_info_iova_range" * deleted "reconstruct device" * deleted "reconstruct hw_caps" * deleted "define hwpt constructors" * separated cpr registration for iommufd be and vfio container * correctly attach to
