>-----Original Message----- >From: Joao Martins <joao.m.mart...@oracle.com> >Subject: [PATCH v4 00/12] hw/iommufd: IOMMUFD Dirty Tracking > >This small series adds support for IOMMU dirty tracking support via the >IOMMUFD backend. The hardware capability is available on most recent x86 >hardware. The series is divided organized as follows: > >* Patch 1-2: Fixes a regression into mdev support with IOMMUFD. This > one is independent of the series but happened to cross it > while testing mdev with this series
I guess VFIO ap/ccw may need fixes too. Will you help on that or I can take it if you want to focus on dirty tracking. The fix may be trivial, just assign VFIODevice->mdev = true. Thanks Zhenzhong > >* Patch 3: Adds a support to iommufd_get_device_info() for capabilities > >* Patches 4 - 10: IOMMUFD backend support for dirty tracking; > >Introduce auto domains -- Patch 5 goes into more detail, but the gist is that >we will find and attach a device to a compatible IOMMU domain, or allocate >a new >hardware pagetable *or* rely on kernel IOAS attach (for mdevs). Afterwards >the >workflow is relatively simple: > >1) Probe device and allow dirty tracking in the HWPT >2) Toggling dirty tracking on/off >3) Read-and-clear of Dirty IOVAs > >The heuristics selected for (1) were to always request the HWPT for >dirty tracking if supported, or rely on device dirty page tracking. This >is a little simplistic and we aren't necessarily utilizing IOMMU dirty >tracking even if we ask during hwpt allocation. > >The unmap case is deferred until further vIOMMU support with migration >is added[3] which will then introduce the usage of >IOMMU_HWPT_GET_DIRTY_BITMAP_NO_CLEAR in GET_DIRTY_BITMAP ioctl >in the >dma unmap bitmap flow. > >* Patches 11-12: Don't block live migration where there's no VF dirty >tracker, considering that we have IOMMU dirty tracking. > >Comments and feedback appreciated. Thanks for all the review thus far! > >Cheers, > Joao > >P.S. Suggest linux-next (or future v6.11) as hypervisor kernel as there's >some bugs fixed there with regards to IOMMU hugepage dirty tracking. > >Changes since v3[5]: >* Skip HostIOMMUDevice::realize for mdev, and introduce a helper to check >if the VFIO > device is mdev. (Zhenzhong) >* Skip setting IOMMU device for mdev (Zhenzhong) >* Add Zhenzhong review tag in patch 3 >* Utilize vbasedev::bcontainer::dirty_pages_supported instead of >introducing > a new HostIOMMUDevice capability and thus remove the cap patch from >the series (Zhenzhong) >* Move the HostIOMMUDevice::realize() to be part of VFIODevice >initialization in attach_device() >while skipping it all together for mdev. (Cedric) >* Due to the previous item, had to remove aw_bits because it depends on >device attach being >finished, instead defer it to when get_cap() gets called. >* Skip auto domains for mdev instead of purposedly erroring out >(Zhenzhong) >* Pass errp in all cases, and instead just free the error in case of -EINVAL > in most of all patches, and also pass Error* in >iommufd_backend_alloc_hwpt() amd > set/query dirty. This is made better thanks in part to skipping auto domains >for mdev (Cedric) > >Changes since RFCv2[4]: >* Always allocate hwpt with IOMMU_HWPT_ALLOC_DIRTY_TRACKING even >if >we end up not actually toggling dirty tracking. (Avihai) >* Fix error handling widely in auto domains logic and all patches (Avihai) >* Reuse iommufd_backend_get_device_info() for capabilities (Zhenzhong) >* New patches 1 and 2 taking into consideration previous comments. >* Store hwpt::flags to know if we have dirty tracking (Avihai) >* New patch 8, that allows to query dirty tracking support after >provisioning. This is a cleaner way to check IOMMU dirty tracking support >when vfio::migration is iniitalized, as opposed to RFCv2 via device caps. >device caps way is still used because at vfio attach we aren't yet with >a fully initialized migration state. >* Adopt error propagation in query,set dirty tracking >* Misc improvements overall broadly and Avihai >* Drop hugepages as it's a bit unrelated; I can pursue that patch >* separately. The main motivation is to provide a way to test >without hugepages similar to what vfio_type1_iommu.disable_hugepages=1 >does. > >Changes since RFCv1[2]: >* Remove intel/amd dirty tracking emulation enabling >* Remove the dirtyrate improvement for VF/IOMMU dirty tracking >[Will pursue these two in separate series] >* Introduce auto domains support >* Enforce dirty tracking following the IOMMUFD UAPI for this >* Add support for toggling hugepages in IOMMUFD >* Auto enable support when VF supports migration to use IOMMU >when it doesn't have VF dirty tracking >* Add a parameter to toggle VF dirty tracking > >[0] https://lore.kernel.org/qemu-devel/20240201072818.327930-1- >zhenzhong.d...@intel.com/ >[1] https://lore.kernel.org/qemu-devel/20240201072818.327930-10- >zhenzhong.d...@intel.com/ >[2] https://lore.kernel.org/qemu-devel/20220428211351.3897-1- >joao.m.mart...@oracle.com/ >[3] https://lore.kernel.org/qemu-devel/20230622214845.3980-1- >joao.m.mart...@oracle.com/ >[4] https://lore.kernel.org/qemu-devel/20240212135643.5858-1- >joao.m.mart...@oracle.com/ >[5] https://lore.kernel.org/qemu-devel/20240708143420.16953-1- >joao.m.mart...@oracle.com/ > >Joao Martins (12): > vfio/pci: Extract mdev check into an helper > vfio/iommufd: Don't initialize nor set a HOST_IOMMU_DEVICE with mdev > backends/iommufd: Extend iommufd_backend_get_device_info() to fetch >HW > capabilities > vfio/iommufd: Return errno in iommufd_cdev_attach_ioas_hwpt() > vfio/iommufd: Introduce auto domain creation > vfio/{iommufd,container}: Remove caps::aw_bits > vfio/{iommufd,container}: Initialize HostIOMMUDeviceCaps during > attach_device() > vfio/iommufd: Probe and request hwpt dirty tracking capability > vfio/iommufd: Implement VFIOIOMMUClass::set_dirty_tracking support > vfio/iommufd: Implement VFIOIOMMUClass::query_dirty_bitmap support > vfio/migration: Don't block migration device dirty tracking is > unsupported > vfio/common: Allow disabling device dirty page tracking > > include/hw/vfio/vfio-common.h | 13 +++ > include/sysemu/host_iommu_device.h | 2 +- > include/sysemu/iommufd.h | 14 ++- > backends/iommufd.c | 89 ++++++++++++++- > hw/vfio/common.c | 17 +-- > hw/vfio/container.c | 11 +- > hw/vfio/helpers.c | 18 +++ > hw/vfio/iommufd.c | 178 ++++++++++++++++++++++++++++- > hw/vfio/migration.c | 4 +- > hw/vfio/pci.c | 22 ++-- > backends/trace-events | 3 + > 11 files changed, 339 insertions(+), 32 deletions(-) > >-- >2.17.2
