>-----Original Message----- >From: Joao Martins <joao.m.mart...@oracle.com> >Subject: Re: [PATCH v4 05/12] vfio/iommufd: Introduce auto domain >creation > >On 18/07/2024 08:44, Duan, Zhenzhong wrote: >>>>>> If existing hwpt doesn't support dirty tracking. >>>>>> Another device supporting dirty tracking attaches to that hwpt, what >>> will >>>>> happen? >>>>>> >>>>> >>>>> Hmm, It succeeds as there's no incompatbility. At the very least I plan >on >>>>> blocking migration if the device neither has VF dirty tracking, nor >IOMMU >>>>> dirty >>>>> tracking (and patch 11 needs to be adjusted to check hwpt_flags >instead >>> of >>>>> container). >>>> >>>> When bcontainer->dirty_pages_supported is true, I think that container >>> should only contains hwpt list that support dirty tracking. All hwpt not >>> supporting dirty tracking should be in other container. >>>> >>> Well but we are adopting this auto domains scheme and works for any >>> device, >>> dirty tracking or not. We already track hwpt flags so we know which ones >>> support >>> dirty tracking. This differentiation would (IMHO) complicate more and I >am >>> not >>> sure the gain >> >> OK, I was trying to make bcontainer->dirty_pages_supported accurate >because it is used in many functions such as vfio_get_dirty_bitmap() which >require an accurate value. If there is mix of hwpt in that container, that's >impossible. >> >> But as you say you want to address the mix issue in a follow-up and >presume all are homogeneous hw for now, then OK, there is no conflict. >> > >Right > >>> >>>> If device supports dirty tracking, it should bypass attaching container >that >>> doesn't support dirty tracking. Vise versa. >>>> This way we can support the mixing environment. >>>> >>> >>> It's not that easy as the whole flow doesn't handle this mixed mode (even >>> excluding this series). We would to have device-dirty-tracking start all >>> non-disabled device trackers first [and stop them as well], and then we >>> would >>> always iterate those first (if device dirty trackers are active), and then >defer >>> to IOMMU tracker for those who don't. >> >> Why is device-dirty-tracking preferred over IOMMU dirty tracking? >> Imagine if many devices attached to same domain. >> > >The heuristic or expectation is that device dirty tracking doesn't involve a >compromise for SW because it can a) perform lowest granularity of IOVA >range >being dirty with b) no DMA penalty. With IOMMU though, SW needs to >worry about >managing page tables to dictate the granularity and those take time to walk >the >deeper the level we descend into. I used to think that IOMMU we have DMA >penalty >(because of the IOTLB flushes to clear dirty bit, and IOTLB cache misses) but I >haven't yet that materialized in the field yet (at least for 100Gbit/s rates). > >TL;DR At the end of the day with device dirty tracking you have less to worry >about, and it's the VF doing most of the heavy lifting. In theory with device >dirty tracking you could even perform sub basepage tracking if the device >allows >it to do so.
Clear, thanks Joao. BRs. Zhenzhong
