Hi Shameer, On 6/28/22 10:14, Shameerali Kolothum Thodi wrote: > >> -----Original Message----- >> From: Yi Liu [mailto:yi.l....@intel.com] >> Sent: 18 May 2022 15:01 >> To: zhangfei....@foxmail.com; Jason Gunthorpe <j...@nvidia.com>; >> Zhangfei Gao <zhangfei....@linaro.org> >> Cc: eric.au...@redhat.com; Alex Williamson <alex.william...@redhat.com>; >> Shameerali Kolothum Thodi <shameerali.kolothum.th...@huawei.com>; >> coh...@redhat.com; qemu-devel@nongnu.org; >> da...@gibson.dropbear.id.au; th...@redhat.com; far...@linux.ibm.com; >> mjros...@linux.ibm.com; akrow...@linux.ibm.com; pa...@linux.ibm.com; >> jjhe...@linux.ibm.com; jasow...@redhat.com; k...@vger.kernel.org; >> nicol...@nvidia.com; eric.auger....@gmail.com; kevin.t...@intel.com; >> chao.p.p...@intel.com; yi.y....@intel.com; pet...@redhat.com >> Subject: Re: [RFC 00/18] vfio: Adopt iommufd >> >> On 2022/5/18 15:22, zhangfei....@foxmail.com wrote: >>> >>> On 2022/5/17 下午4:55, Yi Liu wrote: >>>> Hi Zhangfei, >>>> >>>> On 2022/5/12 17:01, zhangfei....@foxmail.com wrote: >>>>> Hi, Yi >>>>> >>>>> On 2022/5/11 下午10:17, zhangfei....@foxmail.com wrote: >>>>>> >>>>>> On 2022/5/10 下午10:08, Yi Liu wrote: >>>>>>> On 2022/5/10 20:45, Jason Gunthorpe wrote: >>>>>>>> On Tue, May 10, 2022 at 08:35:00PM +0800, Zhangfei Gao wrote: >>>>>>>>> Thanks Yi and Eric, >>>>>>>>> Then will wait for the updated iommufd kernel for the PCI MMIO >> region. >>>>>>>>> Another question, >>>>>>>>> How to get the iommu_domain in the ioctl. >>>>>>>> The ID of the iommu_domain (called the hwpt) it should be returned >> by >>>>>>>> the vfio attach ioctl. >>>>>>> yes, hwpt_id is returned by the vfio attach ioctl and recorded in >>>>>>> qemu. You can query page table related capabilities with this id. >>>>>>> >>>>>>> >> https://lore.kernel.org/kvm/20220414104710.28534-16-yi.l....@intel.com/ >>>>>> Thanks Yi, >>>>>> >>>>>> Do we use iommufd_hw_pagetable_from_id in kernel? >>>>>> >>>>>> The qemu send hwpt_id via ioctl. >>>>>> Currently VFIOIOMMUFDContainer has hwpt_list, >>>>>> Which member is good to save hwpt_id, IOMMUTLBEntry? >>>>> Can VFIOIOMMUFDContainer have multi hwpt? >>>> yes, it is possible >>> Then how to get hwpt_id in map/unmap_notify(IOMMUNotifier *n, >> IOMMUTLBEntry >>> *iotlb) >> in map/unmap, should use ioas_id instead of hwpt_id >> >>>>> Since VFIOIOMMUFDContainer has hwpt_list now. >>>>> If so, how to get specific hwpt from map/unmap_notify in hw/vfio/as.c, >>>>> where no vbasedev can be used for compare. >>>>> >>>>> I am testing with a workaround, adding VFIOIOASHwpt *hwpt in >>>>> VFIOIOMMUFDContainer. >>>>> And save hwpt when vfio_device_attach_container. >>>>> >>>>>> In kernel ioctl: iommufd_vfio_ioctl >>>>>> @dev: Device to get an iommu_domain for >>>>>> iommufd_hw_pagetable_from_id(struct iommufd_ctx *ictx, u32 pt_id, >>>>>> struct device *dev) >>>>>> But iommufd_vfio_ioctl seems no para dev? >>>>> We can set dev=Null since IOMMUFD_OBJ_HW_PAGETABLE does not >> need dev. >>>>> iommufd_hw_pagetable_from_id(ictx, hwpt_id, NULL) >>>> this is not good. dev is passed in to this function to allocate domain >>>> and also check sw_msi things. If you pass in a NULL, it may even unable >>>> to get a domain for the hwpt. It won't work I guess. >>> The iommufd_hw_pagetable_from_id can be used for >>> 1, allocate domain, which need para dev >>> case IOMMUFD_OBJ_IOAS >>> hwpt = iommufd_hw_pagetable_auto_get(ictx, ioas, dev); >> this is used when attaching ioas. >> >>> 2. Just return allocated domain via hwpt_id, which does not need dev. >>> case IOMMUFD_OBJ_HW_PAGETABLE: >>> return container_of(obj, struct iommufd_hw_pagetable, obj); >> yes, this would be the usage in nesting. you may check my below >> branch. It's for nesting integration. >> >> https://github.com/luxis1999/iommufd/tree/iommufd-v5.18-rc4-nesting >> >>> By the way, any plan of the nested mode? >> I'm working with Eric, Nic on it. Currently, I've got the above kernel >> branch, QEMU side is also WIP. > Hi Yi/Eric, > > I had a look at the above nesting kernel and Qemu branches and as mentioned > in the cover letter it is not working on ARM yet. > > IIUC, to get it working via the iommufd the main thing is we need a way to > configure > the phys SMMU in nested mode and setup the mappings for the stage 2. The > Cache/PASID related changes looks more straight forward. > > I had quite a few hacks to get it working on ARM, but still a WIP. So just > wondering > do you guys have something that can be shared yet?
I am working on the respin based on latest iommufd kernel branches and qemu RFC v2 but it is still WIP. I will share as soon as possible. Eric > > Please let me know. > > Thanks, > Shameer