Re: [vfio-users] Is there some method to merge 2 iommu group if I disable ACS
On Tue, 8 Aug 2017 06:58:40 + "Wuzongyong (Euler Dept)"wrote: > > On Mon, 7 Aug 2017 06:43:30 -0600 > > Alex Williamson wrote: > > > > > On Aug 7, 2017 5:39 AM, "Wuzongyong (Euler Dept)" > > > > > > wrote: > > > > > > Hi, > > > > > > > > > > > > Assume that an endpoint device(called ep1) belongs to iommu group 1, > > > and another endpoint device(called ep2) belongs to iommu group 2. > > > > > > Moreover, these two devices locate in different downstream ports of > > > the same switch respectively. If I disable the ACS of these downstream > > > > > > ports, we know ep1 and ep2 should locate in the same iommu group. So > > > the question is if I can regenerate a iommu group to let ep1 and ep2 > > > > > > locate in the same iommu group and let ep1 and ep2 can’t be assigned > > > to two different VMs without host rebooting? > > > > > > Wow, complete formatting fail from gmail on my phone. Sorry for that. > > Question is: > > > > > Why would you want to be able to do this? > > > > Thanks, > > > > Alex > > I want to do peer-to-peer communication between the two pci-e eps. A significant problem here is the I/O virtual address (IOVA) space in which the device operates when used with vfio. Peer-to-peer will operate that the physical address level while userspace drivers like QEMU will operate with IOVAs. The proper why to get both is for the endpoint to support ATS and ACS on the downstream switch port to support Direct Translation. If these two things were both enabled, then a transaction from the endpoint with the address translation field set would be able to do direct p2p with another device while non-translated transactions would go through the IOMMU. Therefore, I don't think what you're about to try is going to work and you're on your own for support. That said, you can manipulate the ACS control bits on the downstream switch ports with setpci to disable ACS. You might then be able to use the "remove" sysfs entry for the PCI endpoint and the /sys/bus/pci/rescan entry to re-add the device, which would hopefully detect the isolation issue and group the endpoints together. You might still run into problems that the downstream ports are already in separate groups and this won't work without also removing and rescanning the downstream port itself, but this risks that ACS will be re-enabled on that port. IOW, there are some hacks you can try, but this is not an encouraged or supported procedure and don't expect support for it. Thanks, Alex ___ vfio-users mailing list vfio-users@redhat.com https://www.redhat.com/mailman/listinfo/vfio-users
[vfio-users] Is there some method to merge 2 iommu group if I disable ACS ?
Hi, Assume that an endpoint device(called ep1) belongs to iommu group 1, and another endpoint device(called ep2) belongs to iommu group 2. Moreover, these two devices locate in different downstream ports of the same switch respectively. If I disable the ACS of these downstream ports, we know ep1 and ep2 should locate in the same iommu group. So the question is if I can regenerate a iommu group to let ep1 and ep2 locate in the same iommu group and let ep1 and ep2 can't be assigned to two different VMs without host rebooting? Thanks, Cordius Wu ___ vfio-users mailing list vfio-users@redhat.com https://www.redhat.com/mailman/listinfo/vfio-users
Re: [vfio-users] Is there some method to merge 2 iommu group if I disable ACS
On Mon, 7 Aug 2017 06:43:30 -0600 Alex Williamsonwrote: > On Aug 7, 2017 5:39 AM, "Wuzongyong (Euler Dept)" > wrote: > > Hi, > > > > Assume that an endpoint device(called ep1) belongs to iommu group 1, and > another endpoint device(called ep2) belongs to iommu group 2. > > Moreover, these two devices locate in different downstream ports of the > same switch respectively. If I disable the ACS of these downstream > > ports, we know ep1 and ep2 should locate in the same iommu group. So the > question is if I can regenerate a iommu group to let ep1 and ep2 > > locate in the same iommu group and let ep1 and ep2 can’t be assigned to > two different VMs without host rebooting? Wow, complete formatting fail from gmail on my phone. Sorry for that. Question is: > Why would you want to be able to do this? Thanks, Alex ___ vfio-users mailing list vfio-users@redhat.com https://www.redhat.com/mailman/listinfo/vfio-users
Re: [vfio-users] Is there some method to merge 2 iommu group if I disable ACS
On Aug 7, 2017 5:39 AM, "Wuzongyong (Euler Dept)"wrote: Hi, Assume that an endpoint device(called ep1) belongs to iommu group 1, and another endpoint device(called ep2) belongs to iommu group 2. Moreover, these two devices locate in different downstream ports of the same switch respectively. If I disable the ACS of these downstream ports, we know ep1 and ep2 should locate in the same iommu group. So the question is if I can regenerate a iommu group to let ep1 and ep2 locate in the same iommu group and let ep1 and ep2 can’t be assigned to two different VMs without host rebooting? Why would you want to be able to do this? ___ vfio-users mailing list vfio-users@redhat.com https://www.redhat.com/mailman/listinfo/vfio-users