Hey Yi, > On 9 Nov 2018, at 06:54, Liu, Yi L <yi.l....@intel.com> wrote: > > Hi James, > > Regards to the relationship of iommu group and domain, the blog written by > Alex > may help you. The blog explained very well on how iommu group is determined > and > why. > > http://vfio.blogspot.com/2014/08/iommu-groups-inside-and-out.html
Thanks for the link, this explains how I expected the group domain relationship to work. > >> From: iommu-boun...@lists.linux-foundation.org [mailto:iommu- >> boun...@lists.linux-foundation.org] On Behalf Of James Sewart via iommu >> Sent: Thursday, November 8, 2018 7:30 PM >> Subject: Re: [RFC] iommu/vt-d: Group and domain relationship >> >> Hey, >> >>> On 8 Nov 2018, at 01:42, Lu Baolu <baolu...@linux.intel.com> wrote: >>> >>> Hi, >>> >>> On 11/8/18 1:55 AM, James Sewart wrote: >>>> Hey, >>>>> On 7 Nov 2018, at 02:10, Lu Baolu <baolu...@linux.intel.com> wrote: >>>>> >>>>> Hi, >>>>> >>>>> On 11/6/18 6:40 PM, James Sewart wrote: >>>>>> Hey Lu, >>>>>> Would you be able to go into more detail about the issues with > > [...] > >>>>> >>>>> Why do we want to open this door? Probably we want the generic iommu >>>>> layer to handle these things (it's called default domain). >>>> I’d like to allocate a domain and attach it to multiple devices in a >>>> group/multiple groups so that they share address translation, but still >>>> allow drivers for devices in those groups to use the dma_map_ops api. >>> >>> Just out of curiosity, why do you want to share a single domain across >>> multiple groups? By default, the groups and DMA domains are normally >>> 1-1 mapped, right? >> >> Currently we see each device in a group with their own domain. >> find_or_alloc_domain looks at dma aliases to determine who shares domains >> whereas pci_device_group in the generic iommu code determines groups using >> a few other checks. We have observed internally that devices under a pcie >> switch will be put in the same group but they do not share a domain within >> that group. > > Really? iommu group is DMA isolation unit. You said they are not sharing a > domain. > Do you mean they have different IOVA address space by mentioning they are not > sharing a domain? Is there any special things(e.g. special IOVA allocation to > avoid > unexpected P2P) done on your system? Normally, devices within an iommu group > should share an IOVA address space. We see a unique set of mappings for each device within a group. Adding debug output to the kernel shows find_or_alloc_domain allocating a new domain for each device in a group. We don’t have any special configuration with regard to IOVA allocation. I think the issue is that find_or_alloc_domain only uses pci_for_each_dma_alias to determine domain sharing. This doesn’t check the configuration of upstream pcie-pcie switches and doesn’t look to see if ACS is enabled or disabled. pci_device_group in drivers/iommu/iommu.c, which is used to determine which group a device is assigned, performs an extra walk of the upstream buses to see if there exists a bus without ACS which would allow peer-to-peer dma and groups the device accordingly. One solution would be to allow the generic iommu code to assign domains, which it does based on the device_group function of the iommu_ops. The reason it doesn’t currently is that the default domain type is IOMMU_DOMAIN_DMA which the intel driver currently disallows allocating via the domain_alloc api. Cheers, James. _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
