On Thu, Jul 08, 2021 at 03:43:42PM +0100, Robin Murphy wrote: > On 2021-07-08 14:57, Kai-Heng Feng wrote: > > On Thu, Jul 8, 2021 at 6:18 PM Robin Murphy <robin.mur...@arm.com> wrote: > > > > > > On 2021-07-08 10:28, Joerg Roedel wrote: > > > > On Thu, Jul 08, 2021 at 03:42:32PM +0800, Kai-Heng Feng wrote: > > > > > @@ -344,6 +344,9 @@ static int iommu_init_device(struct device *dev) > > > > > > > > > > iommu = amd_iommu_rlookup_table[dev_data->devid]; > > > > > dev_data->iommu_v2 = iommu->is_iommu_v2; > > > > > + > > > > > + if (dev_data->iommu_v2) > > > > > + swiotlb = 1; > > > > > > > > This looks like the big hammer, as it will affect all other systems > > > > where the AMD GPUs are in their own group. > > > > > > > > What is needed here is an explicit check whether a non-iommu-v2 device > > > > is direct-mapped because it shares a group with the GPU, and only enable > > > > swiotlb in this case. > > > > > > Right, it's basically about whether any DMA-limited device might at any > > > time end up in an IOMMU_DOMAIN_IDENTITY domain. And given the > > > possibility of device hotplug and the user being silly with the sysfs > > > interface, I don't think we can categorically determine that at boot time. > > > > > > Also note that Intel systems are likely to be similarly affected (in > > > fact intel-iommu doesn't even have the iommu_default_passthough() check > > > so it's probably even easier to blow up). > > > > swiotlb is enabled by pci_swiotlb_detect_4gb() and intel-iommu doesn't > > disable it. > > Oh, right... I did say I found this dance hard to follow. Clearly I > shouldn't have trusted what I thought I remembered from looking at it > yesterday :) > > Also not helped by the fact that it sets iommu_detected which *does* disable > SWIOTLB, but only on IA-64. > > > I wonder if we can take the same approach in amd-iommu? > > Certainly if there's a precedent for leaving SWIOTLB enabled even if it > *might* be redundant, that seems like the easiest option (it's what we do on > arm64 too, but then we have system topologies where some devices may not be > behind IOMMUs even when others are). More fun would be to try to bring it up > at the first sign of IOMMU_DOMAIN_IDENTITY if it was disabled previously, > but I don't have the highest hope of that being practical.
<scratches his head> It is kind of silly to enable SWIOTLB which will just eat 64MB of memory "just in case". The SWIOTLB does have support to do late initialization (xen-pcifront does that for example - so if you add devices that can't do 64-bit it will allocate something like 4MB). Would that be a better choice going forward - that is allocate this under those conditions? > > Robin. _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu