Hi Robin, On 5/8/19 3:59 PM, Robin Murphy wrote: > On 08/04/2019 13:18, Eric Auger wrote: >> On ARM, MSI are translated by the SMMU. An IOVA is allocated >> for each MSI doorbell. If both the host and the guest are exposed >> with SMMUs, we end up with 2 different IOVAs allocated by each. >> guest allocates an IOVA (gIOVA) to map onto the guest MSI >> doorbell (gDB). The Host allocates another IOVA (hIOVA) to map >> onto the physical doorbell (hDB). >> >> So we end up with 2 untied mappings: >> S1 S2 >> gIOVA -> gDB >> hIOVA -> hDB >> >> Currently the PCI device is programmed by the host with hIOVA >> as MSI doorbell. So this does not work. >> >> This patch introduces an API to pass gIOVA/gDB to the host so >> that gIOVA can be reused by the host instead of re-allocating >> a new IOVA. So the goal is to create the following nested mapping: >> >> S1 S2 >> gIOVA -> gDB -> hDB >> >> and program the PCI device with gIOVA MSI doorbell. >> >> In case we have several devices attached to this nested domain >> (devices belonging to the same group), they cannot be isolated >> on guest side either. So they should also end up in the same domain >> on guest side. We will enforce that all the devices attached to >> the host iommu domain use the same physical doorbell and similarly >> a single virtual doorbell mapping gets registered (1 single >> virtual doorbell is used on guest as well). >> >> Signed-off-by: Eric Auger <eric.au...@redhat.com> >> >> --- >> v6 -> v7: >> - remove the device handle parameter. >> - Add comments saying there can only be a single MSI binding >> registered per iommu_domain >> v5 -> v6: >> -fix compile issue when IOMMU_API is not set >> >> v3 -> v4: >> - add unbind >> >> v2 -> v3: >> - add a struct device handle >> --- >> drivers/iommu/iommu.c | 37 +++++++++++++++++++++++++++++++++++++ >> include/linux/iommu.h | 23 +++++++++++++++++++++++ >> 2 files changed, 60 insertions(+) >> >> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c >> index 6d6cb4005ca5..0d160bbd6f81 100644 >> --- a/drivers/iommu/iommu.c >> +++ b/drivers/iommu/iommu.c >> @@ -1575,6 +1575,43 @@ static void __iommu_detach_device(struct >> iommu_domain *domain, >> trace_detach_device_from_domain(dev); >> } >> +/** >> + * iommu_bind_guest_msi - Passes the stage1 GIOVA/GPA mapping of a >> + * virtual doorbell >> + * >> + * @domain: iommu domain the stage 1 mapping will be attached to >> + * @iova: iova allocated by the guest >> + * @gpa: guest physical address of the virtual doorbell >> + * @size: granule size used for the mapping >> + * >> + * The associated IOVA can be reused by the host to create a nested >> + * stage2 binding mapping translating into the physical doorbell used >> + * by the devices attached to the domain. >> + * >> + * All devices within the domain must share the same physical doorbell. >> + * A single MSI GIOVA/GPA mapping can be attached to an iommu_domain. >> + */ >> + >> +int iommu_bind_guest_msi(struct iommu_domain *domain, >> + dma_addr_t giova, phys_addr_t gpa, size_t size) >> +{ >> + if (unlikely(!domain->ops->bind_guest_msi)) >> + return -ENODEV; >> + >> + return domain->ops->bind_guest_msi(domain, giova, gpa, size); >> +} >> +EXPORT_SYMBOL_GPL(iommu_bind_guest_msi); >> + >> +void iommu_unbind_guest_msi(struct iommu_domain *domain, >> + dma_addr_t iova) >> +{ >> + if (unlikely(!domain->ops->unbind_guest_msi)) >> + return; >> + >> + domain->ops->unbind_guest_msi(domain, iova); >> +} >> +EXPORT_SYMBOL_GPL(iommu_unbind_guest_msi); >> + >> void iommu_detach_device(struct iommu_domain *domain, struct device >> *dev) >> { >> struct iommu_group *group; >> diff --git a/include/linux/iommu.h b/include/linux/iommu.h >> index 7c7c6bad1420..a2f3f964ead2 100644 >> --- a/include/linux/iommu.h >> +++ b/include/linux/iommu.h >> @@ -192,6 +192,8 @@ struct iommu_resv_region { >> * @attach_pasid_table: attach a pasid table >> * @detach_pasid_table: detach the pasid table >> * @cache_invalidate: invalidate translation caches >> + * @bind_guest_msi: provides a stage1 giova/gpa MSI doorbell mapping >> + * @unbind_guest_msi: withdraw a stage1 giova/gpa MSI doorbell mapping >> * @pgsize_bitmap: bitmap of all possible supported page sizes >> */ >> struct iommu_ops { >> @@ -243,6 +245,10 @@ struct iommu_ops { >> int (*cache_invalidate)(struct iommu_domain *domain, struct >> device *dev, >> struct iommu_cache_invalidate_info *inv_info); >> + int (*bind_guest_msi)(struct iommu_domain *domain, >> + dma_addr_t giova, phys_addr_t gpa, size_t size); >> + void (*unbind_guest_msi)(struct iommu_domain *domain, dma_addr_t >> giova); >> + >> unsigned long pgsize_bitmap; >> }; >> @@ -356,6 +362,11 @@ extern void iommu_detach_pasid_table(struct >> iommu_domain *domain); >> extern int iommu_cache_invalidate(struct iommu_domain *domain, >> struct device *dev, >> struct iommu_cache_invalidate_info *inv_info); >> +extern int iommu_bind_guest_msi(struct iommu_domain *domain, >> + dma_addr_t giova, phys_addr_t gpa, size_t size); >> +extern void iommu_unbind_guest_msi(struct iommu_domain *domain, >> + dma_addr_t giova); >> + >> extern struct iommu_domain *iommu_get_domain_for_dev(struct device >> *dev); >> extern struct iommu_domain *iommu_get_dma_domain(struct device *dev); >> extern int iommu_map(struct iommu_domain *domain, unsigned long iova, >> @@ -812,6 +823,18 @@ iommu_cache_invalidate(struct iommu_domain *domain, >> return -ENODEV; >> } >> +static inline >> +int iommu_bind_guest_msi(struct iommu_domain *domain, >> + dma_addr_t giova, phys_addr_t gpa, size_t size) >> +{ >> + return -ENODEV; >> +} >> +static inline >> +int iommu_unbind_guest_msi(struct iommu_domain *domain, dma_addr_t >> giova) >> +{ >> + return -ENODEV; > > It's less of a problem than mismatching the other way round, but for > consistency this should return void like the real version. thank you for spotting this.
Eric > > Robin. > >> +} >> + >> #endif /* CONFIG_IOMMU_API */ >> #ifdef CONFIG_IOMMU_DEBUGFS >> _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu