Re: [PATCH V2 09/10] iommu/ipmmu-vmsa: Use refcount for the micro-TLBs
On 27.01.22 13:48, Julien Grall wrote:
Hi Oleksandr,
Hi Julien
On 20/12/2021 21:15, Oleksandr Tyshchenko wrote:
From: Oleksandr Tyshchenko
Reference-count the micro-TLBs as several bus masters can be
connected to the same micro-TLB (and drop TODO comment).
This wasn't an issue so far, since the platform devices
(this driver deals with) get assigned/deassigned together during
domain creation/destruction. But, in order to support PCI devices
(which are hot-pluggable) in the near future we will need to
take care of.
Looking at the code, it is not possible to share the micro-TLB between
domains (or even Xen). So technically, we will still want to {,
un}hotplug the devices using the same u-TLB together.
Therefore, I would clarify that this is necessary because even if we
have to remove all the devices together, the IOMMU driver will be
de-assigning them one-by-one.
I would add a similar comment in the code as well.
ok, will add.
Signed-off-by: Oleksandr Tyshchenko
Reviewed-by: Yoshihiro Shimoda
---
Changes V1 -> V2:
- add R-b
- add ASSERT() in ipmmu_utlb_disable()
---
xen/drivers/passthrough/arm/ipmmu-vmsa.c | 19 +++
1 file changed, 11 insertions(+), 8 deletions(-)
diff --git a/xen/drivers/passthrough/arm/ipmmu-vmsa.c
b/xen/drivers/passthrough/arm/ipmmu-vmsa.c
index 649b9f6..1224ea4 100644
--- a/xen/drivers/passthrough/arm/ipmmu-vmsa.c
+++ b/xen/drivers/passthrough/arm/ipmmu-vmsa.c
@@ -127,6 +127,7 @@ struct ipmmu_vmsa_device {
spinlock_t lock; /* Protects ctx and domains[] */
DECLARE_BITMAP(ctx, IPMMU_CTX_MAX);
struct ipmmu_vmsa_domain *domains[IPMMU_CTX_MAX];
+ unsigned int utlb_refcount[IPMMU_UTLB_MAX];
const struct ipmmu_features *features;
};
@@ -467,13 +468,12 @@ static int ipmmu_utlb_enable(struct
ipmmu_vmsa_domain *domain,
}
}
- /*
- * TODO: Reference-count the micro-TLB as several bus masters
can be
- * connected to the same micro-TLB.
- */
- ipmmu_imuasid_write(mmu, utlb, 0);
- ipmmu_imuctr_write(mmu, utlb, imuctr |
- IMUCTR_TTSEL_MMU(domain->context_id) |
IMUCTR_MMUEN);
+ if ( mmu->utlb_refcount[utlb]++ == 0 )
+ {
+ ipmmu_imuasid_write(mmu, utlb, 0);
+ ipmmu_imuctr_write(mmu, utlb, imuctr |
+ IMUCTR_TTSEL_MMU(domain->context_id) | IMUCTR_MMUEN);
+ }
return 0;
}
@@ -484,7 +484,10 @@ static void ipmmu_utlb_disable(struct
ipmmu_vmsa_domain *domain,
{
struct ipmmu_vmsa_device *mmu = domain->mmu;
- ipmmu_imuctr_write(mmu, utlb, 0);
+ ASSERT(mmu->utlb_refcount[utlb] > 0);
+
+ if ( --mmu->utlb_refcount[utlb] == 0 )
+ ipmmu_imuctr_write(mmu, utlb, 0);
}
/* Domain/Context Management */
Cheers,
--
Regards,
Oleksandr Tyshchenko
Re: [PATCH V2 09/10] iommu/ipmmu-vmsa: Use refcount for the micro-TLBs
Hi Oleksandr,
On 20/12/2021 21:15, Oleksandr Tyshchenko wrote:
From: Oleksandr Tyshchenko
Reference-count the micro-TLBs as several bus masters can be
connected to the same micro-TLB (and drop TODO comment).
This wasn't an issue so far, since the platform devices
(this driver deals with) get assigned/deassigned together during
domain creation/destruction. But, in order to support PCI devices
(which are hot-pluggable) in the near future we will need to
take care of.
Looking at the code, it is not possible to share the micro-TLB between
domains (or even Xen). So technically, we will still want to {,
un}hotplug the devices using the same u-TLB together.
Therefore, I would clarify that this is necessary because even if we
have to remove all the devices together, the IOMMU driver will be
de-assigning them one-by-one.
I would add a similar comment in the code as well.
Signed-off-by: Oleksandr Tyshchenko
Reviewed-by: Yoshihiro Shimoda
---
Changes V1 -> V2:
- add R-b
- add ASSERT() in ipmmu_utlb_disable()
---
xen/drivers/passthrough/arm/ipmmu-vmsa.c | 19 +++
1 file changed, 11 insertions(+), 8 deletions(-)
diff --git a/xen/drivers/passthrough/arm/ipmmu-vmsa.c
b/xen/drivers/passthrough/arm/ipmmu-vmsa.c
index 649b9f6..1224ea4 100644
--- a/xen/drivers/passthrough/arm/ipmmu-vmsa.c
+++ b/xen/drivers/passthrough/arm/ipmmu-vmsa.c
@@ -127,6 +127,7 @@ struct ipmmu_vmsa_device {
spinlock_t lock;/* Protects ctx and domains[] */
DECLARE_BITMAP(ctx, IPMMU_CTX_MAX);
struct ipmmu_vmsa_domain *domains[IPMMU_CTX_MAX];
+unsigned int utlb_refcount[IPMMU_UTLB_MAX];
const struct ipmmu_features *features;
};
@@ -467,13 +468,12 @@ static int ipmmu_utlb_enable(struct ipmmu_vmsa_domain *domain,
}
}
-/*
- * TODO: Reference-count the micro-TLB as several bus masters can be
- * connected to the same micro-TLB.
- */
-ipmmu_imuasid_write(mmu, utlb, 0);
-ipmmu_imuctr_write(mmu, utlb, imuctr |
- IMUCTR_TTSEL_MMU(domain->context_id) | IMUCTR_MMUEN);
+if ( mmu->utlb_refcount[utlb]++ == 0 )
+{
+ipmmu_imuasid_write(mmu, utlb, 0);
+ipmmu_imuctr_write(mmu, utlb, imuctr |
+ IMUCTR_TTSEL_MMU(domain->context_id) |
IMUCTR_MMUEN);
+}
return 0;
}
@@ -484,7 +484,10 @@ static void ipmmu_utlb_disable(struct ipmmu_vmsa_domain
*domain,
{
struct ipmmu_vmsa_device *mmu = domain->mmu;
-ipmmu_imuctr_write(mmu, utlb, 0);
+ASSERT(mmu->utlb_refcount[utlb] > 0);
+
+if ( --mmu->utlb_refcount[utlb] == 0 )
+ipmmu_imuctr_write(mmu, utlb, 0);
}
/* Domain/Context Management */
Cheers,
--
Julien Grall
