Re: [Intel-gfx] [PATCH 1/4] iommu/vt-d: Disable superpage for Geminilake igfx
On 7/14/21 4:30 AM, Ville Syrjälä wrote:
On Tue, Jul 13, 2021 at 09:34:09AM +0800, Lu Baolu wrote:
On 7/12/21 11:47 PM, Ville Syrjälä wrote:
On Mon, Jul 12, 2021 at 07:23:07AM +0800, Lu Baolu wrote:
On 7/10/21 12:47 AM, Ville Syrjala wrote:
From: Ville Syrjälä
While running "gem_exec_big --r single" from igt-gpu-tools on
Geminilake as soon as a 2M mapping is made I tend to get a DMAR
write fault. Strangely the faulting address is always a 4K page
and usually very far away from the 2M page that got mapped.
But if no 2M mappings get used I can't reproduce the fault.
I also tried to dump the PTE for the faulting address but it actually
looks correct to me (ie. definitely seems to have the write bit set):
DMAR: DRHD: handling fault status reg 2
DMAR: [DMA Write] Request device [00:02.0] PASID fault addr
7fa8a78000 [fault reason 05] PTE Write access is not set
DMAR: fault 7fa8a78000 (level=1) PTE = 149efc003
So not really sure what's going on and this might just be full on duct
tape, but it seems to work here. The machine has now survived a whole day
running that test whereas with superpage enabled it fails in less than
a minute usually.
TODO: might be nice to disable superpage only for the igfx iommu
instead of both iommus
If all these quirks are about igfx dedicated iommu's, I would suggest to
disable superpage only for the igfx ones.
Sure. Unfortunately there's no convenient mechanism to do that in
the iommu driver that I can immediately see. So not something I
can just whip up easily. Since you're actually familiar with the
driver maybe you can come up with a decent solution for that?
How about something like below? [no compile, no test...]
diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index 1131b8efb050..2d51ef288a9e 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -338,6 +338,7 @@ static int intel_iommu_strict;
static int intel_iommu_superpage = 1;
static int iommu_identity_mapping;
static int iommu_skip_te_disable;
+static int iommu_skip_igfx_superpage;
#define IDENTMAP_GFX 2
#define IDENTMAP_AZALIA 4
@@ -652,6 +653,27 @@ static bool domain_update_iommu_snooping(struct
intel_iommu *skip)
return ret;
}
+static bool domain_use_super_page(struct dmar_domain *domain)
+{
+ struct dmar_drhd_unit *drhd;
+ struct intel_iommu *iommu;
+ bool ret = true;
+
+ if (!intel_iommu_superpage)
+ return false;
+
+ rcu_read_lock();
+ for_each_active_iommu(iommu, drhd) {
+ if (drhd->gfx_dedicated && iommu_skip_igfx_superpage) {
+ ret = false;
+ break
^
Missing semicolon. Othwerwise seems to work great here. Thanks.
Are you going to turn this into a proper patch, or do you
want me to just squash this into my patches and repost?
Please go ahead with a new version.
Best regards,
baolu
___
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [Intel-gfx] [PATCH 1/4] iommu/vt-d: Disable superpage for Geminilake igfx
On Tue, Jul 13, 2021 at 09:34:09AM +0800, Lu Baolu wrote:
> On 7/12/21 11:47 PM, Ville Syrjälä wrote:
> > On Mon, Jul 12, 2021 at 07:23:07AM +0800, Lu Baolu wrote:
> >> On 7/10/21 12:47 AM, Ville Syrjala wrote:
> >>> From: Ville Syrjälä
> >>>
> >>> While running "gem_exec_big --r single" from igt-gpu-tools on
> >>> Geminilake as soon as a 2M mapping is made I tend to get a DMAR
> >>> write fault. Strangely the faulting address is always a 4K page
> >>> and usually very far away from the 2M page that got mapped.
> >>> But if no 2M mappings get used I can't reproduce the fault.
> >>>
> >>> I also tried to dump the PTE for the faulting address but it actually
> >>> looks correct to me (ie. definitely seems to have the write bit set):
> >>>DMAR: DRHD: handling fault status reg 2
> >>>DMAR: [DMA Write] Request device [00:02.0] PASID fault addr
> >>> 7fa8a78000 [fault reason 05] PTE Write access is not set
> >>>DMAR: fault 7fa8a78000 (level=1) PTE = 149efc003
> >>>
> >>> So not really sure what's going on and this might just be full on duct
> >>> tape, but it seems to work here. The machine has now survived a whole day
> >>> running that test whereas with superpage enabled it fails in less than
> >>> a minute usually.
> >>>
> >>> TODO: might be nice to disable superpage only for the igfx iommu
> >>> instead of both iommus
> >> If all these quirks are about igfx dedicated iommu's, I would suggest to
> >> disable superpage only for the igfx ones.
> > Sure. Unfortunately there's no convenient mechanism to do that in
> > the iommu driver that I can immediately see. So not something I
> > can just whip up easily. Since you're actually familiar with the
> > driver maybe you can come up with a decent solution for that?
> >
>
> How about something like below? [no compile, no test...]
>
> diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
> index 1131b8efb050..2d51ef288a9e 100644
> --- a/drivers/iommu/intel/iommu.c
> +++ b/drivers/iommu/intel/iommu.c
> @@ -338,6 +338,7 @@ static int intel_iommu_strict;
> static int intel_iommu_superpage = 1;
> static int iommu_identity_mapping;
> static int iommu_skip_te_disable;
> +static int iommu_skip_igfx_superpage;
>
> #define IDENTMAP_GFX2
> #define IDENTMAP_AZALIA 4
> @@ -652,6 +653,27 @@ static bool domain_update_iommu_snooping(struct
> intel_iommu *skip)
> return ret;
> }
>
> +static bool domain_use_super_page(struct dmar_domain *domain)
> +{
> + struct dmar_drhd_unit *drhd;
> + struct intel_iommu *iommu;
> + bool ret = true;
> +
> + if (!intel_iommu_superpage)
> + return false;
> +
> + rcu_read_lock();
> + for_each_active_iommu(iommu, drhd) {
> + if (drhd->gfx_dedicated && iommu_skip_igfx_superpage) {
> + ret = false;
> + break
^
Missing semicolon. Othwerwise seems to work great here. Thanks.
Are you going to turn this into a proper patch, or do you
want me to just squash this into my patches and repost?
> + }
> + }
> + rcu_read_unlock();
> +
> + return ret;
> +}
> +
> static int domain_update_iommu_superpage(struct dmar_domain *domain,
>struct intel_iommu *skip)
> {
> @@ -659,7 +681,7 @@ static int domain_update_iommu_superpage(struct
> dmar_domain *domain,
> struct intel_iommu *iommu;
> int mask = 0x3;
>
> - if (!intel_iommu_superpage)
> + if (!domain_use_super_page(domain))
> return 0;
>
> /* set iommu_superpage to the smallest common denominator */
> @@ -5656,6 +5678,14 @@ DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL,
> 0x1632, quirk_iommu_igfx);
> DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x163A, quirk_iommu_igfx);
> DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x163D, quirk_iommu_igfx);
>
> +static void quirk_skip_igfx_superpage(struct pci_dev *dev)
> +{
> + pci_info(dev, "Disabling IOMMU superpage for graphics on this
> chipset\n");
> + iommu_skip_igfx_superpage = 1;
> +}
> +
> +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x3184,
> quirk_skip_igfx_superpage);
> +
> static void quirk_iommu_rwbf(struct pci_dev *dev)
> {
> if (risky_device(dev))
>
> Best regards,
> baolu
--
Ville Syrjälä
Intel
___
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [Intel-gfx] [PATCH 1/4] iommu/vt-d: Disable superpage for Geminilake igfx
On 7/12/21 11:47 PM, Ville Syrjälä wrote:
On Mon, Jul 12, 2021 at 07:23:07AM +0800, Lu Baolu wrote:
On 7/10/21 12:47 AM, Ville Syrjala wrote:
From: Ville Syrjälä
While running "gem_exec_big --r single" from igt-gpu-tools on
Geminilake as soon as a 2M mapping is made I tend to get a DMAR
write fault. Strangely the faulting address is always a 4K page
and usually very far away from the 2M page that got mapped.
But if no 2M mappings get used I can't reproduce the fault.
I also tried to dump the PTE for the faulting address but it actually
looks correct to me (ie. definitely seems to have the write bit set):
DMAR: DRHD: handling fault status reg 2
DMAR: [DMA Write] Request device [00:02.0] PASID fault addr
7fa8a78000 [fault reason 05] PTE Write access is not set
DMAR: fault 7fa8a78000 (level=1) PTE = 149efc003
So not really sure what's going on and this might just be full on duct
tape, but it seems to work here. The machine has now survived a whole day
running that test whereas with superpage enabled it fails in less than
a minute usually.
TODO: might be nice to disable superpage only for the igfx iommu
instead of both iommus
If all these quirks are about igfx dedicated iommu's, I would suggest to
disable superpage only for the igfx ones.
Sure. Unfortunately there's no convenient mechanism to do that in
the iommu driver that I can immediately see. So not something I
can just whip up easily. Since you're actually familiar with the
driver maybe you can come up with a decent solution for that?
How about something like below? [no compile, no test...]
diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index 1131b8efb050..2d51ef288a9e 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -338,6 +338,7 @@ static int intel_iommu_strict;
static int intel_iommu_superpage = 1;
static int iommu_identity_mapping;
static int iommu_skip_te_disable;
+static int iommu_skip_igfx_superpage;
#define IDENTMAP_GFX 2
#define IDENTMAP_AZALIA4
@@ -652,6 +653,27 @@ static bool domain_update_iommu_snooping(struct
intel_iommu *skip)
return ret;
}
+static bool domain_use_super_page(struct dmar_domain *domain)
+{
+ struct dmar_drhd_unit *drhd;
+ struct intel_iommu *iommu;
+ bool ret = true;
+
+ if (!intel_iommu_superpage)
+ return false;
+
+ rcu_read_lock();
+ for_each_active_iommu(iommu, drhd) {
+ if (drhd->gfx_dedicated && iommu_skip_igfx_superpage) {
+ ret = false;
+ break
+ }
+ }
+ rcu_read_unlock();
+
+ return ret;
+}
+
static int domain_update_iommu_superpage(struct dmar_domain *domain,
struct intel_iommu *skip)
{
@@ -659,7 +681,7 @@ static int domain_update_iommu_superpage(struct
dmar_domain *domain,
struct intel_iommu *iommu;
int mask = 0x3;
- if (!intel_iommu_superpage)
+ if (!domain_use_super_page(domain))
return 0;
/* set iommu_superpage to the smallest common denominator */
@@ -5656,6 +5678,14 @@ DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL,
0x1632, quirk_iommu_igfx);
DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x163A, quirk_iommu_igfx);
DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x163D, quirk_iommu_igfx);
+static void quirk_skip_igfx_superpage(struct pci_dev *dev)
+{
+ pci_info(dev, "Disabling IOMMU superpage for graphics on this
chipset\n");
+ iommu_skip_igfx_superpage = 1;
+}
+
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x3184,
quirk_skip_igfx_superpage);
+
static void quirk_iommu_rwbf(struct pci_dev *dev)
{
if (risky_device(dev))
Best regards,
baolu
___
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [Intel-gfx] [PATCH 1/4] iommu/vt-d: Disable superpage for Geminilake igfx
On Mon, Jul 12, 2021 at 07:23:07AM +0800, Lu Baolu wrote:
> On 7/10/21 12:47 AM, Ville Syrjala wrote:
> > From: Ville Syrjälä
> >
> > While running "gem_exec_big --r single" from igt-gpu-tools on
> > Geminilake as soon as a 2M mapping is made I tend to get a DMAR
> > write fault. Strangely the faulting address is always a 4K page
> > and usually very far away from the 2M page that got mapped.
> > But if no 2M mappings get used I can't reproduce the fault.
> >
> > I also tried to dump the PTE for the faulting address but it actually
> > looks correct to me (ie. definitely seems to have the write bit set):
> > DMAR: DRHD: handling fault status reg 2
> > DMAR: [DMA Write] Request device [00:02.0] PASID fault addr
> > 7fa8a78000 [fault reason 05] PTE Write access is not set
> > DMAR: fault 7fa8a78000 (level=1) PTE = 149efc003
> >
> > So not really sure what's going on and this might just be full on duct
> > tape, but it seems to work here. The machine has now survived a whole day
> > running that test whereas with superpage enabled it fails in less than
> > a minute usually.
> >
> > TODO: might be nice to disable superpage only for the igfx iommu
> >instead of both iommus
>
> If all these quirks are about igfx dedicated iommu's, I would suggest to
> disable superpage only for the igfx ones.
Sure. Unfortunately there's no convenient mechanism to do that in
the iommu driver that I can immediately see. So not something I
can just whip up easily. Since you're actually familiar with the
driver maybe you can come up with a decent solution for that?
>
> Best regards,
> baolu
>
> > TODO: would be nice to use the macros from include/drm/i915_pciids.h,
> >but can't do that with DECLARE_PCI_FIXUP_HEADER()
> >
> > Cc: David Woodhouse
> > Cc: Lu Baolu
> > Cc: io...@lists.linux-foundation.org
> > Signed-off-by: Ville Syrjälä
> > ---
> > drivers/iommu/intel/iommu.c | 10 ++
> > 1 file changed, 10 insertions(+)
> >
> > diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
> > index 19c7888cbb86..4fff2c9c86af 100644
> > --- a/drivers/iommu/intel/iommu.c
> > +++ b/drivers/iommu/intel/iommu.c
> > @@ -5617,6 +5617,16 @@ DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL,
> > 0x1632, quirk_iommu_igfx);
> > DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x163A, quirk_iommu_igfx);
> > DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x163D, quirk_iommu_igfx);
> >
> > +static void quirk_iommu_nosp(struct pci_dev *dev)
> > +{
> > + pci_info(dev, "Disabling IOMMU superpage for graphics on this
> > chipset\n");
> > + intel_iommu_superpage = 0;
> > +}
> > +
> > +/* Geminilake igfx appears to have issues with superpage */
> > +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x3184, quirk_iommu_nosp);
> > +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x3185, quirk_iommu_nosp);
> > +
> > static void quirk_iommu_rwbf(struct pci_dev *dev)
> > {
> > if (risky_device(dev))
> >
--
Ville Syrjälä
Intel
___
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [Intel-gfx] [PATCH 1/4] iommu/vt-d: Disable superpage for Geminilake igfx
On 7/10/21 12:47 AM, Ville Syrjala wrote:
From: Ville Syrjälä
While running "gem_exec_big --r single" from igt-gpu-tools on
Geminilake as soon as a 2M mapping is made I tend to get a DMAR
write fault. Strangely the faulting address is always a 4K page
and usually very far away from the 2M page that got mapped.
But if no 2M mappings get used I can't reproduce the fault.
I also tried to dump the PTE for the faulting address but it actually
looks correct to me (ie. definitely seems to have the write bit set):
DMAR: DRHD: handling fault status reg 2
DMAR: [DMA Write] Request device [00:02.0] PASID fault addr
7fa8a78000 [fault reason 05] PTE Write access is not set
DMAR: fault 7fa8a78000 (level=1) PTE = 149efc003
So not really sure what's going on and this might just be full on duct
tape, but it seems to work here. The machine has now survived a whole day
running that test whereas with superpage enabled it fails in less than
a minute usually.
TODO: might be nice to disable superpage only for the igfx iommu
instead of both iommus
If all these quirks are about igfx dedicated iommu's, I would suggest to
disable superpage only for the igfx ones.
Best regards,
baolu
TODO: would be nice to use the macros from include/drm/i915_pciids.h,
but can't do that with DECLARE_PCI_FIXUP_HEADER()
Cc: David Woodhouse
Cc: Lu Baolu
Cc: io...@lists.linux-foundation.org
Signed-off-by: Ville Syrjälä
---
drivers/iommu/intel/iommu.c | 10 ++
1 file changed, 10 insertions(+)
diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index 19c7888cbb86..4fff2c9c86af 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -5617,6 +5617,16 @@ DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x1632,
quirk_iommu_igfx);
DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x163A, quirk_iommu_igfx);
DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x163D, quirk_iommu_igfx);
+static void quirk_iommu_nosp(struct pci_dev *dev)
+{
+ pci_info(dev, "Disabling IOMMU superpage for graphics on this
chipset\n");
+ intel_iommu_superpage = 0;
+}
+
+/* Geminilake igfx appears to have issues with superpage */
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x3184, quirk_iommu_nosp);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x3185, quirk_iommu_nosp);
+
static void quirk_iommu_rwbf(struct pci_dev *dev)
{
if (risky_device(dev))
___
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
