Re: [PATCH 2/2] powerpc/pseries: pass MSI affinity to irq_create_mapping()

[Thomas Gleixner]

On Tue, Nov 24 2020 at 21:03, Laurent Vivier wrote:
> With virtio multiqueue, normally each queue IRQ is mapped to a CPU.
>
> This problem cannot be shown on x86_64 for two reasons:

There is only _ONE_ reason why this is not a problem on x86. x86 uses
the generic PCI/MSI domain which supports this out of the box.

> - the call path traverses arch_setup_msi_irqs() that is arch specific:
>
>virtscsi_probe()
>   virtscsi_init()
>  vp_modern_find_vqs()
> vp_find_vqs()
>vp_find_vqs_msix()
>   pci_alloc_irq_vectors_affinity()
>  __pci_enable_msix_range()
> pci_msi_setup_msi_irqs()
>arch_setup_msi_irqs()
>   rtas_setup_msi_irqs()

This is a problem on _all_ variants of PPC MSI providers, not only for
pseries. It's not restricted to virtscsi devices either, that's just the
device which made you discover this.

Thanks,

tglx

[PATCH 2/2] powerpc/pseries: pass MSI affinity to irq_create_mapping()

[Laurent Vivier]

With virtio multiqueue, normally each queue IRQ is mapped to a CPU.

But since commit 0d9f0a52c8b9f ("virtio_scsi: use virtio IRQ affinity")
this is broken on pseries.

The affinity is correctly computed in msi_desc but this is not applied
to the system IRQs.

It appears the affinity is correctly passed to rtas_setup_msi_irqs() but
lost at this point and never passed to irq_domain_alloc_descs()
(see commit 06ee6d571f0e ("genirq: Add affinity hint to irq allocation"))
because irq_create_mapping() doesn't take an affinity parameter.

As the previous patch has added the affinity parameter to
irq_create_mapping() we can forward the affinity from rtas_setup_msi_irqs()
to irq_domain_alloc_descs().

With this change, the virtqueues are correctly dispatched between the CPUs
on pseries.

This problem cannot be shown on x86_64 for two reasons:

- the call path traverses arch_setup_msi_irqs() that is arch specific:

   virtscsi_probe()
  virtscsi_init()
 vp_modern_find_vqs()
vp_find_vqs()
   vp_find_vqs_msix()
  pci_alloc_irq_vectors_affinity()
 __pci_enable_msix_range()
pci_msi_setup_msi_irqs()
   arch_setup_msi_irqs()
  rtas_setup_msi_irqs()
 irq_create_mapping()
irq_domain_alloc_descs()
  __irq_alloc_descs()

- and x86_64 has CONFIG_PCI_MSI_IRQ_DOMAIN that uses another path:

   virtscsi_probe()
  virtscsi_init()
 vp_modern_find_vqs()
vp_find_vqs()
   vp_find_vqs_msix()
  pci_alloc_irq_vectors_affinity()
 __pci_enable_msix_range()
__msi_domain_alloc_irqs()
   __irq_domain_alloc_irqs()
  __irq_alloc_descs()

Signed-off-by: Laurent Vivier 
---
 arch/powerpc/platforms/pseries/msi.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/powerpc/platforms/pseries/msi.c 
b/arch/powerpc/platforms/pseries/msi.c
index 42ba08eaea91..58197f92c6a2 100644
--- a/arch/powerpc/platforms/pseries/msi.c
+++ b/arch/powerpc/platforms/pseries/msi.c
@@ -458,7 +458,7 @@ static int rtas_setup_msi_irqs(struct pci_dev *pdev, int 
nvec_in, int type)
return hwirq;
}
 
-   virq = irq_create_mapping(NULL, hwirq, NULL);
+   virq = irq_create_mapping(NULL, hwirq, entry->affinity);
 
if (!virq) {
pr_debug("rtas_msi: Failed mapping hwirq %d\n", hwirq);
-- 
2.28.0