I started implementing MSI-X support for arm64 adn tried to use re(4)
for testing. This failed miserably and when I tried to use MSI-X with
re(4) on amd64 I noticed it didn't work either.
Turns out the Realtek hardware doesn't support 64-bit access to MSI-X
table. Reads return 0xffffffffffffffff and writes get ignored.
Looking at the relevant documentation I noticed that the message
address is actually split into two 32-bit "DWORD" entries. Which in
turn suggests that the Realtek folks have the standard on their side...
Diff below fixes the issue. I did already add a "MAU32" define for
this, even though it wasn't quite right ;). I also took the
opportunity to add the "function mask" bit that we'll probably need in
the future if we start being clever with using multiple vectors and
ave a need to guarantee atomicity of making changes to the MSI-X
config.
ok?
Index: dev/pci/pcireg.h
===================================================================
RCS file: /cvs/src/sys/dev/pci/pcireg.h,v
retrieving revision 1.56
diff -u -p -r1.56 pcireg.h
--- dev/pci/pcireg.h 3 Aug 2018 22:18:13 -0000 1.56
+++ dev/pci/pcireg.h 30 May 2019 18:06:39 -0000
@@ -617,6 +617,7 @@ typedef u_int8_t pci_revision_t;
* Extended Message Signaled Interrups; access via capability pointer.
*/
#define PCI_MSIX_MC_MSIXE 0x80000000
+#define PCI_MSIX_MC_FM 0x40000000
#define PCI_MSIX_MC_TBLSZ_MASK 0x07ff0000
#define PCI_MSIX_MC_TBLSZ_SHIFT 16
#define PCI_MSIX_MC_TBLSZ(reg) \
@@ -626,7 +627,7 @@ typedef u_int8_t pci_revision_t;
#define PCI_MSIX_TABLE_OFF ~(PCI_MSIX_TABLE_BIR)
#define PCI_MSIX_MA(i) ((i) * 16 + 0)
-#define PCI_MSIX_MAU32(i) ((i) * 16 + 0)
+#define PCI_MSIX_MAU32(i) ((i) * 16 + 4)
#define PCI_MSIX_MD(i) ((i) * 16 + 8)
#define PCI_MSIX_VC(i) ((i) * 16 + 12)
#define PCI_MSIX_VC_MASK 0x00000001
Index: arch/amd64/pci/pci_machdep.c
===================================================================
RCS file: /cvs/src/sys/arch/amd64/pci/pci_machdep.c,v
retrieving revision 1.69
diff -u -p -r1.69 pci_machdep.c
--- arch/amd64/pci/pci_machdep.c 19 Aug 2018 08:23:47 -0000 1.69
+++ arch/amd64/pci/pci_machdep.c 30 May 2019 18:06:39 -0000
@@ -475,7 +475,8 @@ msix_addroute(struct pic *pic, struct cp
_bus_space_map(memt, base + offset, tblsz * 16, 0, &memh))
panic("%s: cannot map registers", __func__);
- bus_space_write_8(memt, memh, PCI_MSIX_MA(entry), addr);
+ bus_space_write_4(memt, memh, PCI_MSIX_MA(entry), addr);
+ bus_space_write_4(memt, memh, PCI_MSIX_MAU32(entry), 0);
bus_space_write_4(memt, memh, PCI_MSIX_MD(entry), vec);
bus_space_barrier(memt, memh, PCI_MSIX_MA(entry), 16,
BUS_SPACE_BARRIER_WRITE);
