Am Sun, Jan 16, 2022 at 06:38:08PM -0000 schrieb Stuart Henderson: > On 2022-01-16, Ax0n <a...@h-i-r.net> wrote: > > I have a SuperMicro X9DRH-7TF that's been chugging along diligently in a > > data center. I've been upgrading the vmm instances it hosts but I let the > > host get pretty far behind. After running sysupgrade to 6.8, the system > > never came back online. I have the Supermicro BMC IPMI, so I used that to > > mount remote CD-ROMs to complete the upgrade to 6.9 and 7.0. The network > > still doesn't work, from the install images or after it's been upgraded. > > There have been "mem address conflict" messages on this hardware for as > > long as I can remember. OpenBSD 6.4 at least. They may or may not be a red > > herring. > > > > I mounted the OpenBSD 6.7 cd67.iso and I was able to get it on the network > > again. Somewhere between 6.7 and 6.8, ix(4) broke with this hardware, an > > integrated dual-port Intel X540. > > Do both ix0 and ix1 break, or just ix1? > > One important difference visible in dmesg is that ix starts using MSI-X.
Can you give this a go? I remember issues with broken MSI-X bars or so, but that was on some Ryzen. Patrick diff --git a/sys/arch/amd64/pci/pci_machdep.c b/sys/arch/amd64/pci/pci_machdep.c index 72456c32829..1cdc33757ee 100644 --- a/sys/arch/amd64/pci/pci_machdep.c +++ b/sys/arch/amd64/pci/pci_machdep.c @@ -96,6 +96,8 @@ #include <dev/acpi/acpidmar.h> #endif +int pci_check_msix_bar(pci_chipset_tag_t, pcitag_t); + /* * Memory Mapped Configuration space access. * @@ -497,6 +499,36 @@ msix_delroute(struct pic *pic, struct cpu_info *ci, int pin, int vec, int type) pci_msix_table_unmap(pc, tag, memt, memh); } +/* + * BIOS assigns BAR addresses for every PCI device, but some BIOS are bugged and + * map an illegal address, say for instance an address outside the ppb(4) range. + * When this occurs, OpenBSD rewrites the BAR address as 0. + * This BAR might be the MSI-X BAR, therefore we consider address 0 as a bad BAR + * address and fail MSI-X mapping. It's not greatest solution but since we + * never correct bad addresses for BARs, this at least prevents us from using + * MSI-X when we know it won't work. + */ +int +pci_check_msix_bar(pci_chipset_tag_t pc, pcitag_t tag) +{ + pcireg_t table; + bus_addr_t base; + int bir, off; + + if (pci_get_capability(pc, tag, PCI_CAP_MSIX, &off, NULL) == 0) + return (ENODEV); + + table = pci_conf_read(pc, tag, off + PCI_MSIX_TABLE); + bir = (table & PCI_MSIX_TABLE_BIR); + bir = PCI_MAPREG_START + bir * 4; + + if (pci_mem_find(pc, tag, bir, &base, NULL, NULL) || + base == 0) + return (ENODEV); + + return (0); +} + int pci_intr_map_msix(struct pci_attach_args *pa, int vec, pci_intr_handle_t *ihp) { @@ -507,7 +539,8 @@ pci_intr_map_msix(struct pci_attach_args *pa, int vec, pci_intr_handle_t *ihp) KASSERT(PCI_MSIX_VEC(vec) == vec); if ((pa->pa_flags & PCI_FLAGS_MSI_ENABLED) == 0 || mp_busses == NULL || - pci_get_capability(pc, tag, PCI_CAP_MSIX, NULL, ®) == 0) + (pci_get_capability(pc, tag, PCI_CAP_MSIX, NULL, ®) == 0) || + pci_check_msix_bar(pc, tag)) return 1; if (vec > PCI_MSIX_MC_TBLSZ(reg)) diff --git a/sys/dev/pci/pci_map.c b/sys/dev/pci/pci_map.c index 3cc0e61c0df..9567e83c363 100644 --- a/sys/dev/pci/pci_map.c +++ b/sys/dev/pci/pci_map.c @@ -135,7 +135,12 @@ obsd_pci_mem_find(pci_chipset_tag_t pc, pcitag_t tag, int reg, pcireg_t type, u_int64_t waddress, wmask; int s, is64bit; - is64bit = (PCI_MAPREG_MEM_TYPE(type) == PCI_MAPREG_MEM_TYPE_64BIT); + if (type == -1) + is64bit = (PCI_MAPREG_MEM_TYPE(pci_conf_read(pc, tag, reg)) + == PCI_MAPREG_MEM_TYPE_64BIT); + else + is64bit = (PCI_MAPREG_MEM_TYPE(type) == + PCI_MAPREG_MEM_TYPE_64BIT); if (reg < PCI_MAPREG_START || #if 0