On 25.1.2023. 12:24, Joerg Streckfuss wrote: > > Dear List, > > we have problems with Intel nics of type Intel X710 (10 GbE) on a Dell > R740. In total we have three nics with four ports each. With the uprade > to OpenBSD 6.8 we lost two ports (ixl11 and ixl12). Now we upraded > iteratively to OpenBSD 7.1 an we lost another port (ixl10). The update > to OpenBSD 7.2 is pending, but I don't want to risk losing another port. > > Cause seems to be a problem with the interrupt assignment. > > The relevant dmesg part is as fallows: > > <snip> > ixl11 at pci12 dev 0 function 1 "Intel X710 SFP+" rev 0x02: port 2, FW > 7.83.59945 API 1.9, ms1 > failed to allocate interrupt slot for PIC msix pin -2135686911 > ixl11: unable to establish interrupt 1 > ixl12 at pci12 dev 0 function 2 "Intel X710 SFP+" rev 0x02: port 0, FW > 7.83.59945 API 1.9, ms2 > failed to allocate interrupt slot for PIC msix pin -2135686655 > ixl12: unable to establish interrupt 1 > ixl13 at pci12 dev 0 function 3 "Intel X710 SFP+" rev 0x02: port 1, FW > 7.83.59945 API 1.9, ms3 > failed to allocate interrupt slot for PIC msix pin -2135686399 > ixl13: unable to establish interrupt 1 > <snap> >
Hi, I have same problems on few firewalls. One with 16 cores and 4 mcx, second with 24 cores and bnxt,mcx,ixl and ix <- this on is in lab, so not a big problem :) third on 12 core machine with 2 ix and 4 em but only when testing jmatthew@ em multiqueue diff https://marc.info/?l=openbsd-tech&m=165642186010149&w=2 Workaroud for this problem, at least for me, is to lower number of cpu in BIOS or remove some nics. In my case, I lowered 16 cores to 12, then mcx have 8 queues and that was fine for that firewall. For you, maybe try to lower cpu's to 6, then ixl would have 4 queues, and maybe, just maybe, everything will work for you :) Just keep in mind that parallel forwarding is using 4 cpu, so at least you need 4 cpu's or 4 queues on nic. I think that this is nice insight and relevant answer from kettenis@ https://www.mail-archive.com/tech@openbsd.org/msg71790.html I think that this problem isn't ignored by developers, but on other hand more and more people is having this problem. When jmatthew@ em diff would be commited, even more people would report this problem. > full dmesg: > > <snip> > Booting from Hard drive C: > Using drive 0, partition 3. > Loading...... > probing: pc0 com0 mem[624K 1266M 2M 398M 30720M a20=on] > disk: hd0+ >>> OpenBSD/amd64 BOOT 3.53 > switching console to com0 >>> OpenBSD/amd64 BOOT 3.53 > booting hd0a:/bsd: 15639832+3699728+348192+0+1175552 > [1126995+128+1220904+924861]=0x17074c0 > entry point at 0xffffffff81001000 > �[ using 3273920 bytes of bsd ELF > symbol table ] > Copyright (c) 1982, 1986, 1989, 1991, 1993 > The Regents of the University of California. All rights reserved. > Copyright (c) 1995-2022 OpenBSD. All rights reserved. > https://www.OpenBSD.org > > OpenBSD 7.1 (GENERIC.MP) #2: Fri Jan 20 13:16:22 MST 2023 > > t...@syspatch-71-amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP > real mem = 33941528576 (32369MB) > avail mem = 32895590400 (31371MB) > random: good seed from bootblocks > mpath0 at root > scsibus0 at mpath0: 256 targets > mainbus0 at root > bios0 at mainbus0: SMBIOS rev. 3.2 @ 0x68e36000 (75 entries) > bios0: vendor Dell Inc. version "2.10.2" date 02/24/2021 > bios0: Dell Inc. PowerEdge R740 > acpi0 at bios0: ACPI 6.1 > acpi0: sleep states S0 S5 > acpi0: tables DSDT FACP SSDT TPM2 SSDT MCEJ WDAT SLIC HPET APIC MCFG > MIGT MSCT PCAT PCCT RASFJ > acpi0: wakeup devices XHCI(S4) RP17(S4) PXSX(S4) RP18(S4) PXSX(S4) > RP19(S4) PXSX(S4) RP20(S4)] > acpitimer0 at acpi0: 3579545 Hz, 24 bits > acpihpet0 at acpi0: 23999999 Hz > acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat > cpu0 at mainbus0: apid 2 (boot processor) > cpu0: Intel(R) Xeon(R) Gold 5122 CPU @ 3.60GHz, 3692.05 MHz, 06-55-04 > cpu0: > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,N > cpu0: 256KB 64b/line 8-way L2 cache > cpu0: cannot disable silicon debug > cpu0: smt 0, core 1, package 0 > mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges > cpu0: apic clock running at 24MHz > cpu0: mwait min=64, max=64, C-substates=0.2.0.2, IBE > cpu1 at mainbus0: apid 10 (application processor) > cpu1: Intel(R) Xeon(R) Gold 5122 CPU @ 3.60GHz, 3691.34 MHz, 06-55-04 > cpu1: > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,N > cpu1: 256KB 64b/line 8-way L2 cache > cpu1: cannot disable silicon debug > cpu1: smt 0, core 5, package 0 > cpu2 at mainbus0: apid 4 (application processor) > cpu2: Intel(R) Xeon(R) Gold 5122 CPU @ 3.60GHz, 3691.33 MHz, 06-55-04 > cpu2: > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,N > cpu2: 256KB 64b/line 8-way L2 cache > cpu2: cannot disable silicon debug > cpu2: smt 0, core 2, package 0 > cpu3 at mainbus0: apid 22 (application processor) > cpu3: Intel(R) Xeon(R) Gold 5122 CPU @ 3.60GHz, 3691.33 MHz, 06-55-04 > cpu3: > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,N > cpu3: 256KB 64b/line 8-way L2 cache > cpu3: cannot disable silicon debug > cpu3: smt 0, core 11, package 0 > cpu4 at mainbus0: apid 3 (application processor) > cpu4: Intel(R) Xeon(R) Gold 5122 CPU @ 3.60GHz, 3691.32 MHz, 06-55-04 > cpu4: > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,N > cpu4: 256KB 64b/line 8-way L2 cache > cpu4: cannot disable silicon debug > cpu4: smt 1, core 1, package 0 > cpu5 at mainbus0: apid 11 (application processor) > cpu5: Intel(R) Xeon(R) Gold 5122 CPU @ 3.60GHz, 3691.32 MHz, 06-55-04 > cpu5: > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,N > cpu5: 256KB 64b/line 8-way L2 cache > cpu5: cannot disable silicon debug > cpu5: smt 1, core 5, package 0 > cpu6 at mainbus0: apid 5 (application processor) > cpu6: Intel(R) Xeon(R) Gold 5122 CPU @ 3.60GHz, 3691.32 MHz, 06-55-04 > cpu6: > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,N > cpu6: 256KB 64b/line 8-way L2 cache > cpu6: cannot disable silicon debug > cpu6: smt 1, core 2, package 0 > cpu7 at mainbus0: apid 23 (application processor) > cpu7: Intel(R) Xeon(R) Gold 5122 CPU @ 3.60GHz, 3691.32 MHz, 06-55-04 > cpu7: > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,N > cpu7: 256KB 64b/line 8-way L2 cache > cpu7: cannot disable silicon debug > cpu7: smt 1, core 11, package 0 > ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 20, 24 pins > ioapic1 at mainbus0: apid 9 pa 0xfec01000, version 20, 8 pins > ioapic2 at mainbus0: apid 10 pa 0xfec08000, version 20, 8 pins > ioapic3 at mainbus0: apid 11 pa 0xfec10000, version 20, 8 pins > ioapic4 at mainbus0: apid 12 pa 0xfec18000, version 20, 8 pins > acpimadt0: unknown apic structure type 80 > acpimadt0: unknown apic structure type 80 > acpimadt0: unknown apic structure type 80 > acpimadt0: unknown apic structure type 80 > acpimadt0: unknown apic structure type 80 > acpimadt0: unknown apic structure type 80 > acpimadt0: unknown apic structure type 80 > acpimadt0: unknown apic structure type 80 > acpimadt0: unknown apic structure type 80 > acpimadt0: unknown apic structure type 80 > acpimadt0: unknown apic structure type 80 > acpimadt0: unknown apic structure type 80 > acpimadt0: unknown apic structure type 80 > acpimadt0: unknown apic structure type 80 > acpimadt0: unknown apic structure type 80 > acpimadt0: unknown apic structure type 80 > acpimadt0: unknown apic structure type 80 > acpimadt0: unknown apic structure type 80 > acpimadt0: unknown apic structure type 80 > acpimadt0: unknown apic structure type 80 > acpimadt0: unknown apic structure type 80 > acpimadt0: unknown apic structure type 80 > acpimadt0: unknown apic structure type 80 > acpimadt0: unknown apic structure type 80 > acpimadt0: unknown apic structure type 80 > acpimadt0: unknown apic structure type 80 > acpimadt0: unknown apic structure type 80 > acpimadt0: unknown apic structure type 80 > acpimcfg0 at acpi0 > acpimcfg0: addr 0x80000000, bus 0-255 > acpiprt0 at acpi0: bus 0 (PC00) > acpiprt1 at acpi0: bus -1 (RP17) > acpiprt2 at acpi0: bus -1 (RP18) > acpiprt3 at acpi0: bus -1 (RP19) > acpiprt4 at acpi0: bus -1 (RP20) > acpiprt5 at acpi0: bus 1 (RP01) > acpiprt6 at acpi0: bus -1 (RP02) > acpiprt7 at acpi0: bus -1 (RP03) > acpiprt8 at acpi0: bus -1 (RP04) > acpiprt9 at acpi0: bus 2 (RP05) > acpiprt10 at acpi0: bus -1 (RP06) > acpiprt11 at acpi0: bus -1 (RP07) > acpiprt12 at acpi0: bus -1 (RP08) > acpiprt13 at acpi0: bus -1 (RP09) > acpiprt14 at acpi0: bus -1 (RP10) > acpiprt15 at acpi0: bus -1 (RP11) > acpiprt16 at acpi0: bus -1 (RP12) > acpiprt17 at acpi0: bus -1 (RP13) > acpiprt18 at acpi0: bus -1 (RP14) > acpiprt19 at acpi0: bus -1 (RP15) > acpiprt20 at acpi0: bus -1 (RP16) > acpiprt21 at acpi0: bus 22 (PC01) > acpiprt22 at acpi0: bus 23 (BR1A) > acpiprt23 at acpi0: bus -1 (BR1B) > acpiprt24 at acpi0: bus 24 (BR1C) > acpiprt25 at acpi0: bus -1 (BR1D) > acpiprt26 at acpi0: bus 100 (PC02) > acpiprt27 at acpi0: bus 101 (BR2A) > acpiprt28 at acpi0: bus -1 (BR2B) > acpiprt29 at acpi0: bus 102 (BR2C) > acpiprt30 at acpi0: bus -1 (BR2D) > acpiprt31 at acpi0: bus 178 (PC03) > acpiprt32 at acpi0: bus 179 (BR3A) > acpiprt33 at acpi0: bus -1 (BR3B) > acpiprt34 at acpi0: bus 180 (BR3C) > acpiprt35 at acpi0: bus -1 (BR3D) > "ACPI0004" at acpi0 not configured > acpicpu0 at acpi0: C1(1000@1 mwait.1) > acpicpu1 at acpi0: C1(1000@1 mwait.1) > acpicpu2 at acpi0: C1(1000@1 mwait.1) > acpicpu3 at acpi0: C1(1000@1 mwait.1) > acpicpu4 at acpi0: C1(1000@1 mwait.1) > acpicpu5 at acpi0: C1(1000@1 mwait.1) > acpicpu6 at acpi0: C1(1000@1 mwait.1) > acpicpu7 at acpi0: C1(1000@1 mwait.1) > "PNP0C33" at acpi0 not configured > acpipci0 at acpi0 PC00: 0x00000000 0x00000011 0x00000001 > "PNP0003" at acpi0 not configured > acpicmos0 at acpi0 > com0 at acpi0 COMA addr 0x3f8/0x8 irq 4: ns16550a, 16 byte fifo > com0: console > com1 at acpi0 COMB addr 0x2f8/0x8 irq 3: ns16550a, 16 byte fifo > "IPI0001" at acpi0 not configured > acpipci1 at acpi0 PC01: 0x00000000 0x00000011 0x00000001 > acpipci2 at acpi0 PC02: 0x00000000 0x00000011 0x00000001 > acpipci3 at acpi0 PC03: 0x00000000 0x00000011 0x00000001 > "PNP0C14" at acpi0 not configured > "INTL0000" at acpi0 not configured > ipmi at mainbus0 not configured > cpu0: using VERW MDS workaround (except on vmm entry) > pci0 at mainbus0 bus 0 > pchb0 at pci0 dev 0 function 0 "Intel Xeon Scalable Host" rev 0x04 > "Intel Xeon Scalable VT-d" rev 0x04 at pci0 dev 5 function 0 not configured > "Intel Xeon Scalable RAS" rev 0x04 at pci0 dev 5 function 2 not configured > "Intel Xeon Scalable I/O APIC" rev 0x04 at pci0 dev 5 function 4 not > configured > "Intel Xeon Scalable Ubox" rev 0x04 at pci0 dev 8 function 0 not configured > "Intel Xeon Scalable Ubox" rev 0x04 at pci0 dev 8 function 1 not configured > "Intel Xeon Scalable Ubox" rev 0x04 at pci0 dev 8 function 2 not configured > "Intel C620 MROM" rev 0x09 at pci0 dev 17 function 0 not configured > ahci0 at pci0 dev 17 function 5 "Intel C620 AHCI" rev 0x09: msi, AHCI 1.3.1 > scsibus1 at ahci0: 32 targets > xhci0 at pci0 dev 20 function 0 "Intel C620 xHCI" rev 0x09: msi, xHCI 1.0 > usb0 at xhci0: USB revision 3.0 > uhub0 at usb0 configuration 1 interface 0 "Intel xHCI root hub" rev > 3.00/1.00 addr 1 > "Intel C620 Thermal" rev 0x09 at pci0 dev 20 function 2 not configured > "Intel C620 MEI" rev 0x09 at pci0 dev 22 function 0 not configured > "Intel C620 MEI" rev 0x09 at pci0 dev 22 function 1 not configured > "Intel C620 MEI" rev 0x09 at pci0 dev 22 function 4 not configured > ahci1 at pci0 dev 23 function 0 "Intel C620 AHCI" rev 0x09: msi, AHCI 1.3.1 > scsibus2 at ahci1: 32 targets > ppb0 at pci0 dev 28 function 0 "Intel C620 PCIE" rev 0xf9: msi > pci1 at ppb0 bus 1 > em0 at pci1 dev 0 function 0 "Intel I350" rev 0x01: msi, address > e4:43:4b:30:7d:54 > em1 at pci1 dev 0 function 1 "Intel I350" rev 0x01: msi, address > e4:43:4b:30:7d:55 > ppb1 at pci0 dev 28 function 4 "Intel C620 PCIE" rev 0xf9: msi > pci2 at ppb1 bus 2 > ppb2 at pci2 dev 0 function 0 unknown vendor 0x1556 product 0xbe00 rev 0x02 > pci3 at ppb2 bus 3 > vga1 at pci3 dev 0 function 0 vendor "Matrox", unknown product 0x0536 > rev 0x04 > wsdisplay at vga1 not configured > pcib0 at pci0 dev 31 function 0 "Intel C621 LPC" rev 0x09 > "Intel C620 Power" rev 0x09 at pci0 dev 31 function 2 not configured > ichiic0 at pci0 dev 31 function 4 "Intel C620 SMBus" rev 0x09: apic 8 > int 16 > iic0 at ichiic0 > "Intel C620 SPI" rev 0x09 at pci0 dev 31 function 5 not configured > isa0 at pcib0 > isadma0 at isa0 > pcppi0 at isa0 port 0x61 > spkr0 at pcppi0 > pci4 at mainbus0 bus 22 > ppb3 at pci4 dev 0 function 0 "Intel Xeon Scalable PCIE" rev 0x04: msi > pci5 at ppb3 bus 23 > 23:0:0: rom address conflict 0xfff80000/0x80000 > 23:0:1: rom address conflict 0xfff80000/0x80000 > ixl0 at pci5 dev 0 function 0 "Intel X710 SFP+" rev 0x02: port 0, FW > 7.83.59945 API 1.9, msix0 > ixl1 at pci5 dev 0 function 1 "Intel X710 SFP+" rev 0x02: port 1, FW > 7.83.59945 API 1.9, msix1 > ppb4 at pci4 dev 2 function 0 "Intel Xeon Scalable PCIE" rev 0x04 > pci6 at ppb4 bus 24 > 24:0:0: rom address conflict 0xfff80000/0x80000 > 24:0:1: rom address conflict 0xfff80000/0x80000 > ix0 at pci6 dev 0 function 0 "Intel 82599" rev 0x01, msix, 4 queues, > address e4:43:4b:30:7d:50 > ix1 at pci6 dev 0 function 1 "Intel 82599" rev 0x01, msix, 4 queues, > address e4:43:4b:30:7d:52 > "Intel Xeon Scalable VT-d" rev 0x04 at pci4 dev 5 function 0 not configured > "Intel Xeon Scalable RAS" rev 0x04 at pci4 dev 5 function 2 not configured > "Intel Xeon Scalable IOxAPIC" rev 0x04 at pci4 dev 5 function 4 not > configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 8 function 0 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 8 function 1 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 8 function 2 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 8 function 3 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 8 function 4 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 8 function 5 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 8 function 6 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 8 function 7 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 9 function 0 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 9 function 1 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 9 function 2 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 9 function 3 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 9 function 4 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 9 function 5 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 9 function 6 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 9 function 7 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 10 function 0 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 10 function 1 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 10 function 2 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 10 function 3 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 10 function 4 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 10 function 5 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 10 function 6 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 10 function 7 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 11 function 0 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 11 function 1 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 11 function 2 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 11 function 3 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 14 function 0 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 14 function 1 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 14 function 2 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 14 function 3 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 14 function 4 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 14 function 5 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 14 function 6 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 14 function 7 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 15 function 0 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 15 function 1 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 15 function 2 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 15 function 3 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 15 function 4 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 15 function 5 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 15 function 6 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 15 function 7 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 16 function 0 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 16 function 1 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 16 function 2 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 16 function 3 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 16 function 4 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 16 function 5 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 16 function 6 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 16 function 7 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 17 function 0 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 17 function 1 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 17 function 2 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 17 function 3 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 29 function 0 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 29 function 1 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 29 function 2 not configured > "Intel Xeon Scalable CHA" rev 0x04 at pci4 dev 29 function 3 not configured > "Intel Xeon Scalable PCU" rev 0x04 at pci4 dev 30 function 0 not configured > "Intel Xeon Scalable PCU" rev 0x04 at pci4 dev 30 function 1 not configured > "Intel Xeon Scalable PCU" rev 0x04 at pci4 dev 30 function 2 not configured > "Intel Xeon Scalable PCU" rev 0x04 at pci4 dev 30 function 3 not configured > "Intel Xeon Scalable PCU" rev 0x04 at pci4 dev 30 function 4 not configured > "Intel Xeon Scalable PCU" rev 0x04 at pci4 dev 30 function 5 not configured > "Intel Xeon Scalable PCU" rev 0x04 at pci4 dev 30 function 6 not configured > pci7 at mainbus0 bus 100 > ppb5 at pci7 dev 0 function 0 "Intel Xeon Scalable PCIE" rev 0x04: msi > pci8 at ppb5 bus 101 > 101:0:0: rom address conflict 0xfff80000/0x80000 > 101:0:1: rom address conflict 0xfff80000/0x80000 > 101:0:2: rom address conflict 0xfff80000/0x80000 > 101:0:3: rom address conflict 0xfff80000/0x80000 > ixl2 at pci8 dev 0 function 0 "Intel X710 SFP+" rev 0x02: port 3, FW > 7.83.59945 API 1.9, msix0 > ixl3 at pci8 dev 0 function 1 "Intel X710 SFP+" rev 0x02: port 2, FW > 7.83.59945 API 1.9, msix1 > ixl4 at pci8 dev 0 function 2 "Intel X710 SFP+" rev 0x02: port 0, FW > 7.83.59945 API 1.9, msix2 > ixl5 at pci8 dev 0 function 3 "Intel X710 SFP+" rev 0x02: port 1, FW > 7.83.59945 API 1.9, msix3 > ppb6 at pci7 dev 2 function 0 "Intel Xeon Scalable PCIE" rev 0x04 > pci9 at ppb6 bus 102 > 102:0:0: rom address conflict 0xfff00000/0x100000 > mfii0 at pci9 dev 0 function 0 "Symbios Logic MegaRAID SAS3508" rev > 0x01: msi > mfii0: "PERC H740P Mini ", firmware 51.14.0-3900, 8192MB cache > scsibus3 at mfii0: 64 targets > sd0 at scsibus3 targ 0 lun 0: <DELL, PERC H740P Mini, 5.14> > naa.64cd98f07028c90024ba5c9db8f13a > sd0: 285568MB, 512 bytes/sector, 584843264 sectors > scsibus4 at mfii0: 256 targets > "Intel Xeon Scalable VT-d" rev 0x04 at pci7 dev 5 function 0 not configured > "Intel Xeon Scalable RAS" rev 0x04 at pci7 dev 5 function 2 not configured > "Intel Xeon Scalable IOxAPIC" rev 0x04 at pci7 dev 5 function 4 not > configured > "Intel Xeon Scalable IMC" rev 0x04 at pci7 dev 8 function 0 not configured > "Intel Xeon Scalable IMC" rev 0x04 at pci7 dev 9 function 0 not configured > "Intel Xeon Scalable IMC" rev 0x04 at pci7 dev 10 function 0 not configured > "Intel Xeon Scalable IMC" rev 0x04 at pci7 dev 10 function 1 not configured > "Intel Xeon Scalable IMC" rev 0x04 at pci7 dev 10 function 2 not configured > "Intel Xeon Scalable IMC" rev 0x04 at pci7 dev 10 function 3 not configured > "Intel Xeon Scalable IMC" rev 0x04 at pci7 dev 10 function 4 not configured > "Intel Xeon Scalable LM" rev 0x04 at pci7 dev 10 function 5 not configured > "Intel Xeon Scalable LMS" rev 0x04 at pci7 dev 10 function 6 not configured > "Intel Xeon Scalable LMDP" rev 0x04 at pci7 dev 10 function 7 not > configured > "Intel Xeon Scalable DECS" rev 0x04 at pci7 dev 11 function 0 not > configured > "Intel Xeon Scalable LM" rev 0x04 at pci7 dev 11 function 1 not configured > "Intel Xeon Scalable LMS" rev 0x04 at pci7 dev 11 function 2 not configured > "Intel Xeon Scalable LMDP" rev 0x04 at pci7 dev 11 function 3 not > configured > "Intel Xeon Scalable IMC" rev 0x04 at pci7 dev 12 function 0 not configured > "Intel Xeon Scalable IMC" rev 0x04 at pci7 dev 12 function 1 not configured > "Intel Xeon Scalable IMC" rev 0x04 at pci7 dev 12 function 2 not configured > "Intel Xeon Scalable IMC" rev 0x04 at pci7 dev 12 function 3 not configured > "Intel Xeon Scalable IMC" rev 0x04 at pci7 dev 12 function 4 not configured > "Intel Xeon Scalable LM" rev 0x04 at pci7 dev 12 function 5 not configured > "Intel Xeon Scalable LMS" rev 0x04 at pci7 dev 12 function 6 not configured > "Intel Xeon Scalable LMDP" rev 0x04 at pci7 dev 12 function 7 not > configured > "Intel Xeon Scalable DECS" rev 0x04 at pci7 dev 13 function 0 not > configured > "Intel Xeon Scalable LM" rev 0x04 at pci7 dev 13 function 1 not configured > "Intel Xeon Scalable LMS" rev 0x04 at pci7 dev 13 function 2 not configured > "Intel Xeon Scalable LMDP" rev 0x04 at pci7 dev 13 function 3 not > configured > pci10 at mainbus0 bus 178 > ppb7 at pci10 dev 0 function 0 "Intel Xeon Scalable PCIE" rev 0x04: msi > pci11 at ppb7 bus 179 > 179:0:0: rom address conflict 0xfff80000/0x80000 > 179:0:1: rom address conflict 0xfff80000/0x80000 > 179:0:2: rom address conflict 0xfff80000/0x80000 > 179:0:3: rom address conflict 0xfff80000/0x80000 > ixl6 at pci11 dev 0 function 0 "Intel X710 SFP+" rev 0x02: port 3, FW > 7.83.59945 API 1.9, msi0 > ixl7 at pci11 dev 0 function 1 "Intel X710 SFP+" rev 0x02: port 2, FW > 7.83.59945 API 1.9, msi1 > ixl8 at pci11 dev 0 function 2 "Intel X710 SFP+" rev 0x02: port 0, FW > 7.83.59945 API 1.9, msi2 > ixl9 at pci11 dev 0 function 3 "Intel X710 SFP+" rev 0x02: port 1, FW > 7.83.59945 API 1.9, msi3 > ppb8 at pci10 dev 2 function 0 "Intel Xeon Scalable PCIE" rev 0x04: msi > pci12 at ppb8 bus 180 > 180:0:0: rom address conflict 0xfff80000/0x80000 > 180:0:1: rom address conflict 0xfff80000/0x80000 > 180:0:2: rom address conflict 0xfff80000/0x80000 > 180:0:3: rom address conflict 0xfff80000/0x80000 > ixl10 at pci12 dev 0 function 0 "Intel X710 SFP+" rev 0x02: port 3, FW > 7.83.59945 API 1.9, ms0 > ixl11 at pci12 dev 0 function 1 "Intel X710 SFP+" rev 0x02: port 2, FW > 7.83.59945 API 1.9, ms1 > failed to allocate interrupt slot for PIC msix pin -2135686911 > ixl11: unable to establish interrupt 1 > ixl12 at pci12 dev 0 function 2 "Intel X710 SFP+" rev 0x02: port 0, FW > 7.83.59945 API 1.9, ms2 > failed to allocate interrupt slot for PIC msix pin -2135686655 > ixl12: unable to establish interrupt 1 > ixl13 at pci12 dev 0 function 3 "Intel X710 SFP+" rev 0x02: port 1, FW > 7.83.59945 API 1.9, ms3 > failed to allocate interrupt slot for PIC msix pin -2135686399 > ixl13: unable to establish interrupt 1 > "Intel Xeon Scalable VT-d" rev 0x04 at pci10 dev 5 function 0 not > configured > "Intel Xeon Scalable RAS" rev 0x04 at pci10 dev 5 function 2 not configured > "Intel Xeon Scalable IOxAPIC" rev 0x04 at pci10 dev 5 function 4 not > configured > "Intel Xeon Scalable KTI" rev 0x04 at pci10 dev 14 function 0 not > configured > "Intel Xeon Scalable UPI" rev 0x04 at pci10 dev 14 function 1 not > configured > "Intel Xeon Scalable KTI" rev 0x04 at pci10 dev 15 function 0 not > configured > "Intel Xeon Scalable UPI" rev 0x04 at pci10 dev 15 function 1 not > configured > "Intel Xeon Scalable KTI" rev 0x04 at pci10 dev 16 function 0 not > configured > "Intel Xeon Scalable UPI" rev 0x04 at pci10 dev 16 function 1 not > configured > "Intel Xeon Scalable M3KTI" rev 0x04 at pci10 dev 18 function 0 not > configured > "Intel Xeon Scalable M3KTI" rev 0x04 at pci10 dev 18 function 1 not > configured > "Intel Xeon Scalable M3KTI" rev 0x04 at pci10 dev 18 function 2 not > configured > "Intel Xeon Scalable M3KTI" rev 0x04 at pci10 dev 18 function 4 not > configured > "Intel Xeon Scalable M3KTI" rev 0x04 at pci10 dev 18 function 5 not > configured > "Intel Xeon Scalable M2PCI" rev 0x04 at pci10 dev 21 function 0 not > configured > "Intel Xeon Scalable M2PCI" rev 0x04 at pci10 dev 22 function 0 not > configured > "Intel Xeon Scalable M2PCI" rev 0x04 at pci10 dev 22 function 4 not > configured > "Intel Xeon Scalable M2PCI" rev 0x04 at pci10 dev 23 function 0 not > configured > vmm0 at mainbus0: VMX/EPT > uhub1 at uhub0 port 9 configuration 1 interface 0 "Microchip Tech > USB2742" rev 2.10/2.07 addr2 > uhub2 at uhub0 port 14 configuration 1 interface 0 "vendor 0x1604 > product 0x10c0" rev 2.00/0.3 > uhub3 at uhub2 port 1 configuration 1 interface 0 "vendor 0x1604 product > 0x10c0" rev 2.00/0.04 > uhub4 at uhub2 port 4 configuration 1 interface 0 "vendor 0x1604 product > 0x10c0" rev 2.00/0.05 > uhub5 at uhub0 port 18 configuration 1 interface 0 "Microchip Tech > USB5742" rev 3.10/2.07 add6 > vscsi0 at root > scsibus5 at vscsi0: 256 targets > softraid0 at root > scsibus6 at softraid0: 256 targets > root on sd0a (dcdc93eef85cda4c.a) swap on sd0b dump on sd0b > Automatic boot in progress: starting file system checks. > <snap> >
