On 17.6.2020. 12:45, Hrvoje Popovski wrote: > On 17.6.2020. 11:27, Hrvoje Popovski wrote: >> On 17.6.2020. 10:36, David Gwynne wrote: >>> this is an updated version of a diff from christiano haesbaert by way of >>> mpi@ to enable the use of multiple tx and rx rings with msi-x. >>> >>> the high level description is that that driver checks to see if msix is >>> available, and if so how many vectors it has. it then gets an intrmap >>> based on that information, and bumps the number of queues to the number >>> of cpus that intrmap says are available. >>> >>> once the queues are allocated, it then iterates over them and wires up >>> interrupts to the cpus provided by the intrmap. >>> >>> im happy for people to try this out, but i can't commit it until all the >>> architectures that ix(4) is enabled on support the APIs that it's using. >>> this basically means it'll work on amd64 (and a little bit on i386), but >>> not much else. please hold back your tears and cries of anguish. >>> >>> thanks to christiano and mpi for doing most of the work leading up to >>> this diff :) >> >> Hi, >> >> first, thank you all for mq work :) >> >> with this diff, if i'm sending traffic over ix and at the same time >> execute ifconfig ix down/up, forwarding stops until i stop generator, >> wait for few seconds and execute ifconfig ix down/up few times and than >> forwarding start normally >
in vmstat i should see ix0:0-5 and ix1:0-5 ? r620-1# vmstat -i interrupt total rate irq0/clock 3985752 599 irq0/ipi 3462063 520 irq144/acpi0 4 0 irq114/ix0:0 8042709 1209 irq115/ix0:1 2906070 437 irq116/ix0:2 1975350 297 irq117/ix0:3 849089681 127721 irq118/ix0:4 4441608 668 irq119/ix0:5 4330871 651 irq120/ix0 10 0 irq121/ix1:0 43209056 6499 irq127/ix1 16 0 irq97/mfi0 36846 5 irq132/ixl2 7 0 irq133/ixl2:0 459 0 irq134/ixl3 7 0 irq135/ixl3:0 451 0 irq99/ehci0 139 0 irq136/em0 18637 2 irq137/em1 451 0 irq100/ehci1 28 0 irq101/ahci0 1 0 irq146/com1 4411 0 Total 921504627 138613 > dmesg > > ix0 at pci1 dev 0 function 0 "Intel 82599" rev 0x01, msix, 6 queues, > address ec:f4:bb:da:f7:f8 > ix1 at pci1 dev 0 function 1 "Intel 82599" rev 0x01, msix, 6 queues, > address ec:f4:bb:da:f7:fa > > > > OpenBSD 6.7-current (GENERIC.MP) #71: Wed Jun 17 10:53:55 CEST 2020 > [email protected]:/sys/arch/amd64/compile/GENERIC.MP > real mem = 17115840512 (16322MB) > avail mem = 16582168576 (15813MB) > random: good seed from bootblocks > mpath0 at root > scsibus0 at mpath0: 256 targets > mainbus0 at root > bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xcf42c000 (99 entries) > bios0: vendor Dell Inc. version "2.9.0" date 12/06/2019 > bios0: Dell Inc. PowerEdge R620 > acpi0 at bios0: ACPI 3.0 > acpi0: sleep states S0 S4 S5 > acpi0: tables DSDT FACP APIC SPCR HPET DMAR MCFG WDAT SLIC ERST HEST > BERT EINJ TCPA PC__ SRAT SSDT > acpi0: wakeup devices PCI0(S5) > acpitimer0 at acpi0: 3579545 Hz, 24 bits > acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat > cpu0 at mainbus0: apid 4 (boot processor) > cpu0: Intel(R) Xeon(R) CPU E5-2643 v2 @ 3.50GHz, 3600.48 MHz, 06-3e-04 > cpu0: > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN > cpu0: 256KB 64b/line 8-way L2 cache > cpu0: smt 0, core 2, package 0 > mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges > cpu0: apic clock running at 99MHz > cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE > cpu1 at mainbus0: apid 6 (application processor) > cpu1: Intel(R) Xeon(R) CPU E5-2643 v2 @ 3.50GHz, 3600.00 MHz, 06-3e-04 > cpu1: > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN > cpu1: 256KB 64b/line 8-way L2 cache > cpu1: smt 0, core 3, package 0 > cpu2 at mainbus0: apid 8 (application processor) > cpu2: Intel(R) Xeon(R) CPU E5-2643 v2 @ 3.50GHz, 3600.00 MHz, 06-3e-04 > cpu2: > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN > cpu2: 256KB 64b/line 8-way L2 cache > cpu2: smt 0, core 4, package 0 > cpu3 at mainbus0: apid 16 (application processor) > cpu3: Intel(R) Xeon(R) CPU E5-2643 v2 @ 3.50GHz, 3600.00 MHz, 06-3e-04 > cpu3: > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN > cpu3: 256KB 64b/line 8-way L2 cache > cpu3: smt 0, core 8, package 0 > cpu4 at mainbus0: apid 18 (application processor) > cpu4: Intel(R) Xeon(R) CPU E5-2643 v2 @ 3.50GHz, 3600.01 MHz, 06-3e-04 > cpu4: > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN > cpu4: 256KB 64b/line 8-way L2 cache > cpu4: smt 0, core 9, package 0 > cpu5 at mainbus0: apid 20 (application processor) > cpu5: Intel(R) Xeon(R) CPU E5-2643 v2 @ 3.50GHz, 3600.00 MHz, 06-3e-04 > cpu5: > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN > cpu5: 256KB 64b/line 8-way L2 cache > cpu5: smt 0, core 10, package 0 > ioapic0 at mainbus0: apid 0 pa 0xfec00000, version 20, 24 pins > ioapic1 at mainbus0: apid 1 pa 0xfec3f000, version 20, 24 pins, remapped > acpihpet0 at acpi0: 14318179 Hz > acpimcfg0 at acpi0 > acpimcfg0: addr 0xe0000000, bus 0-255 > acpiprt0 at acpi0: bus 0 (PCI0) > acpiprt1 at acpi0: bus 1 (PEX1) > acpiprt2 at acpi0: bus -1 (PE1C) > acpiprt3 at acpi0: bus 3 (PEX2) > acpiprt4 at acpi0: bus 2 (PEX3) > acpiprt5 at acpi0: bus 4 (PEX4) > acpiprt6 at acpi0: bus -1 (PEX5) > acpiprt7 at acpi0: bus 8 (PEX6) > acpiprt8 at acpi0: bus 7 (PEX7) > acpicpu0 at acpi0: C1(@1 halt!) > acpicpu1 at acpi0: C1(@1 halt!) > acpicpu2 at acpi0: C1(@1 halt!) > acpicpu3 at acpi0: C1(@1 halt!) > acpicpu4 at acpi0: C1(@1 halt!) > acpicpu5 at acpi0: C1(@1 halt!) > acpipci0 at acpi0 PCI0: 0x00000000 0x00000011 0x00000001 > extent `acpipci0 pcibus' (0x0 - 0xff), flags=0 > 0x3f - 0xff > extent `pciio' (0x0 - 0xffffffff), flags=0 > 0x10000 - 0xffffffff > extent `pcimem' (0x0 - 0xffffffffffffffff), flags=0 > 0x0 - 0x9bfff > 0xa0000 - 0xcfffffff > 0xe0000000 - 0xefffffff > 0xfe000000 - 0xffffffff > 0x40000000000 - 0xffffffffffffffff > acpicmos0 at acpi0 > acpipci1 at acpi0 P0B1: 0x00000000 0x00000011 0x00000001 > extent `acpipci1 pcibus' (0x0 - 0xff), flags=0 > 0x0 - 0x3e > 0x40 - 0xff > extent `pciio' (0x0 - 0xffffffff), flags=0 > 0x10000 - 0xffffffff > extent `pcimem' (0x0 - 0xffffffffffffffff), flags=0 > 0x0 - 0x9bfff > 0xa0000 - 0xcfffffff > 0xe0000000 - 0xefffffff > 0xfe000000 - 0xffffffff > 0x40000000000 - 0xffffffffffffffff > acpipci2 at acpi0 P1B1: 0x00000000 0x00000011 0x00000001 > extent `acpipci2 pcibus' (0x0 - 0xff), flags=0 > 0x0 - 0x7e > 0x80 - 0xff > extent `pciio' (0x0 - 0xffffffff), flags=0 > 0x10000 - 0xffffffff > extent `pcimem' (0x0 - 0xffffffffffffffff), flags=0 > 0x0 - 0x9bfff > 0xa0000 - 0xcfffffff > 0xe0000000 - 0xefffffff > 0xfe000000 - 0xffffffff > 0x40000000000 - 0xffffffffffffffff > "PNP0C14" at acpi0 not configured > ipmi at mainbus0 not configured > cpu0: using VERW MDS workaround (except on vmm entry) > pci0 at mainbus0 bus 0 > pchb0 at pci0 dev 0 function 0 "Intel E5 v2 Host" rev 0x04 > ppb0 at pci0 dev 1 function 0 "Intel E5 v2 PCIE" rev 0x04 > pci1 at ppb0 bus 1 > 1:0:1: rom address conflict 0xd8000000/0x80000 > ix0 at pci1 dev 0 function 0 "Intel 82599" rev 0x01, msix, 6 queues, > address ec:f4:bb:da:f7:f8 > ix1 at pci1 dev 0 function 1 "Intel 82599" rev 0x01, msix, 6 queues, > address ec:f4:bb:da:f7:fa > ppb1 at pci0 dev 2 function 0 "Intel E5 v2 PCIE" rev 0x04: msi > pci2 at ppb1 bus 3 > ppb2 at pci0 dev 2 function 2 "Intel E5 v2 PCIE" rev 0x04 > pci3 at ppb2 bus 2 > mfi0 at pci3 dev 0 function 0 "Symbios Logic MegaRAID SAS2008" rev 0x03: > apic 1 int 10 > mfi0: "PERC H310 Mini", firmware 20.13.3-0001 > scsibus1 at mfi0: 16 targets > sd0 at scsibus1 targ 0 lun 0: <DELL, PERC H310, 2.12> > naa.6c81f660f14398001af3a49619e0d82f > sd0: 285568MB, 512 bytes/sector, 584843264 sectors > scsibus2 at mfi0: 256 targets > ppb3 at pci0 dev 3 function 0 "Intel E5 v2 PCIE" rev 0x04: msi > pci4 at ppb3 bus 4 > 4:0:1: rom address conflict 0xda000000/0x80000 > 4:0:2: rom address conflict 0xda000000/0x80000 > 4:0:3: rom address conflict 0xda000000/0x80000 > ixl0 at pci4 dev 0 function 0 "Intel X710 SFP+" rev 0x01: port 3, FW > 7.3.60988 API 1.10, msix, 1 queue, address 3c:fd:fe:b3:26:c0 > ixl1 at pci4 dev 0 function 1 "Intel X710 SFP+" rev 0x01: port 2, FW > 7.3.60988 API 1.10, msix, 1 queue, address 3c:fd:fe:b3:26:c1 > ixl2 at pci4 dev 0 function 2 "Intel X710 SFP+" rev 0x01: port 0, FW > 7.3.60988 API 1.10, msix, 1 queue, address 3c:fd:fe:b3:26:c2 > ixl3 at pci4 dev 0 function 3 "Intel X710 SFP+" rev 0x01: port 1, FW > 7.3.60988 API 1.10, msix, 1 queue, address 3c:fd:fe:b3:26:c3 > "Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 0 not configured > "Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 1 not configured > "Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 2 not configured > "Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 3 not configured > "Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 4 not configured > "Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 5 not configured > "Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 6 not configured > "Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 7 not configured > "Intel E5 v2 Address Map" rev 0x04 at pci0 dev 5 function 0 not configured > "Intel E5 v2 IIO RAS" rev 0x04 at pci0 dev 5 function 2 not configured > ppb4 at pci0 dev 17 function 0 "Intel C600 Virtual PCIE" rev 0x05 > pci5 at ppb4 bus 5 > "Intel C600 MEI" rev 0x05 at pci0 dev 22 function 0 not configured > "Intel C600 MEI" rev 0x05 at pci0 dev 22 function 1 not configured > ehci0 at pci0 dev 26 function 0 "Intel C600 USB" rev 0x05: apic 0 int 23 > usb0 at ehci0: USB revision 2.0 > uhub0 at usb0 configuration 1 interface 0 "Intel EHCI root hub" rev > 2.00/1.00 addr 1 > ppb5 at pci0 dev 28 function 0 "Intel C600 PCIE" rev 0xb5 > pci6 at ppb5 bus 6 > ppb6 at pci0 dev 28 function 4 "Intel C600 PCIE" rev 0xb5 > pci7 at ppb6 bus 7 > em0 at pci7 dev 0 function 0 "Intel I350" rev 0x01: msi, address > ec:f4:bb:da:f7:fc > em1 at pci7 dev 0 function 1 "Intel I350" rev 0x01: msi, address > ec:f4:bb:da:f7:fd > ppb7 at pci0 dev 28 function 7 "Intel C600 PCIE" rev 0xb5 > pci8 at ppb7 bus 8 > ppb8 at pci8 dev 0 function 0 "Renesas SH7757 PCIE Switch" rev 0x00 > pci9 at ppb8 bus 9 > ppb9 at pci9 dev 0 function 0 "Renesas SH7757 PCIE Switch" rev 0x00 > pci10 at ppb9 bus 10 > ppb10 at pci10 dev 0 function 0 "Renesas SH7757 PCIE-PCI" rev 0x00 > pci11 at ppb10 bus 11 > vga1 at pci11 dev 0 function 0 "Matrox MGA G200eR" rev 0x00 > wsdisplay at vga1 not configured > ppb11 at pci9 dev 1 function 0 "Renesas SH7757 PCIE Switch" rev 0x00 > pci12 at ppb11 bus 12 > ehci1 at pci0 dev 29 function 0 "Intel C600 USB" rev 0x05: apic 0 int 22 > usb1 at ehci1: USB revision 2.0 > uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev > 2.00/1.00 addr 1 > ppb12 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0xa5 > pci13 at ppb12 bus 13 > pcib0 at pci0 dev 31 function 0 "Intel C600 LPC" rev 0x05 > ahci0 at pci0 dev 31 function 2 "Intel C600 AHCI" rev 0x05: msi, AHCI 1.3 > ahci0: port 4: 1.5Gb/s > scsibus3 at ahci0: 32 targets > cd0 at scsibus3 targ 4 lun 0: <TSSTcorp, DVD+-RW SU-208FB, D150> removable > isa0 at pcib0 > isadma0 at isa0 > com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo > com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo > com1: console > pcppi0 at isa0 port 0x61 > spkr0 at pcppi0 > pci14 at mainbus0 bus 63 > "Intel E5 v2 QPI Link" rev 0x04 at pci14 dev 8 function 0 not configured > "Intel E5 v2 QPI Link" rev 0x04 at pci14 dev 9 function 0 not configured > "Intel E5 v2 PCU" rev 0x04 at pci14 dev 10 function 0 not configured > "Intel E5 v2 PCU" rev 0x04 at pci14 dev 10 function 1 not configured > "Intel E5 v2 PCU" rev 0x04 at pci14 dev 10 function 2 not configured > "Intel E5 v2 PCU" rev 0x04 at pci14 dev 10 function 3 not configured > "Intel E5 v2 UBOX" rev 0x04 at pci14 dev 11 function 0 not configured > "Intel E5 v2 UBOX" rev 0x04 at pci14 dev 11 function 3 not configured > "Intel E5 v2 Unicast" rev 0x04 at pci14 dev 12 function 0 not configured > "Intel E5 v2 Unicast" rev 0x04 at pci14 dev 12 function 1 not configured > "Intel E5 v2 Unicast" rev 0x04 at pci14 dev 12 function 2 not configured > "Intel E5 v2 Unicast" rev 0x04 at pci14 dev 12 function 3 not configured > "Intel E5 v2 Unicast" rev 0x04 at pci14 dev 12 function 4 not configured > "Intel E5 v2 Unicast" rev 0x04 at pci14 dev 13 function 0 not configured > "Intel E5 v2 Unicast" rev 0x04 at pci14 dev 13 function 1 not configured > "Intel E5 v2 Unicast" rev 0x04 at pci14 dev 13 function 2 not configured > "Intel E5 v2 Unicast" rev 0x04 at pci14 dev 13 function 3 not configured > "Intel E5 v2 Unicast" rev 0x04 at pci14 dev 13 function 4 not configured > "Intel E5 v2 Home Agent" rev 0x04 at pci14 dev 14 function 0 not configured > "Intel E5 v2 Home Agent" rev 0x04 at pci14 dev 14 function 1 not configured > "Intel E5 v2 TA" rev 0x04 at pci14 dev 15 function 0 not configured > "Intel E5 v2 RAS" rev 0x04 at pci14 dev 15 function 1 not configured > "Intel E5 v2 TAD" rev 0x04 at pci14 dev 15 function 2 not configured > "Intel E5 v2 TAD" rev 0x04 at pci14 dev 15 function 3 not configured > "Intel E5 v2 TAD" rev 0x04 at pci14 dev 15 function 4 not configured > "Intel E5 v2 TAD" rev 0x04 at pci14 dev 15 function 5 not configured > "Intel E5 v2 Thermal" rev 0x04 at pci14 dev 16 function 0 not configured > "Intel E5 v2 Thermal" rev 0x04 at pci14 dev 16 function 1 not configured > "Intel E5 v2 Error" rev 0x04 at pci14 dev 16 function 2 not configured > "Intel E5 v2 Error" rev 0x04 at pci14 dev 16 function 3 not configured > "Intel E5 v2 Thermal" rev 0x04 at pci14 dev 16 function 4 not configured > "Intel E5 v2 Thermal" rev 0x04 at pci14 dev 16 function 5 not configured > "Intel E5 v2 Error" rev 0x04 at pci14 dev 16 function 7 not configured > "Intel E5 v2 R2PCIE" rev 0x04 at pci14 dev 19 function 0 not configured > "Intel E5 v2 QPI Link Monitor" rev 0x04 at pci14 dev 19 function 1 not > configured > "Intel E5 v2 QPI" rev 0x04 at pci14 dev 19 function 4 not configured > "Intel E5 v2 QPI Link Monitor" rev 0x04 at pci14 dev 19 function 5 not > configured > "Intel E5 v2 SAD" rev 0x04 at pci14 dev 22 function 0 not configured > "Intel E5 v2 Broadcast" rev 0x04 at pci14 dev 22 function 1 not configured > "Intel E5 v2 Broadcast" rev 0x04 at pci14 dev 22 function 2 not configured > pci15 at mainbus0 bus 127 > vmm0 at mainbus0: VMX/EPT > uhub2 at uhub0 port 1 configuration 1 interface 0 "Intel Rate Matching > Hub" rev 2.00/0.00 addr 2 > uhub3 at uhub2 port 6 configuration 1 interface 0 "no manufacturer > Gadget USB HUB" rev 2.00/0.00 addr 3 > uhidev0 at uhub3 port 1 configuration 1 interface 0 "Avocent > Keyboard/Mouse Function" rev 2.00/0.00 addr 4 > uhidev0: iclass 3/1 > ukbd0 at uhidev0: 8 variable keys, 6 key codes > wskbd0 at ukbd0 mux 1 > uhidev1 at uhub3 port 1 configuration 1 interface 1 "Avocent > Keyboard/Mouse Function" rev 2.00/0.00 addr 4 > uhidev1: iclass 3/1 > ums0 at uhidev1: 3 buttons, Z dir > wsmouse0 at ums0 mux 0 > uhidev2 at uhub3 port 1 configuration 1 interface 2 "Avocent > Keyboard/Mouse Function" rev 2.00/0.00 addr 4 > uhidev2: iclass 3/1 > ums1 at uhidev2: 3 buttons, Z dir > wsmouse1 at ums1 mux 0 > uhub4 at uhub1 port 1 configuration 1 interface 0 "Intel Rate Matching > Hub" rev 2.00/0.00 addr 2 > vscsi0 at root > scsibus4 at vscsi0: 256 targets > softraid0 at root > scsibus5 at softraid0: 256 targets > root on sd0a (38afb4d76f75540e.a) swap on sd0b dump on sd0b > wskbd0 detached > ukbd0 detached > uhidev0 detached > wsmouse0 detached > ums0 detached > uhidev1 detached > wsmouse1 detached > ums1 detached > uhidev2 detached > uhidev0 at uhub3 port 1 configuration 1 interface 0 "Avocent > Keyboard/Mouse Function" rev 2.00/0.00 addr 4 > uhidev0: iclass 3/1 > ukbd0 at uhidev0: 8 variable keys, 6 key codes > wskbd0 at ukbd0 mux 1 > uhidev1 at uhub3 port 1 configuration 1 interface 1 "Avocent > Keyboard/Mouse Function" rev 2.00/0.00 addr 4 > uhidev1: iclass 3/1 > ums0 at uhidev1: 3 buttons, Z dir > wsmouse0 at ums0 mux 0 > uhidev2 at uhub3 port 1 configuration 1 interface 2 "Avocent > Keyboard/Mouse Function" rev 2.00/0.00 addr 4 > uhidev2: iclass 3/1 > ums1 at uhidev2: 3 buttons, Z dir > wsmouse1 at ums1 mux 0 >
