> Date: Tue, 26 Mar 2024 23:05:49 +0100
> From: Patrick Wildt <patr...@blueri.se>
>
> On Fri, Mar 01, 2024 at 12:00:29AM +0100, Alexander Bluhm wrote:
> > Hi,
> >
> > When doing flood ping transmit from a machine and simultaneously
> > ifconfig down/up in a loop, dwqe(4) interface driver crashes.
> >
> > dwqe_down() contains an interrupt barrier, but somehow it does not
> > work. Immediately after Xspllower() a transmit interrupt is
> > processed.
> >
> > bluhm
>
> Unfortunately I can't see it in the dmesg, but I wonder: Is it MSIs?
> Maybe the edge-triggered interrupt stays in the controller because it
> isn't cleared. But things you could try are:
>
> * Clear the IRQ status in addition to disabling them. This might not
> do something in case the MSI is already in the IRQ, there are no
> takebacks. But then maybe when the interrupt fires, the code path
> sees the cleared status and doesn't run the tx/rx proc.
> * Don't run TX/RX proc in case the interface is down?
Another thing... Is that intr_barrier() called while we're at
IPL_NET? That might not have the desired effect if intr_barrier()
runs on the same CPU that is handling the interrupts for the device.
And I fear that would be an issue in other drivers too...
> > kernel: protection fault trap, code=0
> > Stopped at m_tag_delete_chain+0x30: movq 0(%rsi),%rax
> >
> > ddb{0}> trace
> > m_tag_delete_chain(fffffd806bfa5300) at m_tag_delete_chain+0x30
> > m_free(fffffd806bfa5300) at m_free+0x9e
> > m_freem(fffffd806bfa5300) at m_freem+0x38
> > dwqe_tx_proc(ffff800000304800) at dwqe_tx_proc+0x194
> > dwqe_intr(ffff800000304800) at dwqe_intr+0x9b
> > intr_handler(ffff80003f86e760,ffff8000005f4f80) at intr_handler+0x72
> > Xintr_ioapic_edge36_untramp() at Xintr_ioapic_edge36_untramp+0x18f
> > Xspllower() at Xspllower+0x1d
> > dwqe_ioctl(ffff800000304870,80206910,ffff80003f86e990) at dwqe_ioctl+0x18c
> > ifioctl(fffffd81ffabe1e8,80206910,ffff80003f86e990,ffff80003f94e550) at
> > ifioctl+0x726
> > sys_ioctl(ffff80003f94e550,ffff80003f86eb50,ffff80003f86eac0) at
> > sys_ioctl+0x2af
> > syscall(ffff80003f86eb50) at syscall+0x55b
> > Xsyscall() at Xsyscall+0x128
> > end of kernel
> > end trace frame: 0x73ef48509270, count: -13
> >
> > ddb{0}> show register
> > rdi 0xfffffd806bfa5300
> > rsi 0xdeafbeaddeafbead
> > rbp 0xffff80003f86e5f0
> > rbx 0xf40
> > rdx 0
> > rcx 0
> > rax 0xab56 __ALIGN_SIZE+0x9b56
> > r8 0x90
> > r9 0x24634ac __kernel_rodata_phys+0x3624ac
> > r10 0xe676ed611cc13e4f
> > r11 0xd2619954b795f246
> > r12 0xffff800001110f48
> > r13 0xfffffd8072820000
> > r14 0xfffffd806bfa5300
> > r15 0xfffffd805f6def00
> > rip 0xffffffff81daae80 m_tag_delete_chain+0x30
> > cs 0x8
> > rflags 0x10282 __ALIGN_SIZE+0xf282
> > rsp 0xffff80003f86e5d0
> > ss 0x10
> > m_tag_delete_chain+0x30: movq 0(%rsi),%rax
> >
> > ddb{0}> x/s version
> > version: OpenBSD 7.5 (GENERIC.MP) #2: Thu Feb 29 23:42:26 CET
> > 2024\012
> > r...@ot50.obsd-lab.genua.de:/usr/src/sys/arch/amd64/compile/GENERIC.MP\012
> >
> > ddb{0}> ps
> > PID TID PPID UID S FLAGS WAIT COMMAND
> > *70039 16536 80360 0 7 0x8000003 ifconfig
> > 41531 214934 36719 51 3 0x8100033 netlock ping
> >
> > OpenBSD 7.5 (GENERIC.MP) #2: Thu Feb 29 23:42:26 CET 2024
> > r...@ot50.obsd-lab.genua.de:/usr/src/sys/arch/amd64/compile/GENERIC.MP
> > real mem = 8038207488 (7665MB)
> > avail mem = 7773556736 (7413MB)
> > random: good seed from bootblocks
> > mpath0 at root
> > scsibus0 at mpath0: 256 targets
> > mainbus0 at root
> > bios0 at mainbus0: SMBIOS rev. 3.3 @ 0x769c7000 (85 entries)
> > bios0: vendor American Megatrends Inc. version "1.02.10" date 06/27/2022
> > efi0 at bios0: UEFI 2.7
> > efi0: American Megatrends rev 0x50013
> > acpi0 at bios0: ACPI 6.2
> > acpi0: sleep states S0 S5
> > acpi0: tablesfg0: addr 0xc0000000, bus 0-255
> > acpihpet0 at acpi0: 19200000 Hz
> > acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
> > cpu0 at mainbus0: apid 0 (boot processor)
> > cpu0: Intel Atom(R) x6425RE Processor @ 1.90GHz, 1895.90 MHz, 06-96-01,
> > patch 00000017
> > cpu0:
> > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,RDRAND,NXE,RDTSCP,LONG,LAHF,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SMEP,ERMS,RDSEED,SMAP,CLFLUSHOPT,CLWB,PT,SHA,UMIP,WAITPKG,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,IBRS_ALL,SKIP_L1DFL,MDS_NO,IF_PSCHANGE,MISC_PKG_CT,ENERGY_FILT,FB_CLEAR,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
> > cpu0: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 1MB
> > 64b/line 12-way L2 cache, 4MB 64b/line 16-way L3 cache
> > cpu0: smt 0, core 0, package 0
> > mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
> > cpu0: apic clock running at 38MHz
> > cpu0: mwait min=64, max=64, C-substates=0.2.0.2.2.1.1.1, IBE
> > cpu1 at mainbus0: apid 2 (application processor)
> > cpu1: Intel Atom(R) x6425RE Processor @ 1.90GHz, 1895.90 MHz, 06-96-01,
> > patch 00000017
> > cpu1:
> > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,RDRAND,NXE,RDTSCP,LONG,LAHF,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SMEP,ERMS,RDSEED,SMAP,CLFLUSHOPT,CLWB,PT,SHA,UMIP,WAITPKG,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,IBRS_ALL,SKIP_L1DFL,MDS_NO,IF_PSCHANGE,MISC_PKG_CT,ENERGY_FILT,FB_CLEAR,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
> > cpu1: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 1MB
> > 64b/line 12-way L2 cache, 4MB 64b/line 16-way L3 cache
> > cpu1: smt 0, core 1, package 0
> > cpu2 at mainbus0: apid 4 (application processor)
> > cpu2: Intel Atom(R) x6425RE Processor @ 1.90GHz, 1895.90 MHz, 06-96-01,
> > patch 00000017
> > cpu2:
> > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,RDRAND,NXE,RDTSCP,LONG,LAHF,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SMEP,ERMS,RDSEED,SMAP,CLFLUSHOPT,CLWB,PT,SHA,UMIP,WAITPKG,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,IBRS_ALL,SKIP_L1DFL,MDS_NO,IF_PSCHANGE,MISC_PKG_CT,ENERGY_FILT,FB_CLEAR,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
> > cpu2: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 1MB
> > 64b/line 12-way L2 cache, 4MB 64b/line 16-way L3 cache
> > cpu2: smt 0, core 2, package 0
> > cpu3 at mainbus0: apid 6 (application processor)
> > cpu3: Intel Atom(R) x6425RE Processor @ 1.90GHz, 1895.90 MHz, 06-96-01,
> > patch 00000017
> > cpu3:
> > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,RDRAND,NXE,RDTSCP,LONG,LAHF,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SMEP,ERMS,RDSEED,SMAP,CLFLUSHOPT,CLWB,PT,SHA,UMIP,WAITPKG,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,IBRS_ALL,SKIP_L1DFL,MDS_NO,IF_PSCHANGE,MISC_PKG_CT,ENERGY_FILT,FB_CLEAR,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
> > cpu3: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 1MB
> > 64b/line 12-way L2 cache, 4MB 64b/line 16-way L3 cache
> > cpu3: smt 0, core 3, package 0
> > ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 20, 120 pins
> > acpiprt0 at acpi0: bus 0 (PC00)
> > acpiprt1 at acpi0: bus -1 (PEG0)
> > acpiprt2 at acpi0: bus -1 (PEG1)
> > acpiprt3 at acpi0: bus -1 (PEG2)
> > acpiprt4 at acpi0: bus -1 (PEG3)
> > acpiprt5 at acpi0: bus 1 (RP01)
> > acpiprt6 at acpi0: bus -1 (RP02)
> > acpiprt7 at acpi0: bus 2 (RP03)
> > acpiprt8 at acpi0: bus 6 (RP04)
> > acpiprt9 at acpi0: bus -1 (RP05)
> > acpiprt10 at acpi0: bus -1 (RP06)
> > acpiprt11 at acpi0: bus 7 (RP07)
> > acpipci0 at acpi0 PC00: 0x00000010 0x00000011 0x00000000
> > com0 at acpi0 UAR1 addr 0x3f8/0x8 irq 4: ns16550a, 16 byte fifo
> > com0: console
> > com1 at acpi0 UAR2 addr 0x2f8/0x8 irq 3: ns16550a, 16 byte fifo
> > "INTC1035" at acpi0 not configured
> > acpiac0 at acpi0: AC unit online
> > "ACPI000E" at acpi0 not configured
> > "INTC1020" at acpi0 not configured
> > "INTC1020" at acpi0 not configured
> > "INTC1020" at acpi0 not configured
> > "INTC1020" at acpi0 not configured
> > "INTC1020" at acpi0 not configured
> > "PNP0C14" at acpi0 not configured
> > acpibtn0 at acpi0: SLPB
> > acpibtn1 at acpi0: PWRB(wakeup)
> > "INTC1033" at acpi0 not configured
> > "INTC1033" at acpi0 not configured
> > "PNP0C14" at acpi0 not configured
> > "PNP0C14" at acpi0 not configured
> > "INT33A1" at acpi0 not configured
> > acpihid0 at acpi0: HIDDacpihid0: failed getting button array capability
> > , 5 button array
> > "MSFT8000" at acpi0 not configured
> > tpm0 at acpi0 TPM_ 2.0 (TIS) addr 0xfed40000/0x5000, device 0x001b15d1 rev
> > 0x16
> > "ADS0345" at acpi0 not configured
> > "IFX3100" at acpi0 not configured
> > "TXNW1010" at acpi0 not configured
> > acpipwrres0 at acpi0: WRST
> > acpicpu0 at acpi0: C3(200@1048 mwait.1@0x60), C2(200@253 mwait.1@0x31),
> > C1(1000@1 mwait.1), PSS
> > acpicpu1 at acpi0: C3(200@1048 mwait.1@0x60), C2(200@253 mwait.1@0x31),
> > C1(1000@1 mwait.1), PSS
> > acpicpu2 at acpi0: C3(200@1048 mwait.1@0x60), C2(200@253 mwait.1@0x31),
> > C1(1000@1 mwait.1), PSS
> > acpicpu3 at acpi0: C3(200@1048 mwait.1@0x60), C2(200@253 mwait.1@0x31),
> > C1(1000@1 mwait.1), PSS
> > acpipwrres1 at acpi0: FN00, resource for FAN0
> > acpipwrres2 at acpi0: FN01, resource for FAN1
> > acpipwrres3 at acpi0: FN02, resource for FAN2
> > acpipwrres4 at acpi0: FN03, resource for FAN3
> > acpipwrres5 at acpi0: FN04, resource for FAN4
> > acpitz0 at acpi0: critical temperature is 127 degC
> > acpipwrres6 at acpi0: PIN_
> > cpu0: Enhanced SpeedStep 1895 MHz: speeds: 1901, 1900, 1800, 1700, 1600,
> > 1500, 1400, 1300, 1200, 1100, 1000, 900, 800 MHz
> > pci0 at mainbus0 bus 0
> > 0:31:5: mem address conflict 0xfe010000/0x1000
> > pchb0 at pci0 dev 0 function 0 "Intel Elkhart Lake Host" rev 0x01
> > inteldrm0 at pci0 dev 2 function 0 "Intel UHD Graphics" rev 0x01
> > drm0 at inteldrm0
> > inteldrm0: msi, ELKHARTLAKE, gen 11
> > "Intel Elkhart Lake GNA" rev 0x01 at pci0 dev 8 function 0 not configured
> > xhci0 at pci0 dev 20 function 0 "Intel Elkhart Lake xHCI" rev 0x11: msi,
> > xHCI 1.20
> > usb0 at xhci0: USB revision 3.0
> > uhub0 at usb0 configuration 1 interface 0 "Intel xHCI root hub" rev
> > 3.00/1.00 addr 1
> > "Intel Elkhart Lake SRAM" rev 0x11 at pci0 dev 20 function 2 not configured
> > dwiic0 at pci0 dev 21 function 0 "Intel Elkhart Lake I2C" rev 0x11: apic 2
> > int 27
> > iic0 at dwiic0
> > dwiic1 at pci0 dev 21 function 1 "Intel Elkhart Lake I2C" rev 0x11: apic 2
> > int 28
> > iic1 at dwiic1
> > dwiic2 at pci0 dev 21 function 2 "Intel Elkhart Lake I2C" rev 0x11: apic 2
> > int 29
> > iic2 at dwiic2
> > dwiic3 at pci0 dev 21 function 3 "Intel Elkhart Lake I2C" rev 0x11: apic 2
> > int 30
> > iic3 at dwiic3
> > "Intel Elkhart Lake HECI" rev 0x11 at pci0 dev 22 function 0 not configured
> > ahci0 at pci0 dev 23 function 0 "Intel Elkhart Lake AHCI" rev 0x11: msi,
> > AHCI 1.3.1
> > ahci0: port 0: 6.0Gb/s
> > ahci0: PHY offline on port 1
> > scsibus1 at ahci0: 32 targets
> > sd0 at scsibus1 targ 0 lun 0: <ATA, innodisk M.2(S42, S206>
> > naa.524693e000367188
> > sd0: 57241MB, 512 bytes/sector, 117231408 sectors, thin
> > "Intel Elkhart Lake I2C" rev 0x11 at pci0 dev 24 function 0 not configured
> > "Intel Elkhart Lake CAN" rev 0x11 at pci0 dev 24 function 1 not configured
> > "Intel Elkhart Lake CAN" rev 0x11 at pci0 dev 24 function 2 not configured
> > dwiic4 at pci0 dev 25 function 0 "Intel Elkhart Lake I2C" rev 0x11: apic 2
> > int 31
> > iic4 at dwiic4
> > "Intel Elkhart Lake UART" rev 0x11 at pci0 dev 25 function 2 not configured
> > "Intel Elkhart Lake I2C" rev 0x11 at pci0 dev 27 function 0 not configured
> > ppb0 at pci0 dev 28 function 0 "Intel Elkhart Lake PCIE" rev 0x11
> > pci1 at ppb0 bus 1
> > ppb1 at pci0 dev 28 function 2 "Intel Elkhart Lake PCIE" rev 0x11: msi
> > pci2 at ppb1 bus 2
> > ppb2 at pci2 dev 0 function 0 vendor "Pericom", unknown product 0x2303 rev
> > 0x05
> > pci3 at ppb2 bus 3
> > ppb3 at pci3 dev 1 function 0 vendor "Pericom", unknown product 0x2303 rev
> > 0x05: msi
> > pci4 at ppb3 bus 4
> > em0 at pci4 dev 0 function 0 "Intel I210" rev 0x03: msi, address
> > f8:02:78:21:a4:35
> > ppb4 at pci3 dev 2 function 0 vendor "Pericom", unknown product 0x2303 rev
> > 0x05: msi
> > pci5 at ppb4 bus 5
> > em1 at pci5 dev 0 function 0 "Intel I210" rev 0x03: msi, address
> > f8:02:78:21:a4:36
> > ppb5 at pci0 dev 28 function 3 "Intel Elkhart Lake PCIE" rev 0x11: msi
> > pci6 at ppb5 bus 6
> > iwx0 at pci6 dev 0 function 0 "Intel Wi-Fi 6 AX210" rev 0x1a, msix
> > ppb6 at pci0 dev 28 function 6 "Intel Elkhart Lake PCIE" rev 0x11: msi
> > pci7 at ppb6 bus 7
> > igc0 at pci7 dev 0 function 0 "Intel I225-IT" rev 0x03, msix, 4 queues,
> > address 34:73:5a:ff:a5:b1
> > "Intel Elkhart Lake LH2OSE" rev 0x11 at pci0 dev 29 function 0 not
> > configured
> > dwqe0 at pci0 dev 29 function 1 "Intel Elkhart Lake Ethernet" rev 0x11: rev
> > 0x52, address 34:73:5a:ff:a5:b2
> > eephy0 at dwqe0 phy 1: 88E1512 10/100/1000 PHY, rev. 1
> > "Intel Elkhart Lake Ethernet" rev 0x11 at pci0 dev 29 function 2 not
> > configured
> > "Intel Elkhart Lake UART" rev 0x11 at pci0 dev 30 function 0 not configured
> > pcib0 at pci0 dev 31 function 0 "Intel Elkhart Lake eSPI" rev 0x11
> > azalia0 at pci0 dev 31 function 3 "Intel Elkhart Lake HD Audio" rev 0x11:
> > msi
> > azalia0: codecs: IDT/0x76e0, Intel/0x281b, using IDT/0x76e0
> > audio0 at azalia0
> > ichiic0 at pci0 dev 31 function 4 "Intel Elkhart Lake SMBus" rev 0x11: apic
> > 2 int 16
> > iic5 at ichiic0
> > "Intel Elkhart Lake SPI" rev 0x11 at pci0 dev 31 function 5 not configured
> > isa0 at pcib0
> > isadma0 at isa0
> > pcppi0 at isa0 port 0x61
> > spkr0 at pcppi0
> > vmm0 at mainbus0: VMX/EPT
> > efifb at mainbus0 not configured
> > uhub1 at uhub0 port 3 configuration 1 interface 0 "Texas Instruments
> > product 0x8027" rev 2.10/1.10 addr 2
> > ugen0 at uhub0 port 5 "Intel Bluetooth" rev 2.01/0.00 addr 3
> > uhub2 at uhub0 port 15 configuration 1 interface 0 "Texas Instruments
> > product 0x8025" rev 3.00/1.10 addr 4
> > vscsi0 at root
> > scsibus2 at vscsi0: 256 targets
> > softraid0 at root
> > scsibus3 at softraid0: 256 targets
> > root on sd0a (d6d91d6e9bdd6dbb.a) swap on sd0b dump on sd0b
> > inteldrm0: 1024x768, 32bpp
> > wsdisplay0 at inteldrm0 mux 1
> > wsdisplay0: screen 0-5 added (std, vt100 emulation)
> > iwx0: hw rev 0x420, fw 77.f92b5fed.0, pnvm 181407b3, address
> > 8c:b8:7e:ee:89:0f
> > <4>arp: attempt to add entry for 10.0.1.1 on igc0 by 00:30:48:57:ed:0a on
> > dwqe0
> >
>
>
