On Mon, May 20, 2024 at 11:53:26AM +0200, Martin Pieuchot wrote: > On 19/05/24(Sun) 23:50, Vitaliy Makkoveev wrote: > > > > > > > On 19 May 2024, at 22:05, Anthony J. Bentley <bent...@openbsd.org> wrote: > > > > > > Vitaliy Makkoveev writes: > > >>> On 17 May 2024, at 12:06, Stuart Henderson <s...@spacehopper.org> = > > >> wrote: > > >>> =20 > > >>> There are problems with wg(4) that people with some workloads have = > > >> been > > >>> seeing after upgrading past 7.3, though looking at this thread from = > > >> when > > >>> it last came up https://marc.info/?t=3D170940892700001&r=3D1&w=3D2 I'm = > > >> not > > >>> sure if we'd be expecting to see trouble on non-MP=E2=80=A6 > > >>> =20 > > >> > > >> We do. The problem is not MP related. > > >> > > >> Antony, does the diff [1] help? > > >> > > >> 1. https://marc.info/?l=3Dopenbsd-bugs&m=3D170980835807159&w=3D2 > > > > > > Crashes continue to occur with the same frequency after patching. > > > > > > > This could be vio(4) bug. Please try this [1] diff. > > > > 1. https://marc.info/?l=openbsd-tech&m=171588941332420&w=2 > > The traces all point to a use-after-free in a mbuf that has been through > the wg(4) machinery. The fact that using a SP system makes the crash > disappear points that this driver is not MP-safe and somehow there is a > race which ends up corrupting memory associated to mbufs.
But only for some kind of workload / packets. I've a machine that is running a service (HTTPS) behind a wireguard connexion on OpenBSD-current. It has been running stable for several months (I upgrade the machine almost every week). OpenBSD 7.5-current (GENERIC.MP) #76: Fri May 17 10:28:20 MDT 2024 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 2051219456 (1956MB) avail mem = 1968062464 (1876MB) random: good seed from bootblocks mpath0 at root scsibus0 at mpath0: 256 targets mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.8 @ 0x7b923000 (51 entries) bios0: vendor American Megatrends Inc. version "YB1007" date 08/17/2017 bios0: AZW Z83 II efi0 at bios0: UEFI 2.4 efi0: American Megatrends rev 0x5000b acpi0 at bios0: ACPI 5.0 acpi0: sleep states S0 S4 S5 acpi0: tables DSDT FACP APIC FPDT FIDT MCFG SSDT SSDT SSDT UEFI HPET SSDT SSDT SSDT SSDT TPM2 LPIT BCFG PRAM BGRT CSRT WDAT acpi0: wakeup devices acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: Intel(R) Atom(TM) x5-Z8350 CPU @ 1.44GHz, 1440.23 MHz, 06-4c-04, patch 00000411 cpu0: cpuid 1 edx=bfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> ecx=43d8e3bf<SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,MOVBE,POPCNT,DEADLINE,AES,RDRAND> cpu0: cpuid 6 eax=7<SENSOR,ARAT> ecx=9<EFFFREQ> cpu0: cpuid 7.0 ebx=2282<TSC_ADJUST,SMEP,ERMS> edx=c000400<MD_CLEAR,IBRS,IBPB,STIBP> cpu0: cpuid a vers=3, gp=2, gpwidth=40, ff=3, ffwidth=40 cpu0: cpuid 80000001 edx=28100800<NXE,RDTSCP,LONG> ecx=101<LAHF,3DNOWP> cpu0: cpuid 80000007 edx=100<ITSC> cpu0: MELTDOWN cpu0: 24KB 64b/line 6-way D-cache, 32KB 64b/line 8-way I-cache, 1MB 64b/line 16-way L2 cache cpu0: smt 0, core 0, package 0 mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges cpu0: apic clock running at 79MHz cpu0: mwait min=64, max=64, C-substates=0.2, IBE cpu1 at mainbus0: apid 2 (application processor) cpu1: Intel(R) Atom(TM) x5-Z8350 CPU @ 1.44GHz, 1440.46 MHz, 06-4c-04, patch 00000411 cpu1: smt 0, core 1, package 0 cpu2 at mainbus0: apid 4 (application processor) cpu2: Intel(R) Atom(TM) x5-Z8350 CPU @ 1.44GHz, 1440.75 MHz, 06-4c-04, patch 00000411 cpu2: smt 0, core 2, package 0 cpu3 at mainbus0: apid 6 (application processor) cpu3: Intel(R) Atom(TM) x5-Z8350 CPU @ 1.44GHz, 1440.78 MHz, 06-4c-04, patch 00000411 cpu3: smt 0, core 3, package 0 ioapic0 at mainbus0: apid 1 pa 0xfec00000, version 20, 115 pins acpimcfg0 at acpi0 acpimcfg0: addr 0xe0000000, bus 0-255 acpihpet0 at acpi0: 14318179 Hz acpiprt0 at acpi0: bus 0 (PCI0) acpiprt1 at acpi0: bus 1 (RP01) acpiprt2 at acpi0: bus -1 (RP02) acpiprt3 at acpi0: bus -1 (RP03) acpiprt4 at acpi0: bus -1 (RP04) "INT33A4" at acpi0 not configured iosf0 at acpi0 MBID: mbi chvgpio0 at acpi0 GPO1 uid 2 addr 0xfed88000/0x8000 irq 48, 59 pins chvgpio1 at acpi0 GPO3 uid 4 addr 0xfed98000/0x8000 irq 91, 55 pins dwiic0 at acpi0 I2C7 addr 0x91526000/0x1000 irq 38, sem iic0 at dwiic0 "INT33F4" at iic0 addr 0x34 not configured chvgpio2 at acpi0 GPO0 uid 1 addr 0xfed80000/0x8000 irq 49, 56 pins acpipci0 at acpi0 PCI0: 0x00000000 0x00000011 0x00000001 com0 at acpi0 IURT addr 0x3f8/0x8 irq 4: ns16550a, 16 byte fifo sdhc0 at acpi0 SDHA addr 0x9153a000/0x1000 irq 45 sdhc0: SDHC 3.00, 200 MHz base clock sdmmc0 at sdhc0: 8-bit, sd high-speed, mmc high-speed, ddr52, dma sdhc1 at acpi0 SDHB addr 0x91538000/0x1000 irq 46 chvgpio3 at acpi0 GPO2 uid 3 addr 0xfed90000/0x8000 irq 50, 24 pins sdhc1: SDHC 3.00, 200 MHz base clock sdmmc1 at sdhc1: 4-bit, sd high-speed, mmc high-speed, ddr52, dma sdhc2 at acpi0 SHC1 addr 0x91536000/0x1000 irq 47, gpio sdhc2: SDHC 3.00, 200 MHz base clock sdmmc2 at sdhc2: 4-bit, sd high-speed, mmc high-speed, ddr52, dma "INTL9C60" at acpi0 not configured "INTL9C60" at acpi0 not configured "8086228A" at acpi0 not configured "BCM2EA4" at acpi0 not configured "8086228A" at acpi0 not configured "8086228E" at acpi0 not configured "8086228E" at acpi0 not configured "8086228E" at acpi0 not configured dwiic1 at acpi0 I2C1 addr 0x91532000/0x1000 irq 32 iic1 at dwiic1 dwiic2 at acpi0 I2C2 addr 0x91530000/0x1000 irq 33 iic2 at dwiic2 "10EC5651" at iic2 addr 0x1a not configured dwiic3 at acpi0 I2C3 addr 0x9152e000/0x1000 irq 34 iic3 at dwiic3 dwiic4 at acpi0 I2C4 addr 0x9152c000/0x1000 irq 35 iic4 at dwiic4 dwiic5 at acpi0 I2C5 addr 0x9152a000/0x1000 irq 36 iic5 at dwiic5 dwiic6 at acpi0 I2C6 addr 0x91528000/0x1000 irq 37 iic6 at dwiic6 "808622A8" at acpi0 not configured acpicmos0 at acpi0 acpibtn0 at acpi0: PWRB chvgpio4 at acpi0 GPO4 uid 5 "ACPI0011" at acpi0 not configured "ACPI000C" at acpi0 not configured acpiac0 at acpi0: AC unit online "INT3400" at acpi0 not configured "INT3403" at acpi0 not configured tpm0 at acpi0 CTPM: unsupported TPM2 start method 2 acpicpu0 at acpi0: C2 bad (state 6 has no substates): C3 bad (state 7 has no substates): C1(1000@1 mwait.1), PSS acpicpu1 at acpi0: C2 bad (state 6 has no substates): C3 bad (state 7 has no substates): C1(1000@1 mwait.1), PSS acpicpu2 at acpi0: C2 bad (state 6 has no substates): C3 bad (state 7 has no substates): C1(1000@1 mwait.1), PSS acpicpu3 at acpi0: C2 bad (state 6 has no substates): C3 bad (state 7 has no substates): C1(1000@1 mwait.1), PSS acpipwrres0 at acpi0: ID3C, resource for ISP3 acpipwrres1 at acpi0: WWPR, resource for HS03, MDM1 acpipwrres2 at acpi0: WWPR, resource for HS13, MDM1 acpipwrres3 at acpi0: WWPR, resource for SSC1, MDM3 acpipwrres4 at acpi0: WWPR, resource for SSCW, MDM3 acpipwrres5 at acpi0: WWPR, resource for HSC1, MDM2 acpipwrres6 at acpi0: WWPR, resource for HSC3, MDM4 acpipwrres7 at acpi0: CLK3, resource for RTEK, RTK1, RTK2 acpipwrres8 at acpi0: CLK4 acpipwrres9 at acpi0: CLK2, resource for NFC2 acpipwrres10 at acpi0: CLK1 acpipwrres11 at acpi0: CLK0 acpipwrres12 at acpi0: CLK1 acpipwrres13 at acpi0: USBC, resource for XHC1, OTG1 acpipwrres14 at acpi0: P28X acpipwrres15 at acpi0: P18X acpipwrres16 at acpi0: P12X acpipwrres17 at acpi0: P28P acpipwrres18 at acpi0: P18P acpipwrres19 at acpi0: P19X acpipwrres20 at acpi0: P06X acpipwrres21 at acpi0: P28T acpipwrres22 at acpi0: P18D acpipwrres23 at acpi0: P18T acpipwrres24 at acpi0: P3P3 acpipwrres25 at acpi0: P12T acpipwrres26 at acpi0: P28W acpipwrres27 at acpi0: P18W acpipwrres28 at acpi0: P12W acpipwrres29 at acpi0: P33W acpipwrres30 at acpi0: P33X acpitz0 at acpi0: critical temperature is 90 degC acpivideo0 at acpi0: GFX0 acpivout0 at acpivideo0: DD01 acpivout1 at acpivideo0: DD02 acpivout2 at acpivideo0: DD03 acpivout3 at acpivideo0: DD04 acpivout4 at acpivideo0: DD05 acpivout5 at acpivideo0: DD06 acpivout6 at acpivideo0: DD07 acpivout7 at acpivideo0: DD08 cpu0: using VERW MDS workaround cpu0: Enhanced SpeedStep 1440 MHz: speeds: 1441, 1440, 1360, 1280, 1200, 1120, 1040, 960, 880, 800, 720, 640, 560, 480 MHz pci0 at mainbus0 bus 0 iosf1 at pci0 dev 0 function 0 "Intel Braswell Host" rev 0x36: mbi inteldrm0 at pci0 dev 2 function 0 "Intel HD Graphics" rev 0x36 drm0 at inteldrm0 inteldrm0: msi, CHERRYVIEW, gen 8 "Intel Braswell Power" rev 0x36 at pci0 dev 11 function 0 not configured xhci0 at pci0 dev 20 function 0 "Intel Braswell xHCI" rev 0x36: msi, xHCI 1.0 usb0 at xhci0: USB revision 3.0 uhub0 at usb0 configuration 1 interface 0 "Intel xHCI root hub" rev 3.00/1.00 addr 1 "Intel Braswell TXE" rev 0x36 at pci0 dev 26 function 0 not configured ppb0 at pci0 dev 28 function 0 "Intel Braswell PCIE" rev 0x36: msi pci1 at ppb0 bus 1 re0 at pci1 dev 0 function 0 "Realtek 8168" rev 0x0c: RTL8168G/8111G (0x4c00), msi, address 84:39:be:66:a1:0c rgephy0 at re0 phy 7: RTL8251 PHY, rev. 0 pcib0 at pci0 dev 31 function 0 "Intel Braswell PCU LPC" rev 0x36 isa0 at pcib0 isadma0 at isa0 pckbc0 at isa0 port 0x60/5 irq 1 irq 12 pckbd0 at pckbc0 (kbd slot) wskbd0 at pckbd0: console keyboard pcppi0 at isa0 port 0x61 spkr0 at pcppi0 vmm0 at mainbus0: VMX/EPT (using slow L1TF mitigation) efifb at mainbus0 not configured scsibus1 at sdmmc0: 2 targets, initiator 0 sd0 at scsibus1 targ 1 lun 0: <Sandisk, DF4032, 0000> sd0: 29820MB, 512 bytes/sector, 61071360 sectors uhub1 at uhub0 port 1 configuration 1 interface 0 "GenesysLogic USB2.0 Hub" rev 2.10/92.12 addr 2 bwfm0 at sdmmc1 function 1 manufacturer 0x02d0, product 0xa9bf at sdmmc1 function 2 not configured manufacturer 0x02d0, product 0xa9bf at sdmmc1 function 3 not configured uhidev0 at uhub1 port 1 configuration 1 interface 0 "Logitech USB Laser Mouse" rev 2.00/31.00 addr 3 uhidev0: iclass 3/1 ums0 at uhidev0: 8 buttons, Z and W dir wsmouse0 at ums0 mux 0 uhidev1 at uhub1 port 2 configuration 1 interface 0 "Fujitsu Component Type 6 Keyboard" rev 1.00/1.02 addr 4 uhidev1: iclass 3/1 ukbd0 at uhidev1: 8 variable keys, 6 key codes, country code 33 wskbd1 at ukbd0 mux 1 uhub2 at uhub0 port 8 configuration 1 interface 0 " " rev 3.00/92.12 addr 5 vscsi0 at root scsibus2 at vscsi0: 256 targets softraid0 at root scsibus3 at softraid0: 256 targets root on sd0a (3c9a89c702bc0dff.a) swap on sd0b dump on sd0b inteldrm0: 1920x1080, 32bpp wsdisplay0 at inteldrm0 mux 1: console (std, vt100 emulation), using wskbd0 wskbd1: connecting to wsdisplay0 wsdisplay0: screen 1-5 added (std, vt100 emulation) bwfm0: failed loadfirmware of file brcmfmac43455-sdio.txt # ifconfig wg0 wg0: flags=80c3<UP,BROADCAST,RUNNING,NOARP,MULTICAST> mtu 1420 index 5 priority 0 llprio 3 wgport 22261 wgrtable 1 wgpubkey <redacted> groups: wg egress inet xxx.xxx.xxx.3 netmask 0xffffffff inet6 xxxx:xxxx:xxxx::300::1 prefixlen 64 # pfctl -vsi Status: Enabled for 1 days 22:47:58 Debug: err Hostid: 0x5024b36f Checksum: 0x2e318795003ad15597e2bcb176967283 Interface Stats for wg0 IPv4 IPv6 Bytes In 11544219462 25656625948 Bytes Out 32415882461 5804695997 Packets In Passed 34457033 30533698 Blocked 14605 614 Packets Out Passed 51892935 26730708 Blocked 22 26