On Mon, May 20, 2024 at 11:53:26AM +0200, Martin Pieuchot wrote:
> On 19/05/24(Sun) 23:50, Vitaliy Makkoveev wrote:
> >
> >
> > > On 19 May 2024, at 22:05, Anthony J. Bentley <bent...@openbsd.org> wrote:
> > >
> > > Vitaliy Makkoveev writes:
> > >>> On 17 May 2024, at 12:06, Stuart Henderson <s...@spacehopper.org> =
> > >> wrote:
> > >>> =20
> > >>> There are problems with wg(4) that people with some workloads have =
> > >> been
> > >>> seeing after upgrading past 7.3, though looking at this thread from =
> > >> when
> > >>> it last came up https://marc.info/?t=3D170940892700001&r=3D1&w=3D2 I'm =
> > >> not
> > >>> sure if we'd be expecting to see trouble on non-MP=E2=80=A6
> > >>> =20
> > >>
> > >> We do. The problem is not MP related.
> > >>
> > >> Antony, does the diff [1] help?
> > >>
> > >> 1. https://marc.info/?l=3Dopenbsd-bugs&m=3D170980835807159&w=3D2
> > >
> > > Crashes continue to occur with the same frequency after patching.
> > >
> >
> > This could be vio(4) bug. Please try this [1] diff.
> >
> > 1. https://marc.info/?l=openbsd-tech&m=171588941332420&w=2
>
> The traces all point to a use-after-free in a mbuf that has been through
> the wg(4) machinery. The fact that using a SP system makes the crash
> disappear points that this driver is not MP-safe and somehow there is a
> race which ends up corrupting memory associated to mbufs.
But only for some kind of workload / packets.
I've a machine that is running a service (HTTPS) behind a wireguard
connexion on OpenBSD-current. It has been running stable for several
months (I upgrade the machine almost every week).
OpenBSD 7.5-current (GENERIC.MP) #76: Fri May 17 10:28:20 MDT 2024
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 2051219456 (1956MB)
avail mem = 1968062464 (1876MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.8 @ 0x7b923000 (51 entries)
bios0: vendor American Megatrends Inc. version "YB1007" date 08/17/2017
bios0: AZW Z83 II
efi0 at bios0: UEFI 2.4
efi0: American Megatrends rev 0x5000b
acpi0 at bios0: ACPI 5.0
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP APIC FPDT FIDT MCFG SSDT SSDT SSDT UEFI HPET SSDT SSDT
SSDT SSDT TPM2 LPIT BCFG PRAM BGRT CSRT WDAT
acpi0: wakeup devices
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Atom(TM) x5-Z8350 CPU @ 1.44GHz, 1440.23 MHz, 06-4c-04, patch
00000411
cpu0: cpuid 1
edx=bfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
ecx=43d8e3bf<SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,MOVBE,POPCNT,DEADLINE,AES,RDRAND>
cpu0: cpuid 6 eax=7<SENSOR,ARAT> ecx=9<EFFFREQ>
cpu0: cpuid 7.0 ebx=2282<TSC_ADJUST,SMEP,ERMS>
edx=c000400<MD_CLEAR,IBRS,IBPB,STIBP>
cpu0: cpuid a vers=3, gp=2, gpwidth=40, ff=3, ffwidth=40
cpu0: cpuid 80000001 edx=28100800<NXE,RDTSCP,LONG> ecx=101<LAHF,3DNOWP>
cpu0: cpuid 80000007 edx=100<ITSC>
cpu0: MELTDOWN
cpu0: 24KB 64b/line 6-way D-cache, 32KB 64b/line 8-way I-cache, 1MB 64b/line
16-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 79MHz
cpu0: mwait min=64, max=64, C-substates=0.2, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Atom(TM) x5-Z8350 CPU @ 1.44GHz, 1440.46 MHz, 06-4c-04, patch
00000411
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Atom(TM) x5-Z8350 CPU @ 1.44GHz, 1440.75 MHz, 06-4c-04, patch
00000411
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
cpu3: Intel(R) Atom(TM) x5-Z8350 CPU @ 1.44GHz, 1440.78 MHz, 06-4c-04, patch
00000411
cpu3: smt 0, core 3, package 0
ioapic0 at mainbus0: apid 1 pa 0xfec00000, version 20, 115 pins
acpimcfg0 at acpi0
acpimcfg0: addr 0xe0000000, bus 0-255
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 1 (RP01)
acpiprt2 at acpi0: bus -1 (RP02)
acpiprt3 at acpi0: bus -1 (RP03)
acpiprt4 at acpi0: bus -1 (RP04)
"INT33A4" at acpi0 not configured
iosf0 at acpi0 MBID: mbi
chvgpio0 at acpi0 GPO1 uid 2 addr 0xfed88000/0x8000 irq 48, 59 pins
chvgpio1 at acpi0 GPO3 uid 4 addr 0xfed98000/0x8000 irq 91, 55 pins
dwiic0 at acpi0 I2C7 addr 0x91526000/0x1000 irq 38, sem
iic0 at dwiic0
"INT33F4" at iic0 addr 0x34 not configured
chvgpio2 at acpi0 GPO0 uid 1 addr 0xfed80000/0x8000 irq 49, 56 pins
acpipci0 at acpi0 PCI0: 0x00000000 0x00000011 0x00000001
com0 at acpi0 IURT addr 0x3f8/0x8 irq 4: ns16550a, 16 byte fifo
sdhc0 at acpi0 SDHA addr 0x9153a000/0x1000 irq 45
sdhc0: SDHC 3.00, 200 MHz base clock
sdmmc0 at sdhc0: 8-bit, sd high-speed, mmc high-speed, ddr52, dma
sdhc1 at acpi0 SDHB addr 0x91538000/0x1000 irq 46
chvgpio3 at acpi0 GPO2 uid 3 addr 0xfed90000/0x8000 irq 50, 24 pins
sdhc1: SDHC 3.00, 200 MHz base clock
sdmmc1 at sdhc1: 4-bit, sd high-speed, mmc high-speed, ddr52, dma
sdhc2 at acpi0 SHC1 addr 0x91536000/0x1000 irq 47, gpio
sdhc2: SDHC 3.00, 200 MHz base clock
sdmmc2 at sdhc2: 4-bit, sd high-speed, mmc high-speed, ddr52, dma
"INTL9C60" at acpi0 not configured
"INTL9C60" at acpi0 not configured
"8086228A" at acpi0 not configured
"BCM2EA4" at acpi0 not configured
"8086228A" at acpi0 not configured
"8086228E" at acpi0 not configured
"8086228E" at acpi0 not configured
"8086228E" at acpi0 not configured
dwiic1 at acpi0 I2C1 addr 0x91532000/0x1000 irq 32
iic1 at dwiic1
dwiic2 at acpi0 I2C2 addr 0x91530000/0x1000 irq 33
iic2 at dwiic2
"10EC5651" at iic2 addr 0x1a not configured
dwiic3 at acpi0 I2C3 addr 0x9152e000/0x1000 irq 34
iic3 at dwiic3
dwiic4 at acpi0 I2C4 addr 0x9152c000/0x1000 irq 35
iic4 at dwiic4
dwiic5 at acpi0 I2C5 addr 0x9152a000/0x1000 irq 36
iic5 at dwiic5
dwiic6 at acpi0 I2C6 addr 0x91528000/0x1000 irq 37
iic6 at dwiic6
"808622A8" at acpi0 not configured
acpicmos0 at acpi0
acpibtn0 at acpi0: PWRB
chvgpio4 at acpi0 GPO4 uid 5
"ACPI0011" at acpi0 not configured
"ACPI000C" at acpi0 not configured
acpiac0 at acpi0: AC unit online
"INT3400" at acpi0 not configured
"INT3403" at acpi0 not configured
tpm0 at acpi0 CTPM: unsupported TPM2 start method 2
acpicpu0 at acpi0: C2 bad (state 6 has no substates): C3 bad (state 7 has no
substates): C1(1000@1 mwait.1), PSS
acpicpu1 at acpi0: C2 bad (state 6 has no substates): C3 bad (state 7 has no
substates): C1(1000@1 mwait.1), PSS
acpicpu2 at acpi0: C2 bad (state 6 has no substates): C3 bad (state 7 has no
substates): C1(1000@1 mwait.1), PSS
acpicpu3 at acpi0: C2 bad (state 6 has no substates): C3 bad (state 7 has no
substates): C1(1000@1 mwait.1), PSS
acpipwrres0 at acpi0: ID3C, resource for ISP3
acpipwrres1 at acpi0: WWPR, resource for HS03, MDM1
acpipwrres2 at acpi0: WWPR, resource for HS13, MDM1
acpipwrres3 at acpi0: WWPR, resource for SSC1, MDM3
acpipwrres4 at acpi0: WWPR, resource for SSCW, MDM3
acpipwrres5 at acpi0: WWPR, resource for HSC1, MDM2
acpipwrres6 at acpi0: WWPR, resource for HSC3, MDM4
acpipwrres7 at acpi0: CLK3, resource for RTEK, RTK1, RTK2
acpipwrres8 at acpi0: CLK4
acpipwrres9 at acpi0: CLK2, resource for NFC2
acpipwrres10 at acpi0: CLK1
acpipwrres11 at acpi0: CLK0
acpipwrres12 at acpi0: CLK1
acpipwrres13 at acpi0: USBC, resource for XHC1, OTG1
acpipwrres14 at acpi0: P28X
acpipwrres15 at acpi0: P18X
acpipwrres16 at acpi0: P12X
acpipwrres17 at acpi0: P28P
acpipwrres18 at acpi0: P18P
acpipwrres19 at acpi0: P19X
acpipwrres20 at acpi0: P06X
acpipwrres21 at acpi0: P28T
acpipwrres22 at acpi0: P18D
acpipwrres23 at acpi0: P18T
acpipwrres24 at acpi0: P3P3
acpipwrres25 at acpi0: P12T
acpipwrres26 at acpi0: P28W
acpipwrres27 at acpi0: P18W
acpipwrres28 at acpi0: P12W
acpipwrres29 at acpi0: P33W
acpipwrres30 at acpi0: P33X
acpitz0 at acpi0: critical temperature is 90 degC
acpivideo0 at acpi0: GFX0
acpivout0 at acpivideo0: DD01
acpivout1 at acpivideo0: DD02
acpivout2 at acpivideo0: DD03
acpivout3 at acpivideo0: DD04
acpivout4 at acpivideo0: DD05
acpivout5 at acpivideo0: DD06
acpivout6 at acpivideo0: DD07
acpivout7 at acpivideo0: DD08
cpu0: using VERW MDS workaround
cpu0: Enhanced SpeedStep 1440 MHz: speeds: 1441, 1440, 1360, 1280, 1200, 1120,
1040, 960, 880, 800, 720, 640, 560, 480 MHz
pci0 at mainbus0 bus 0
iosf1 at pci0 dev 0 function 0 "Intel Braswell Host" rev 0x36: mbi
inteldrm0 at pci0 dev 2 function 0 "Intel HD Graphics" rev 0x36
drm0 at inteldrm0
inteldrm0: msi, CHERRYVIEW, gen 8
"Intel Braswell Power" rev 0x36 at pci0 dev 11 function 0 not configured
xhci0 at pci0 dev 20 function 0 "Intel Braswell xHCI" rev 0x36: msi, xHCI 1.0
usb0 at xhci0: USB revision 3.0
uhub0 at usb0 configuration 1 interface 0 "Intel xHCI root hub" rev 3.00/1.00
addr 1
"Intel Braswell TXE" rev 0x36 at pci0 dev 26 function 0 not configured
ppb0 at pci0 dev 28 function 0 "Intel Braswell PCIE" rev 0x36: msi
pci1 at ppb0 bus 1
re0 at pci1 dev 0 function 0 "Realtek 8168" rev 0x0c: RTL8168G/8111G (0x4c00),
msi, address 84:39:be:66:a1:0c
rgephy0 at re0 phy 7: RTL8251 PHY, rev. 0
pcib0 at pci0 dev 31 function 0 "Intel Braswell PCU LPC" rev 0x36
isa0 at pcib0
isadma0 at isa0
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pckbd0 at pckbc0 (kbd slot)
wskbd0 at pckbd0: console keyboard
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
vmm0 at mainbus0: VMX/EPT (using slow L1TF mitigation)
efifb at mainbus0 not configured
scsibus1 at sdmmc0: 2 targets, initiator 0
sd0 at scsibus1 targ 1 lun 0: <Sandisk, DF4032, 0000>
sd0: 29820MB, 512 bytes/sector, 61071360 sectors
uhub1 at uhub0 port 1 configuration 1 interface 0 "GenesysLogic USB2.0 Hub" rev
2.10/92.12 addr 2
bwfm0 at sdmmc1 function 1
manufacturer 0x02d0, product 0xa9bf at sdmmc1 function 2 not configured
manufacturer 0x02d0, product 0xa9bf at sdmmc1 function 3 not configured
uhidev0 at uhub1 port 1 configuration 1 interface 0 "Logitech USB Laser Mouse"
rev 2.00/31.00 addr 3
uhidev0: iclass 3/1
ums0 at uhidev0: 8 buttons, Z and W dir
wsmouse0 at ums0 mux 0
uhidev1 at uhub1 port 2 configuration 1 interface 0 "Fujitsu Component Type 6
Keyboard" rev 1.00/1.02 addr 4
uhidev1: iclass 3/1
ukbd0 at uhidev1: 8 variable keys, 6 key codes, country code 33
wskbd1 at ukbd0 mux 1
uhub2 at uhub0 port 8 configuration 1 interface 0 " " rev 3.00/92.12 addr 5
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
root on sd0a (3c9a89c702bc0dff.a) swap on sd0b dump on sd0b
inteldrm0: 1920x1080, 32bpp
wsdisplay0 at inteldrm0 mux 1: console (std, vt100 emulation), using wskbd0
wskbd1: connecting to wsdisplay0
wsdisplay0: screen 1-5 added (std, vt100 emulation)
bwfm0: failed loadfirmware of file brcmfmac43455-sdio.txt
# ifconfig wg0
wg0: flags=80c3<UP,BROADCAST,RUNNING,NOARP,MULTICAST> mtu 1420
index 5 priority 0 llprio 3
wgport 22261
wgrtable 1
wgpubkey <redacted>
groups: wg egress
inet xxx.xxx.xxx.3 netmask 0xffffffff
inet6 xxxx:xxxx:xxxx::300::1 prefixlen 64
# pfctl -vsi
Status: Enabled for 1 days 22:47:58 Debug: err
Hostid: 0x5024b36f
Checksum: 0x2e318795003ad15597e2bcb176967283
Interface Stats for wg0 IPv4 IPv6
Bytes In 11544219462 25656625948
Bytes Out 32415882461 5804695997
Packets In
Passed 34457033 30533698
Blocked 14605 614
Packets Out
Passed 51892935 26730708
Blocked 22 26
