Re: Interesting behavior of 7.4 -> 7.5 upgrade on Protectli VP2420
I had a similar experience on a VP2410, but solved it in a different way. I did not connect a display and keyboard. I attempted a remote 7.4 -> 7.5 sysupgrade a few days ago on a Protectli VP2410; mine also came back after the upgrade reboot as 7.4. So today I dispatched to the client site with a console cable to see what was happening, here is what I see on the console after a reboot - https://imgur.com/a/o7WMgkc I found the message - "/bsd.upgrade is not u+x" - I looked at the upgrade kernel and it did lack the execute bit; it was 600. Here is the rest of the sequence of events: 1 - I manually set the execute bit with "chmod 700 /bsd.upgrade" 2 - I rebooted, and it went through the auto upgrade procedure and appeared to succeed 3 - but again it was booting into 7.4 after the reboot 4 - I saw that bsd.upgrade still existed and the execute bit was gone, it was back to 600 5 - I then deleted bsd.upgrade and ran sysupgrade again 6 - this time the process was successful 7 - I was in via ssh simultaneously along with watching the console 8 - I never hooked up a keyboard or display 9 - FYI - I ordered my Protectli with coreboot firmware (not the AMI option) I saved the Putty session and pasted below. This will not show my actions from the ssh session though, such as changing the execute bit and deleting /bsd.upgrade. Ollie Strickland --- DEL to enter Setup F11 to enter Boot Manager Menu ENTER to boot directly probing: pc0 mem[636K 255M 1377M 19M 44K 2M 2048M] disk: hd0 hd1* hd2* >> OpenBSD/amd64 BOOTX64 3.65 /bsd.upgrade is not u+x boot> booting hd0a:/bsd: 17245516+4142088+364576+0+1241088 [1347582+128+1321104+1013340]=0x1973370 entry point at 0x1001000 DEL to enter Setup F11 to enter Boot Manager Menu ENTER to boot directly probing: pc0 mem[636K 255M 1377M 19M 44K 2M 2048M] disk: hd0 hd1* hd2* >> OpenBSD/amd64 BOOTX64 3.65 upgrade detected: switching to /bsd.upgrade boot> booting hd0a:/bsd.upgrade: 4076463+1688576+3891240+0+708608 [109+464016+317541]=0xaa40e8 entry point at 0x1001000 DEL to enter Setup F11 to enter Boot Manager Menu ENTER to boot directly probing: pc0 mem[636K 255M 1377M 19M 44K 2M 2048M] disk: hd0 hd1* hd2* >> OpenBSD/amd64 BOOTX64 3.65 /bsd.upgrade is not u+x boot> NOTE: random seed is being reused. booting hd0a:/bsd: 17245516+4142088+364576+0+1241088 [1347582+128+1321104+1013340]=0x1973370 entry point at 0x1001000 DEL to enter Setup F11 to enter Boot Manager Menu ENTER to boot directly probing: pc0 mem[636K 255M 1377M 19M 44K 2M 2048M] disk: hd0 hd1* hd2* >> OpenBSD/amd64 BOOTX64 3.65 /bsd.upgrade is not u+x boot> set tty com0 switching console to com0 >> OpenBSD/amd64 BOOTX64 3.65 boot> booting hd0a:/bsd: 17245516+4142088+364576+0+1241088 [1347582+128+1321104+1013340]=0x1973370 entry point at 0x1001000 [ using 3683184 bytes of bsd ELF symbol table ] Copyright (c) 1982, 1986, 1989, 1991, 1993 The Regents of the University of California. All rights reserved. Copyright (c) 1995-2023 OpenBSD. All rights reserved. https://www.OpenBSD.org OpenBSD 7.4 (GENERIC.MP) #3: Wed Feb 28 06:23:33 MST 2024 r...@syspatch-74-amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 3843108864 (3665MB) avail mem = 3706884096 (3535MB) random: good seed from bootblocks mpath0 at root scsibus0 at mpath0: 256 targets mainbus0 at root bios0 at mainbus0: SMBIOS rev. 3.3 @ 0x697be000 (15 entries) bios0: vendor 3mdeb version "coreboot 4.13, Dasharo 1.0.15" date 05/31/2022 bios0: Protectli VP2410 efi0 at bios0: UEFI 2.7 efi0: EDK II rev 0x1 acpi0 at bios0: ACPI 6.0 acpi0: sleep states S0 S3 S4 S5 acpi0: tables DSDT FACP SSDT MCFG TPM2 APIC DMAR HPET acpi0: wakeup devices XHCI(S3) HDAS(S3) SLP_(S3) acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimcfg0 at acpi0 acpimcfg0: addr 0xe000, bus 0-255 acpimadt0 at acpi0 addr 0xfee0 cpu0 at mainbus0: apid 0 (boot processor) cpu0: Intel(R) Celeron(R) J4125 CPU @ 2.00GHz, 2000.00 MHz, 06-7a-08, patch 0024 cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SGX,SMEP,ERMS,MPX,RDSEED,SMAP,CLFLUSHOPT,PT,SHA,UMIP,MD_CLEAR,IBRS,IBPB,STIBP,SSBD,SENSOR,ARAT,IBRS_ALL,SKIP_L1DFL,MDS_NO,IF_PSCHANGE,MISC_PKG_CT,ENERGY_FILT,GDS_NO,XSAVEOPT,XSAVEC,XGETBV1,XSAVES cpu0: 24KB 64b/line 6-way D-cache, 32KB 64b/line 8-way I-cache, 4MB 64b/line 16-way L2 cache cpu0: smt 0, core 0, package 0 mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges cpu0: apic clock running at 19MHz cpu0: mwait min=64, max=64, C-substates=0.2.0.2.4.2.1.1, IBE cpu1 at mainbus0: apid 2 (application processor) cpu1: Intel(R) Celeron(R) J4125 CPU @ 2.00GHz
Re: OpenBSD 7.5 - relayd -> vaultwarden - websockets payload not working
Pardon me for not sending plain text the first time. Got trigger happy.
Ollie
I have been running the Vaultwarden password manager behind relayd for a couple
of years now, and have spun up a new 7.5 VM on Vultr to test.
I'm using pkg_add to install the binary package for the 7.5 release -
vaultwarden-1.30.5, so nothing nonstandard.
The problem - Vaultwarden uses a websockets connection to push changes to user
data in real time to all connected devices, and on 7.5 with relayd acting as
reverse proxy, websockets sessions get established successfully, but no payload
is able to pass from the server to the client.
Here are two images that show the dev console in Firefox -
https://imgur.com/a/msvyXbX
The first image shows websockets working correctly when public traffic is
directed to Vaultwarden's Rocket server without using relayd as a reverse proxy.
The second image shows relayd in place; no websockets payload can pass and the
Vaultwarden application cannot push changes to user data.
Relayd worked great for Vaultwarden in 7.4 and earlier. I saw that relayd got
touched in the changelogs.
My relayd.conf is:
table { localhost }
# protocol definition for vaultwarden with tls
http protocol vaultwarden-https {
# forward connections to vaultwarden rocket
match request path "/*" forward to
# add headers vaultwarden may need
match request header append "Host" value "$HOST"
match request header append "X-Real-IP" value "$REMOTE_ADDR"
match request header append "X-Forwarded-For" value "$REMOTE_ADDR"
match request header append "X-Forwarded-By" value
"$SERVER_ADDR:$SERVER_PORT"
match request header append "CF-Connecting-IP" value "$REMOTE_ADDR"
# various TCP options
tcp { nodelay, sack, backlog 128 }
# tls config
tls keypair vault.example.com
tls { no tlsv1.0, ciphers HIGH }
# allow websockets - this is nice it handles all the headers no need
for manual header edits
http websockets
}
# relay definition for vaultwarden - forward inbound 443 tls on the egress
interface to rocket on default port 8000
relay vaultwarden-https-relay {
listen on egress port 443 tls
protocol vaultwarden-https
forward to port 8000
}
And dmesg (VM on vultr) is:
OpenBSD 7.5 (GENERIC.MP) #82: Wed Mar 20 15:48:40 MDT 2024
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 4278042624 (4079MB)
avail mem = 4127375360 (3936MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0
acpi0 at bios0: ACPI 3.0
acpi0: sleep states S3 S4 S5
acpi0: tables DSDT FACP APIC HPET MCFG WAET
acpi0: wakeup devices
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: AMD EPYC-Rome Processor, 1996.57 MHz, 17-31-00
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,AMCR8,ABM,SSE4A,MASSE,OSVW,TOPEXT,CPCTR,FSGSBASE,BMI1,AVX2,SMEP,BMI2,CLFLUSHOPT,CLWB,SHA,UMIP,IBRS,IBPB,SSBD,IBPB,STIBP,XSAVEOPT,XSAVES
cpu0: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 512KB 64b/line
8-way L2 cache, 16MB 64b/line 16-way L3 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 1000MHz
cpu1 at mainbus0: apid 1 (application processor)
cpu1: AMD EPYC-Rome Processor, 1996.74 MHz, 17-31-00
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,AMCR8,ABM,SSE4A,MASSE,OSVW,TOPEXT,CPCTR,FSGSBASE,BMI1,AVX2,SMEP,BMI2,CLFLUSHOPT,CLWB,SHA,UMIP,IBRS,IBPB,SSBD,IBPB,STIBP,XSAVEOPT,XSAVES
cpu1: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 512KB 64b/line
8-way L2 cache, 16MB 64b/line 16-way L3 cache
cpu1: smt 1, core 0, package 0
ioapic0 at mainbus0: apid 0 pa 0xfec0, version 11, 24 pins
acpihpet0 at acpi0: 1 Hz
acpimcfg0 at acpi0
acpimcfg0: addr 0xb000, bus 0-255
acpiprt0 at acpi0: bus 0 (PCI0)
"ACPI0006" at acpi0 not configured
acpipci0 at acpi0 PCI0: 0x0010 0x0011 0x
"PNP0A06" at acpi0 not configured
"PNP0A06" at acpi0 not configured
"PNP0A06" at acpi0 not configured
"QEMU0002" at acpi0 not configured
acpicmos0 at acpi0
"ACPI0010" at acpi0 not configured
acpicpu0
OpenBSD 7.5 - relayd -> vaultwarden - websockets payload not working
I have been running the Vaultwarden password manager behind relayd for a couple
of years now, and have spun up a new 7.5 VM on Vultr to test.
I'm using pkg_add to install the binary package for the 7.5 release -
vaultwarden-1.30.5, so nothing nonstandard.
The problem - Vaultwarden uses a websockets connection to push changes to user
data in real time to all connected devices, and on 7.5 with relayd acting as
reverse proxy, websockets sessions get established successfully, but no payload
is able to pass from the server to the client.
Here are two images that show the dev console in Firefox -
https://imgur.com/a/msvyXbX
The first image shows websockets working correctly when public traffic is
directed to Vaultwarden's Rocket server without using relayd as a reverse proxy.
The second image shows relayd in place; no websockets payload can pass and the
Vaultwarden application cannot push changes to user data.
Relayd worked great for Vaultwarden in 7.4 and earlier. I saw that relayd got
touched in the changelogs.
My relayd.conf is:
table { localhost }
# protocol definition for vaultwarden with tls
http protocol vaultwarden-https {
# forward connections to vaultwarden rocket
match request path "/*" forward to
# add headers vaultwarden may need
match request header append "Host" value "$HOST"
match request header append "X-Real-IP" value "$REMOTE_ADDR"
match request header append "X-Forwarded-For" value "$REMOTE_ADDR"
match request header append "X-Forwarded-By" value
"$SERVER_ADDR:$SERVER_PORT"
match request header append "CF-Connecting-IP" value "$REMOTE_ADDR"
# various TCP options
tcp { nodelay, sack, backlog 128 }
# tls config
tls keypair vault.example.com
tls { no tlsv1.0, ciphers HIGH }
# allow websockets - this is nice it handles all the headers no need
for manual header edits
http websockets
}
# relay definition for vaultwarden - forward inbound 443 tls on the egress
interface to rocket on default port 8000
relay vaultwarden-https-relay {
listen on egress port 443 tls
protocol vaultwarden-https
forward to port 8000
}
And dmesg (VM on vultr) is:
OpenBSD 7.5 (GENERIC.MP) #82: Wed Mar 20 15:48:40 MDT 2024
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 4278042624 (4079MB)
avail mem = 4127375360 (3936MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0
acpi0 at bios0: ACPI 3.0
acpi0: sleep states S3 S4 S5
acpi0: tables DSDT FACP APIC HPET MCFG WAET
acpi0: wakeup devices
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: AMD EPYC-Rome Processor, 1996.57 MHz, 17-31-00
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,AMCR8,ABM,SSE4A,MASSE,OSVW,TOPEXT,CPCTR,FSGSBASE,BMI1,AVX2,SMEP,BMI2,CLFLUSHOPT,CLWB,SHA,UMIP,IBRS,IBPB,SSBD,IBPB,STIBP,XSAVEOPT,XSAVES
cpu0: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 512KB 64b/line
8-way L2 cache, 16MB 64b/line 16-way L3 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 1000MHz
cpu1 at mainbus0: apid 1 (application processor)
cpu1: AMD EPYC-Rome Processor, 1996.74 MHz, 17-31-00
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,AMCR8,ABM,SSE4A,MASSE,OSVW,TOPEXT,CPCTR,FSGSBASE,BMI1,AVX2,SMEP,BMI2,CLFLUSHOPT,CLWB,SHA,UMIP,IBRS,IBPB,SSBD,IBPB,STIBP,XSAVEOPT,XSAVES
cpu1: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 512KB 64b/line
8-way L2 cache, 16MB 64b/line 16-way L3 cache
cpu1: smt 1, core 0, package 0
ioapic0 at mainbus0: apid 0 pa 0xfec0, version 11, 24 pins
acpihpet0 at acpi0: 1 Hz
acpimcfg0 at acpi0
acpimcfg0: addr 0xb000, bus 0-255
acpiprt0 at acpi0: bus 0 (PCI0)
"ACPI0006" at acpi0 not configured
acpipci0 at acpi0 PCI0: 0x0010 0x0011 0x
"PNP0A06" at acpi0 not configured
"PNP0A06" at acpi0 not configured
"PNP0A06" at acpi0 not configured
"QEMU0002" at acpi0 not configured
acpicmos0 at acpi0
"ACPI0010" at acpi0 not configured
acpicpu0 at acpi0: C1(@1 halt!)
acpicpu1 at acpi0: C1(@1 halt!)
pvbus0 at mainbus0: KVM
pvclock0 at pvbus0
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "I
