On 2026/10/14 10:15, Stuart Henderson wrote: > any difference if you connect via a switch as a test?
I am also wondering if different cable lengths might make a difference.. > On 2026/10/14 08:58, Lars Reunamo wrote: > > >Synopsis: bse(4) receive throughput around 0.3 Mbit/s with Starlink Mini > > >Category: kernel > > >Environment: > > System : OpenBSD 7.9 > > Details : OpenBSD 7.9 (GENERIC.MP) #16: Mon Sep 28 02:46:43 MDT 2026 > > > > [email protected]:/usr/src/sys/arch/arm64/compile/GENERIC.MP > > > > Architecture: OpenBSD.arm64 > > Machine : arm64 > > > > Raspberry Pi 4 Model B Rev 1.5, 2 GB RAM, BCM2711/GENET v5. > > PHY: BCM54210E, rev. 2 (brgphy0). > > Boot: U-Boot 2025.07; full boot dmesg and usbdevs are below. > > Stock GENERIC.MP kernel, no local kernel or driver modifications. > > All syspatches 001 through 034 are installed. The machine was > > rebooted after installing them on 2026-10-14. > > Ethernet: bse0, MTU 1500, autoselect, 1000baseT full-duplex. > > Direct wired connection to Starlink Mini, currently using its > > public IPv4/bypass configuration and DHCP. IPv6 autoconf is active. > > curl 8.19.0 with LibreSSL 4.3.0 was used for the latest tests. > > > > >Description: > > The receive throughput of this RPi4/OpenBSD system is consistently > > around 0.2-0.3 Mbit/s when directly connected to Starlink Mini's > > Ethernet port. The link reports 1000baseT full-duplex and there > > are no interface input/output errors or collisions. > > > > This is reproducible across several download servers, IPv4 and > > IPv6, and HTTPS and SSH/WireGuard traffic. The problem predates > > the upgrade to 7.9 and persists after that upgrade, syspatching > > and rebooting. All rates below distinguish bytes/s from bits/s. > > > > Latest measurements, 2026-10-14, after syspatch and reboot: > > > > CDN IPv4, https://cdn.openbsd.org/pub/OpenBSD/7.9/arm64/bsd.rd > > HTTP 206; 375321 bytes in 15.002606 s; 25017 bytes/s. > > Same CDN IPv6: > > HTTP 206; 376900 bytes in 15.002815 s; 25121 bytes/s. > > Separate mirror, IPv4: > > https://ftp.eu.openbsd.org/pub/OpenBSD/7.9/arm64/bsd.rd > > HTTP 206; 409600 bytes in 12.010057 s; 34104 bytes/s. > > > > These are bounded range downloads. curl exit 28 is expected > > here because the deliberately short time limit expires before > > the requested 1 MiB has been transferred. The measured rates > > are 0.200-0.273 Mbit/s, not 25-34 Mbit/s. > > > > Earlier investigation on 2026-09-03/04 found: > > > > - One TCP download: about 34.2 kB/s. > > - Four simultaneous TCP downloads: about 34.7 kB/s TOTAL. > > - SSH/WireGuard data into this server: about 29.1 kB/s. > > - IPv4 about 34.7 kB/s, IPv6 about 32.8 kB/s. > > - No increase in TCP retransmissions during the controlled > > download, and no PF queue or bandwidth-shaping rules. > > - CPU mostly idle during the tests. > > - Captured large RX segments at bse0 had timestamp spacing > > around 33.554 ms, or multiples of that interval. These are > > software capture timestamps, not a measurement of the > > actual arrival time of frames on the wire. > > > > Control tests reported during that investigation: > > > > - Another computer using the same Starlink Ethernet port and > > cable achieved about 90 Mbit/s. > > - Devuan Linux on the same RPi4 achieved about 100 Mbit/s. > > - The same RPi4/OpenBSD on the house Ethernet network downloaded > > at about 6 MB/s (48 Mbit/s). > > - Starlink's own speed test reported about 186 Mbit/s down, > > 38 Mbit/s up and 44 ms latency. > > - Disabling Starlink Public IP and trying DHCP/NAT did not > > remove the slow throughput. > > - Forcing 100baseTX full-duplex improved throughput to about > > 119 kB/s, but did not restore normal performance. This was > > only a temporary test; the interface is back on autoselect. > > A duplex mismatch is a possible confounder in that test. > > > > Thus a general Starlink service bandwidth limit does not fit > > the control results. The symptom appears to depend on the > > OpenBSD GENET/PHY path and this link partner/traffic pattern. > > A specific driver or PHY cause has NOT been demonstrated. > > > > >How-To-Repeat: > > On the affected stock OpenBSD RPi4, boot with bse0 directly > > connected to Starlink Mini. Use the normal DHCP configuration: > > > > /etc/hostname.bse0: > > inet autoconf > > inet6 autoconf > > > > Verify that the link and default route have come up: > > > > ifconfig bse0 > > netstat -rn -f inet > > netstat -in -I bse0 > > > > A minimal current download reproducer, requiring installed curl: > > > > /usr/local/bin/curl -4 --silent --show-error --fail \ > > --connect-timeout 5 --max-time 15 --max-filesize 1048576 \ > > --range 0-1048575 --output /dev/null \ > > --write-out 'HTTP=%{http_code} bytes=%{size_download} > > time=%{time_total}s speed=%{speed_download} B/s\n' \ > > https://cdn.openbsd.org/pub/OpenBSD/7.9/arm64/bsd.rd > > > > Repeat with -6 for IPv6, and with the alternative mirror: > > https://ftp.eu.openbsd.org/pub/OpenBSD/7.9/arm64/bsd.rd > > > > The expected result, based on the same-port/same-cable control > > tests, is throughput well above 0.3 Mbit/s. On the affected > > system the command prints rates near those shown above and > > times out before receiving the full 1 MiB. > > > > >Fix: > > No verified fix or patch is available from me. > > > > A source comparison suggests a possible lead: explicit RX > > interrupt threshold and ring timeout initialization. > > > > OpenBSD sys/dev/ic/bcmgenet.c rev. 1.12 (2026-01-24) writes > > GENET_TX_DMA_MBUF_DONE_THRES but does not explicitly write the > > corresponding RX threshold or RX ring timeout. The RX register > > definitions are also absent from bcmgenetreg.h. This was checked > > against the public source on 2026-10-14; I have NOT boot-tested > > a -current kernel. > > > > NetBSD added RX/TX interrupt mitigation on 2021-03-08: > > > > https://github.com/NetBSD/src/commit/7b6ab7c6d3273ce6b0bd23b327ad32dea1358dfe > > Its genet_set_rxthresh(sc, qid, 57, 10) explicitly programs the > > RX packet threshold and timeout (requested 57 us, rounded to > > seven 8.192 us ticks). > > > > Linux bcmgenet_set_rx_coalesce() also explicitly writes both > > settings; the non-adaptive defaults are one packet and zero > > timeout. Its source describes a timeout tick of 8.192 us. > > > > The observed 33.554 ms spacing is numerically close to > > 4096 * 8.192 us = 33.554432 ms. This is a hypothesis only: > > the live RX timeout register value has not been confirmed, > > nor has a test kernel demonstrated that this initialization > > difference causes or fixes the problem. > > > > FreeBSD's current if_genet.c also lacks the corresponding > > explicit RX initialization. I have not tested FreeBSD on this > > hardware and do not claim that it fixes this symptom. > > > > Relevant public sources: > > https://github.com/openbsd/src/blob/master/sys/dev/ic/bcmgenet.c > > > > https://github.com/openbsd/src/blob/master/sys/dev/ic/bcmgenetreg.h > > https://github.com/NetBSD/src/blob/trunk/sys/dev/ic/bcmgenet.c > > > > https://github.com/torvalds/linux/blob/master/drivers/net/ethernet/broadcom/genet/bcmgenet.c > > > > https://github.com/freebsd/freebsd-src/blob/main/sys/arm64/broadcom/genet/if_genet.c > > > > I checked the 7.9 errata through 034 and the published changes > > toward 8.0; I found no listed fix addressing this symptom. The > > public OpenBSD bcmgenet.c still shows the initialization > > difference described above. > > > > Testing limitation: all Raspberry Pis available to me are in > > production running an EV charging/OCPP service. I cannot > > currently boot-test a custom kernel, a snapshot, or disruptive > > link/PHY changes, and have no spare RPi for such tests. I can > > provide additional read-only diagnostics where practical. > > > > I would appreciate a review of whether the RX register > > initialization deserves a change, or whether another known > > GENET/PHY issue better explains these observations. > > > > dmesg: > > OpenBSD 7.9 (GENERIC.MP) #16: Mon Sep 28 02:46:43 MDT 2026 > > > > [email protected]:/usr/src/sys/arch/arm64/compile/GENERIC.MP > > real mem = 2066153472 (1970MB) > > avail mem = 1959325696 (1868MB) > > random: good seed from bootblocks > > mainbus0 at root: Raspberry Pi 4 Model B Rev 1.5 > > efi0 at mainbus0: UEFI 2.11 > > efi0: Das U-Boot rev 0x20250700 > > smbios0 at efi0: SMBIOS 3.7.0 > > smbios0: vendor U-Boot version "2025.07" date 07/01/2025 > > smbios0: raspberrypi Raspberry Pi 4 Model B Rev 1.5 > > cpu0 at mainbus0 mpidr 0: ARM Cortex-A72 r0p3 > > cpu0: 48KB 64b/line 3-way L1 PIPT I-cache, 32KB 64b/line 2-way L1 D-cache > > cpu0: 1024KB 64b/line 16-way L2 cache > > cpu0: CRC32,ASID16 > > cpu1 at mainbus0 mpidr 1: ARM Cortex-A72 r0p3 > > cpu1: 48KB 64b/line 3-way L1 PIPT I-cache, 32KB 64b/line 2-way L1 D-cache > > cpu1: 1024KB 64b/line 16-way L2 cache > > cpu2 at mainbus0 mpidr 2: ARM Cortex-A72 r0p3 > > cpu2: 48KB 64b/line 3-way L1 PIPT I-cache, 32KB 64b/line 2-way L1 D-cache > > cpu2: 1024KB 64b/line 16-way L2 cache > > cpu3 at mainbus0 mpidr 3: ARM Cortex-A72 r0p3 > > cpu3: 48KB 64b/line 3-way L1 PIPT I-cache, 32KB 64b/line 2-way L1 D-cache > > cpu3: 1024KB 64b/line 16-way L2 cache > > "linux,cma" at mainbus0 not configured > > "nvram" at mainbus0 not configured > > "nvram" at mainbus0 not configured > > apm0 at mainbus0 > > simplefb0 at mainbus0: 1920x1200, 32bpp > > wsdisplay0 at simplefb0 mux 1: console (std, vt100 emulation) > > wsdisplay0: screen 1-5 added (std, vt100 emulation) > > "system" at mainbus0 not configured > > "axi" at mainbus0 not configured > > simplebus0 at mainbus0: "soc" > > bcmclock0 at simplebus0 > > bcmmbox0 at simplebus0 > > bcmgpio0 at simplebus0 > > bcmaux0 at simplebus0 > > ampintc0 at simplebus0 nirq 256, ncpu 4 ipi 0: "interrupt-controller" > > bcmtmon0 at simplebus0 > > bcmdmac0 at simplebus0: DMA0 DMA2 DMA4 DMA5 DMA6 DMA7 DMA8 DMA9 DMA10 > > pluart0 at simplebus0: rev 3, 32 byte fifo > > "interrupt-controller" at simplebus0 not configured > > bcmdog0 at simplebus0 > > bcmirng0 at simplebus0 > > "firmware" at simplebus0 not configured > > "power" at simplebus0 not configured > > "mailbox" at simplebus0 not configured > > sdhc0 at simplebus0 > > sdhc0: SDHC 3.00, 250 MHz base clock > > sdmmc0 at sdhc0: 4-bit, sd high-speed, mmc high-speed > > "gpiomem" at simplebus0 not configured > > "fb" at simplebus0 not configured > > simplebus1 at simplebus0: "nvmem" > > "nvmem_otp" at simplebus1 not configured > > "nvmem_cust" at simplebus1 not configured > > "nvmem_priv" at simplebus1 not configured > > simplebus2 at mainbus0: "clocks" > > "clk-osc" at simplebus2 not configured > > "clk-usb" at simplebus2 not configured > > "phy" at mainbus0 not configured > > "clk-27M" at mainbus0 not configured > > "clk-108M" at mainbus0 not configured > > simplebus3 at mainbus0: "emmc2bus" > > sdhc1 at simplebus3 > > sdhc1: SDHC 3.00, 100 MHz base clock > > sdmmc1 at sdhc1: 8-bit, sd high-speed, mmc high-speed, ddr52, dma > > "pmu" at mainbus0 not configured > > agtimer0 at mainbus0: 54000 kHz > > simplebus4 at mainbus0: "scb" > > bcmpcie0 at simplebus4 > > pci0 at bcmpcie0 > > ppb0 at pci0 dev 0 function 0 "Broadcom BCM2711" rev 0x20 > > pci1 at ppb0 bus 1 > > xhci0 at pci1 dev 0 function 0 "VIA VL805 xHCI" rev 0x01: intx, xHCI 1.0 > > usb0 at xhci0: USB revision 3.0 > > uhub0 at usb0 configuration 1 interface 0 "VIA xHCI root hub" rev 3.00/1.00 > > addr 1 > > bse0 at simplebus4: address 2c:cf:67:0e:24:23 > > brgphy0 at bse0 phy 1: BCM54210E, rev. 2 > > "codec" at simplebus4 not configured > > "dma" at simplebus4 not configured > > gpioleds0 at mainbus0: "ACT", "PWR" > > "cam_dummy_reg" at mainbus0 not configured > > "regulator-cam1" at mainbus0 not configured > > "regulator-sd-io-1v8" at mainbus0 not configured > > "regulator-sd-vcc" at mainbus0 not configured > > "fixedregulator_3v3" at mainbus0 not configured > > "fixedregulator_5v0" at mainbus0 not configured > > "zone_dma" at mainbus0 not configured > > simplebus5 at mainbus0: "v3dbus" > > scsibus0 at sdmmc1: 2 targets, initiator 0 > > sd0 at scsibus0 targ 1 lun 0: <SD/MMC, USD00, 0010> removable > > sd0: 30250MB, 512 bytes/sector, 61952000 sectors > > uhub1 at uhub0 port 1 configuration 1 interface 0 "VIA Labs USB2.0 Hub" rev > > 2.10/4.21 addr 2 > > bwfm0 at sdmmc0 function 1 > > manufacturer 0x02d0, product 0xa9a6 at sdmmc0 function 2 not configured > > manufacturer 0x02d0, product 0xa9a6 at sdmmc0 function 3 not configured > > uhidev0 at uhub1 port 1 configuration 1 interface 0 "Microsoft Wired > > Keyboard 400" rev 1.10/1.10 addr 3 > > uhidev0: iclass 3/1 > > ukbd0 at uhidev0: 8 variable keys, 6 key codes > > wskbd0 at ukbd0: console keyboard, using wsdisplay0 > > vscsi0 at root > > scsibus1 at vscsi0: 256 targets > > softraid0 at root > > scsibus2 at softraid0: 256 targets > > root on sd0a (38f1aa3aa8c2cb0e.a) swap on sd0b dump on sd0b > > WARNING: CHECK AND RESET THE DATE! > > gpio0 at bcmgpio0: 58 pins > > bwfm0: address 2c:cf:67:0e:24:24 > > > > usbdevs: > > Controller /dev/usb0: > > addr 01: 1106:0000 VIA, xHCI root hub > > super speed, self powered, config 1, rev 1.00 > > driver: uhub0 > > addr 02: 2109:3431 VIA Labs, USB2.0 Hub > > high speed, self powered, config 1, rev 4.21 > > driver: uhub1 > > addr 03: 045e:0752 Microsoft, Wired Keyboard 400 > > low speed, power 100 mA, config 1, rev 1.10 > > driver: uhidev0 > > >
