>Synopsis: bse(4) receive throughput around 0.3 Mbit/s with Starlink Mini
>Category: kernel
>Environment:
        System      : OpenBSD 7.9
        Details     : OpenBSD 7.9 (GENERIC.MP) #16: Mon Sep 28 02:46:43 MDT 2026
                         
[email protected]:/usr/src/sys/arch/arm64/compile/GENERIC.MP

        Architecture: OpenBSD.arm64
        Machine     : arm64

        Raspberry Pi 4 Model B Rev 1.5, 2 GB RAM, BCM2711/GENET v5.
        PHY: BCM54210E, rev. 2 (brgphy0).
        Boot: U-Boot 2025.07; full boot dmesg and usbdevs are below.
        Stock GENERIC.MP kernel, no local kernel or driver modifications.
        All syspatches 001 through 034 are installed. The machine was
        rebooted after installing them on 2026-10-14.
        Ethernet: bse0, MTU 1500, autoselect, 1000baseT full-duplex.
        Direct wired connection to Starlink Mini, currently using its
        public IPv4/bypass configuration and DHCP. IPv6 autoconf is active.
        curl 8.19.0 with LibreSSL 4.3.0 was used for the latest tests.

>Description:
        The receive throughput of this RPi4/OpenBSD system is consistently
        around 0.2-0.3 Mbit/s when directly connected to Starlink Mini's
        Ethernet port. The link reports 1000baseT full-duplex and there
        are no interface input/output errors or collisions.

        This is reproducible across several download servers, IPv4 and
        IPv6, and HTTPS and SSH/WireGuard traffic. The problem predates
        the upgrade to 7.9 and persists after that upgrade, syspatching
        and rebooting. All rates below distinguish bytes/s from bits/s.

        Latest measurements, 2026-10-14, after syspatch and reboot:

        CDN IPv4, https://cdn.openbsd.org/pub/OpenBSD/7.9/arm64/bsd.rd
          HTTP 206; 375321 bytes in 15.002606 s; 25017 bytes/s.
        Same CDN IPv6:
          HTTP 206; 376900 bytes in 15.002815 s; 25121 bytes/s.
        Separate mirror, IPv4:
          https://ftp.eu.openbsd.org/pub/OpenBSD/7.9/arm64/bsd.rd
          HTTP 206; 409600 bytes in 12.010057 s; 34104 bytes/s.

        These are bounded range downloads. curl exit 28 is expected
        here because the deliberately short time limit expires before
        the requested 1 MiB has been transferred. The measured rates
        are 0.200-0.273 Mbit/s, not 25-34 Mbit/s.

        Earlier investigation on 2026-09-03/04 found:

        - One TCP download: about 34.2 kB/s.
        - Four simultaneous TCP downloads: about 34.7 kB/s TOTAL.
        - SSH/WireGuard data into this server: about 29.1 kB/s.
        - IPv4 about 34.7 kB/s, IPv6 about 32.8 kB/s.
        - No increase in TCP retransmissions during the controlled
          download, and no PF queue or bandwidth-shaping rules.
        - CPU mostly idle during the tests.
        - Captured large RX segments at bse0 had timestamp spacing
          around 33.554 ms, or multiples of that interval. These are
          software capture timestamps, not a measurement of the
          actual arrival time of frames on the wire.

        Control tests reported during that investigation:

        - Another computer using the same Starlink Ethernet port and
          cable achieved about 90 Mbit/s.
        - Devuan Linux on the same RPi4 achieved about 100 Mbit/s.
        - The same RPi4/OpenBSD on the house Ethernet network downloaded
          at about 6 MB/s (48 Mbit/s).
        - Starlink's own speed test reported about 186 Mbit/s down,
          38 Mbit/s up and 44 ms latency.
        - Disabling Starlink Public IP and trying DHCP/NAT did not
          remove the slow throughput.
        - Forcing 100baseTX full-duplex improved throughput to about
          119 kB/s, but did not restore normal performance. This was
          only a temporary test; the interface is back on autoselect.
          A duplex mismatch is a possible confounder in that test.

        Thus a general Starlink service bandwidth limit does not fit
        the control results. The symptom appears to depend on the
        OpenBSD GENET/PHY path and this link partner/traffic pattern.
        A specific driver or PHY cause has NOT been demonstrated.

>How-To-Repeat:
        On the affected stock OpenBSD RPi4, boot with bse0 directly
        connected to Starlink Mini. Use the normal DHCP configuration:

          /etc/hostname.bse0:
            inet autoconf
            inet6 autoconf

        Verify that the link and default route have come up:

          ifconfig bse0
          netstat -rn -f inet
          netstat -in -I bse0

        A minimal current download reproducer, requiring installed curl:

          /usr/local/bin/curl -4 --silent --show-error --fail \
            --connect-timeout 5 --max-time 15 --max-filesize 1048576 \
            --range 0-1048575 --output /dev/null \
            --write-out 'HTTP=%{http_code} bytes=%{size_download} 
time=%{time_total}s speed=%{speed_download} B/s\n' \
            https://cdn.openbsd.org/pub/OpenBSD/7.9/arm64/bsd.rd

        Repeat with -6 for IPv6, and with the alternative mirror:
          https://ftp.eu.openbsd.org/pub/OpenBSD/7.9/arm64/bsd.rd

        The expected result, based on the same-port/same-cable control
        tests, is throughput well above 0.3 Mbit/s. On the affected
        system the command prints rates near those shown above and
        times out before receiving the full 1 MiB.

>Fix:
        No verified fix or patch is available from me.

        A source comparison suggests a possible lead: explicit RX
        interrupt threshold and ring timeout initialization.

        OpenBSD sys/dev/ic/bcmgenet.c rev. 1.12 (2026-01-24) writes
        GENET_TX_DMA_MBUF_DONE_THRES but does not explicitly write the
        corresponding RX threshold or RX ring timeout. The RX register
        definitions are also absent from bcmgenetreg.h. This was checked
        against the public source on 2026-10-14; I have NOT boot-tested
        a -current kernel.

        NetBSD added RX/TX interrupt mitigation on 2021-03-08:
          
https://github.com/NetBSD/src/commit/7b6ab7c6d3273ce6b0bd23b327ad32dea1358dfe
        Its genet_set_rxthresh(sc, qid, 57, 10) explicitly programs the
        RX packet threshold and timeout (requested 57 us, rounded to
        seven 8.192 us ticks).

        Linux bcmgenet_set_rx_coalesce() also explicitly writes both
        settings; the non-adaptive defaults are one packet and zero
        timeout. Its source describes a timeout tick of 8.192 us.

        The observed 33.554 ms spacing is numerically close to
        4096 * 8.192 us = 33.554432 ms. This is a hypothesis only:
        the live RX timeout register value has not been confirmed,
        nor has a test kernel demonstrated that this initialization
        difference causes or fixes the problem.

        FreeBSD's current if_genet.c also lacks the corresponding
        explicit RX initialization. I have not tested FreeBSD on this
        hardware and do not claim that it fixes this symptom.

        Relevant public sources:
          https://github.com/openbsd/src/blob/master/sys/dev/ic/bcmgenet.c
          https://github.com/openbsd/src/blob/master/sys/dev/ic/bcmgenetreg.h
          https://github.com/NetBSD/src/blob/trunk/sys/dev/ic/bcmgenet.c
          
https://github.com/torvalds/linux/blob/master/drivers/net/ethernet/broadcom/genet/bcmgenet.c
          
https://github.com/freebsd/freebsd-src/blob/main/sys/arm64/broadcom/genet/if_genet.c

        I checked the 7.9 errata through 034 and the published changes
        toward 8.0; I found no listed fix addressing this symptom. The
        public OpenBSD bcmgenet.c still shows the initialization
        difference described above.

        Testing limitation: all Raspberry Pis available to me are in
        production running an EV charging/OCPP service. I cannot
        currently boot-test a custom kernel, a snapshot, or disruptive
        link/PHY changes, and have no spare RPi for such tests. I can
        provide additional read-only diagnostics where practical.

        I would appreciate a review of whether the RX register
        initialization deserves a change, or whether another known
        GENET/PHY issue better explains these observations.

dmesg:
OpenBSD 7.9 (GENERIC.MP) #16: Mon Sep 28 02:46:43 MDT 2026
    
[email protected]:/usr/src/sys/arch/arm64/compile/GENERIC.MP
real mem  = 2066153472 (1970MB)
avail mem = 1959325696 (1868MB)
random: good seed from bootblocks
mainbus0 at root: Raspberry Pi 4 Model B Rev 1.5
efi0 at mainbus0: UEFI 2.11
efi0: Das U-Boot rev 0x20250700
smbios0 at efi0: SMBIOS 3.7.0
smbios0: vendor U-Boot version "2025.07" date 07/01/2025
smbios0: raspberrypi Raspberry Pi 4 Model B Rev 1.5
cpu0 at mainbus0 mpidr 0: ARM Cortex-A72 r0p3
cpu0: 48KB 64b/line 3-way L1 PIPT I-cache, 32KB 64b/line 2-way L1 D-cache
cpu0: 1024KB 64b/line 16-way L2 cache
cpu0: CRC32,ASID16
cpu1 at mainbus0 mpidr 1: ARM Cortex-A72 r0p3
cpu1: 48KB 64b/line 3-way L1 PIPT I-cache, 32KB 64b/line 2-way L1 D-cache
cpu1: 1024KB 64b/line 16-way L2 cache
cpu2 at mainbus0 mpidr 2: ARM Cortex-A72 r0p3
cpu2: 48KB 64b/line 3-way L1 PIPT I-cache, 32KB 64b/line 2-way L1 D-cache
cpu2: 1024KB 64b/line 16-way L2 cache
cpu3 at mainbus0 mpidr 3: ARM Cortex-A72 r0p3
cpu3: 48KB 64b/line 3-way L1 PIPT I-cache, 32KB 64b/line 2-way L1 D-cache
cpu3: 1024KB 64b/line 16-way L2 cache
"linux,cma" at mainbus0 not configured
"nvram" at mainbus0 not configured
"nvram" at mainbus0 not configured
apm0 at mainbus0
simplefb0 at mainbus0: 1920x1200, 32bpp
wsdisplay0 at simplefb0 mux 1: console (std, vt100 emulation)
wsdisplay0: screen 1-5 added (std, vt100 emulation)
"system" at mainbus0 not configured
"axi" at mainbus0 not configured
simplebus0 at mainbus0: "soc"
bcmclock0 at simplebus0
bcmmbox0 at simplebus0
bcmgpio0 at simplebus0
bcmaux0 at simplebus0
ampintc0 at simplebus0 nirq 256, ncpu 4 ipi 0: "interrupt-controller"
bcmtmon0 at simplebus0
bcmdmac0 at simplebus0: DMA0 DMA2 DMA4 DMA5 DMA6 DMA7 DMA8 DMA9 DMA10
pluart0 at simplebus0: rev 3, 32 byte fifo
"interrupt-controller" at simplebus0 not configured
bcmdog0 at simplebus0
bcmirng0 at simplebus0
"firmware" at simplebus0 not configured
"power" at simplebus0 not configured
"mailbox" at simplebus0 not configured
sdhc0 at simplebus0
sdhc0: SDHC 3.00, 250 MHz base clock
sdmmc0 at sdhc0: 4-bit, sd high-speed, mmc high-speed
"gpiomem" at simplebus0 not configured
"fb" at simplebus0 not configured
simplebus1 at simplebus0: "nvmem"
"nvmem_otp" at simplebus1 not configured
"nvmem_cust" at simplebus1 not configured
"nvmem_priv" at simplebus1 not configured
simplebus2 at mainbus0: "clocks"
"clk-osc" at simplebus2 not configured
"clk-usb" at simplebus2 not configured
"phy" at mainbus0 not configured
"clk-27M" at mainbus0 not configured
"clk-108M" at mainbus0 not configured
simplebus3 at mainbus0: "emmc2bus"
sdhc1 at simplebus3
sdhc1: SDHC 3.00, 100 MHz base clock
sdmmc1 at sdhc1: 8-bit, sd high-speed, mmc high-speed, ddr52, dma
"pmu" at mainbus0 not configured
agtimer0 at mainbus0: 54000 kHz
simplebus4 at mainbus0: "scb"
bcmpcie0 at simplebus4
pci0 at bcmpcie0
ppb0 at pci0 dev 0 function 0 "Broadcom BCM2711" rev 0x20
pci1 at ppb0 bus 1
xhci0 at pci1 dev 0 function 0 "VIA VL805 xHCI" rev 0x01: intx, xHCI 1.0
usb0 at xhci0: USB revision 3.0
uhub0 at usb0 configuration 1 interface 0 "VIA xHCI root hub" rev 3.00/1.00 
addr 1
bse0 at simplebus4: address 2c:cf:67:0e:24:23
brgphy0 at bse0 phy 1: BCM54210E, rev. 2
"codec" at simplebus4 not configured
"dma" at simplebus4 not configured
gpioleds0 at mainbus0: "ACT", "PWR"
"cam_dummy_reg" at mainbus0 not configured
"regulator-cam1" at mainbus0 not configured
"regulator-sd-io-1v8" at mainbus0 not configured
"regulator-sd-vcc" at mainbus0 not configured
"fixedregulator_3v3" at mainbus0 not configured
"fixedregulator_5v0" at mainbus0 not configured
"zone_dma" at mainbus0 not configured
simplebus5 at mainbus0: "v3dbus"
scsibus0 at sdmmc1: 2 targets, initiator 0
sd0 at scsibus0 targ 1 lun 0: <SD/MMC, USD00, 0010> removable
sd0: 30250MB, 512 bytes/sector, 61952000 sectors
uhub1 at uhub0 port 1 configuration 1 interface 0 "VIA Labs USB2.0 Hub" rev 
2.10/4.21 addr 2
bwfm0 at sdmmc0 function 1
manufacturer 0x02d0, product 0xa9a6 at sdmmc0 function 2 not configured
manufacturer 0x02d0, product 0xa9a6 at sdmmc0 function 3 not configured
uhidev0 at uhub1 port 1 configuration 1 interface 0 "Microsoft Wired Keyboard 
400" rev 1.10/1.10 addr 3
uhidev0: iclass 3/1
ukbd0 at uhidev0: 8 variable keys, 6 key codes
wskbd0 at ukbd0: console keyboard, using wsdisplay0
vscsi0 at root
scsibus1 at vscsi0: 256 targets
softraid0 at root
scsibus2 at softraid0: 256 targets
root on sd0a (38f1aa3aa8c2cb0e.a) swap on sd0b dump on sd0b
WARNING: CHECK AND RESET THE DATE!
gpio0 at bcmgpio0: 58 pins
bwfm0: address 2c:cf:67:0e:24:24

usbdevs:
Controller /dev/usb0:
addr 01: 1106:0000 VIA, xHCI root hub
         super speed, self powered, config 1, rev 1.00
         driver: uhub0
addr 02: 2109:3431 VIA Labs, USB2.0 Hub
         high speed, self powered, config 1, rev 4.21
         driver: uhub1
addr 03: 045e:0752 Microsoft, Wired Keyboard 400
         low speed, power 100 mA, config 1, rev 1.10
         driver: uhidev0

Reply via email to