Hello, I'm new to the BSD world and I've been *really* enjoying using NetBSD so far. I run mostly Linux on my servers but I'd love to migrate at least one to NetBSD. Sadly I'm hitting a big wall and I need your help.
In my local network NetBSD (running on Proxmox/KVM) saturates the 1 Gbps link no problem, but the issue arises over the Internet. On a Proxmox node located abroad (209ms RTT) I get these speeds: Alpine Linux: - iperf3 UDP: 609 Mbit/s - iperf3 TCP: 90 Mbit/s - HTTP: 9.70 MB/s FreeBSD: - iperf3 UDP: 607 Mbit/s - iperf3 TCP: 91 Mbit/s - HTTP: 9.65 MB/s OpenBSD: - iperf3 UDP: 619 Mbit/s - iperf3 TCP: 62 Mbit/s - HTTP: 9.07 MB/s NetBSD: - iperf3 UDP: 612 Mbit/s - iperf3 TCP: 9.10 Mbit/s - HTTP: 986 KB/s (hard capped and unstable) UDP speeds are great, but NetBSD specifically has an issue with TCP/HTTP. The speed seems to be capped to exactly 986 KB/s and it's very unstable (congestion handling issues I assume). I tried tuning the following: net.inet.tcp.recvbuf_auto=1 net.inet.tcp.sendbuf_auto=1 net.bpf.jit=1 net.bpf.maxbufsize=4194304 net.inet.tcp.sendbuf_max=16777216 net.inet.tcp.recvbuf_max=16777216 net.inet.ip.ifq.maxlen=4096 net.inet.tcp.mssdflt=1072 This fixed the unstability, but it's still seemingly hard capped to 986 KB/s. Since post-tuning it's very stable I don't think it's about congestion anymore - but I'm not sure why I'm hitting this hard cap, even after increasing the buffer sizes. Could it be that NetBSD is not increasing the window size? I wrote about it in detail (and made some graphs + uploaded tcpdump output) here: https://omaera.org/wlog/tech/bsd_network I saw someone last year around here mention that they got around 17 MB/s with default settings over the Internet in a VPS, so I'm lost about what's going on. I also tried moving the VM to Germany and Japan, but similar result (different speed cap, all below 1 MB/s). Since I only serve HTTP content I don't need huge speeds, just something good enough. I'm not knowledgeable about networking so any pointers are really helpful.
