Hi Kernel 4.9 finally landed in Debian testing so I could finally test BBR in a real life environment that I have struggled with getting any kind of performance out of.
The challenge at hand is UDP based OpenVPN through europe at around 35 ms rtt to my VPN-provider with plenty of available bandwith available in both ends and everything completely unknown in between. After tuning the UDP-buffers up to make room for my 500 mbit/s symmetrical bandwith at 35 ms the download part seemed to work nicely at an unreliable 150 to 300 mbit/s, while the upload was stuck at 30 to 60 mbit/s. Just by activating BBR the bandwith instantly shot up to around 150 mbit/s using a fat tcp test to a public iperf3 server located near my VPN exit point in the Netherlands. Replace BBR with qubic again and the performance is once again all over the place ranging from very bad to bad, but never better than 1/3 of BBRs "steady state". In other words "instant WIN!" However, seeing the requirement of fq and pacing for BBR and noticing that I am running pfifo_fast within a VM with virtio NIC on a Proxmox VE host with fq_codel on all physical interfaces, I was surprised to see that it worked so well. I then replaced pfifo_fast with fq and the performance went right down to only 1-4 mbit/s from around 150 mbit/s. Removing the fq again regained the performance at once. I have got some questions to you guys that know a lot more than me about these things: 1. Do fq (and fq_codel) even work reliably in a VM? What is the best choice for default qdisc to use in a VM in general? 2. Why do BBR immediately "fix" all my issues with upload through that "unreliable" big BDP link with pfifo_fast when fq pacing is a requirement? 3. Could fq_codel on the physical host be the reason that it still works? 4. Do BBR _only_ work with fq pacing or could fq_codel be used as a replacement? 5. Is BBR perhaps modified to do the right thing without having to change the qdisc in the current kernel 4.9? Sorry for long post, but this is an interesting topic! Regards, Hans-Kristian Bakke
_______________________________________________ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat