[Bloat] a new mechanism for congestion control
sayeth nvidia. Anyone got a clue as to how? https://www.zdnet.com/article/nvidia-unveils-new-kind-of-ethernet-for-ai-grace-hopper-superchip-in-full-production/ -- Podcast: https://www.linkedin.com/feed/update/urn:li:activity:7058793910227111937/ Dave Täht CSO, LibreQos ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] SQM tuning question
Hi Aaron, > On Jun 3, 2023, at 20:04, Aaron Wood via Bloat > wrote: > > I’ve found that _usually_ I can set cake’s bandwidth limits to 90-95% of the > advertised bandwidth, and everything “just works”. So long as you’re > routinely able to achieve the bandwidth, it tends to work. > > I’ve found in my testing over the years (I’ve been a user of fq_codel since > 2013) that limiting the upstream bandwidth is the most important one to do. > Downstream bandwidth limits, especially if you have well above 100Mbps > downstream aren’t as critical, but it still can make a big difference if you > routinely saturate your downstream connection. > > It’s especially important to manage the upstream bandwidth when the > connection is asymmetric. For instance my 1Gbps Comcast internet service > only has a 35Mbps upstream limit, which is just a bit more than you need for > the acks that go back upstream in response to 1Gbps of downstream traffic. > > But if your connection quality suffers with time of day (congestion on shared > local ISP infrastructure), or due to weather (radio links can degrade during > the rain), then that can be a bit more annoying. > > But as I said, I’ve found with my Comcast connection that I can just set my > bandwidth limits to 90-95% of advertised and it just works. > > The other thing to look into is DNS. Most of the weird pauses that I see > are, DNS related. I e switches to using a local DNS cache/lookup-forwarding > server, and use DNS over HTTPS to parallel requests to both cloudflare and > google’s HTTPS DNS proxies. That seems to be more stable (long-lived > connections with faster recovery from a dropped packed than UDP has). > > (More continued below) > > On Sat, Jun 3, 2023 at 10:17 AM John D via Bloat > wrote: > Thanks for the detail. It makes sense but it kind of feels like in some > (maybe many) cases the router could know the internet link performance. > Particularly home router-modems often monitor this already. Maybe that's just > not exposed in any standardised way? > > There are two issues I’ve seen here: > > 1) there’s not a standardized way (or even usually any api) for getting the > provisioned rate, or the current link speed, from the modem portion of things. > > 2) local congestion at the ISP can degrade things below the provisioned > rates, and that’s even harder to detect or have available. > > If your upstream connection is DOCSIS 3.1, you should have the PIE AQM > running in the modem, which _should_ help considerably. But at least at my > cable headend, while downstream runs DOCSIS 3.1, upstream is only DOCSIS 3.0, > and definitely isn’t using PIE. > > The PIE AQM, running on the modem itself has the advantage of running on what > is almost always the bottle neck for upstream traffic in my experience: the > provisioned upstream bandwidth in the modem. [SM] Even better, the cable modem requests mini-slots from the scheduler and really has control over the true upstream queue, so even in an overloaded segment (where mini-slot scheduling gets slow) the PIE instance sees the relevant queueing delay to do its thing even if the current rate is well below the "provisioned" rate (which is just a maximum peak rate anyway). Now this will only help for congestion in the DOCSIS segment, as you said that is likely to be the most likely cause for upstream delay anyways... > > I'm guessing if I was into openwrt I could maybe do something, but I prefer > just to find something off the shelf with half decent SQM... If "auto > configuration" isn't a feature then that answers my question and I can get on > choosing the best option. > > I’m not aware of any decent “off the shelf” solutions that can track > bandwidth correctly, aside from DOCSIS 3.1 modems (which still requires the > cable company to provision it correctly to enable it) > > But, as I said, if your connection is stable, you can get the majority of the > benefits just by setting it up once and leaving it be. > > I’ve only tweaked things 1-2 times a year, for the last three years, and that > was only because I was moving. Once I had setup a router that could run cake > at my line rates, I’ve just left it there and it’s been fine. > > Especially since for me, WiFi is my usual bottleneck (my APs only do about > 250-300Mbps) > > -Aaron > > > On Sat, Jun 3, 2023, 16:44 Jonathan Morton wrote: > > On 3 Jun, 2023, at 4:56 pm, John D via Bloat > > wrote: > > > > On the website it says the following: > > > > CoDel is a novel “no knobs”, “just works”, “handles variable bandwidth and > > RTT”, and simple AQM algorithm. > > > > • It is parameterless — no knobs are required for operators, users, > > or implementers to adjust. > > • It treats good queue and bad queue differently - that is, it keeps > > the delays low while permitting bursts of traffic. > > • It controls delay, while insensitive to round-trip delays, link >
Re: [Bloat] SQM tuning question
Hi John, > On Jun 3, 2023, at 19:17, John D via Bloat > wrote: > > Thanks for the detail. It makes sense but it kind of feels like in some > (maybe many) cases the router could know the internet link performance. Sometimes they do, sometimes they do not, and sometimes the number the router might now is well above the contractual limit, e.g. my ISP for some time configured my VDSL2 link to sync at ~100/40 Mbps and restrict my capacity at an upstream device (BNG) to my contracted ~50/10 Mbps, so knowing the link speed will only give you an upper bound for some link technologies. > Particularly home router-modems often monitor this already. Maybe that's just > not exposed in any standardised way? Yes and no, all the link technologies I looked at (DSL/DOCSIS/PON) have some channel between the ISP side equipment (DSLAM/CMTS/OLT) and the equipment on the user side (CPE: DSL-/cable-/pon-modem) by which the ISP gear can query information from the CPE; but these three seem to be using three different approaches, and more importantly, just because the head-end can query these, does not mean that devices in the home network can do so as well... Plus, as mentioned above, the reported "sync" capacity might not be the capacity required to configure sqm's traffic shaper. > I'm guessing if I was into openwrt I could maybe do something, but I prefer > just to find something off the shelf with half decent SQM... If "auto > configuration" isn't a feature then that answers my question and I can get on > choosing the best option. There are the work-in-progress *-autorate approaches mentioned in my earlier post... Kind Regards Sebastian > > On Sat, Jun 3, 2023, 16:44 Jonathan Morton wrote: > > On 3 Jun, 2023, at 4:56 pm, John D via Bloat > > wrote: > > > > On the website it says the following: > > > > CoDel is a novel “no knobs”, “just works”, “handles variable bandwidth and > > RTT”, and simple AQM algorithm. > > > > • It is parameterless — no knobs are required for operators, users, > > or implementers to adjust. > > • It treats good queue and bad queue differently - that is, it keeps > > the delays low while permitting bursts of traffic. > > • It controls delay, while insensitive to round-trip delays, link > > rates, and traffic loads. > > • It adapts to dynamically changing link rates with no negative > > impact on utilization. > > > > But everywhere I have read about about hardware which implements SQM > > (including the bufferbloat website) it describes the need to tune based on > > actual internet connection speed. > > These seem to conflict especially that "handles variable bandwidth" bit. > > Have I misunderstood or do the algorithms used in modern hardware just not > > provide this part typically? My connection performance is quite variable > > and I'm worried about crippling SQM to the lowest speed seen. > > SQM in practice requires three components: > > 1: Flow isolation, so that different flows don't affect each others' latency > and are delivered fairly; > > 2: Active Queue Management (AQM) to signal flows to slow down transmissions > when link capacity is exceeded; > > 3: Bandwidth shaping to match the queue to the available capacity. > > CoDel is, in itself, only the AQM component. It does indeed work pretty well > with no additional tuning - but only in combination with the other two > components, or when applied directly to the actual bottleneck. Unfortunately > in most consumer internet links, the actual bottleneck is inaccessible for > this purpose. Thus an artificial bottleneck must be introduced, at which SQM > is applied. > > The most convenient tool for applying all three SQM components at once is > Cake. This includes implementations of advanced flow isolation, CoDel AQM, > and a deficit-mode bandwidth shaper. All you really need to do is to tell it > how much bandwidth you have in each direction, minus a small margin to ensure > it becomes the actual bottleneck and can exert the necessary control. > > When your available bandwidth varies over time, that can be inconvenient. > There are methods, however, of observing how available capacity tends to > change over time (typically on diurnal and weekly patterns, if the variations > are due to congestion in the ISP backhaul or peering) and scheduling > adjustments on that basis. If you have more information on your situation, > we might be able to give more detailed advice. > > - Jonathan Morton > ___ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] SQM tuning question
Have a look at: cake-autorate: https://github.com/lynxthecat/cake-autorate/blob/master/cake-autorate.sh lua sqm-autorate: https://github.com/sqm-autorate/sqm-autorate perl sqm-autorate: https://github.com/tievolu/sqm-autorate all three use active latency probes to detect an increase in delay and interpret that as increase in queueing delay which happens if the offered load exceeds the available capacity, and all three will, by slightly different heuristics, adjust the traffic shaper so that latency under loads stays (mostly) within acceptable (configurable) limits. Note the "organic" traffic over the link is measured and used as "load" so no cyclic saturating speedtest is necessary. I think that evenroute's iqrouter uses a different approach in that they periodically perform speedtests and then set the traffic shaper in accordance to that result. However the three different *-autorates should be runnable on essentially every linux distribution, while for evenroute's implementation one would need to buy their hardware IIRC. Best Regards Sebastian > On Jun 3, 2023, at 18:54, Kenneth Porter via Bloat > wrote: > > --On Saturday, June 03, 2023 7:44 PM +0300 Jonathan Morton via Bloat > wrote: > >> When your available bandwidth varies over time, that can be inconvenient. >> There are methods, however, of observing how available capacity tends to >> change over time (typically on diurnal and weekly patterns, if the >> variations are due to congestion in the ISP backhaul or peering) and >> scheduling adjustments on that basis. If you have more information on >> your situation, we might be able to give more detailed advice. > > Are there any good solutions for regularly recomputing the bandwidth on a > consumer link? I'm running the Linux SQM scripts to launch cake on a recent > Debian system and CoDel on a CentOS 7 gateway. I set the bandwidth manually > based on periodic manual speed tests. I'd love to automate that. > > ___ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Dual WAN home router with decent SQM?
any router that you can run OpenWRT on will do the job look at the mwan3 package to mange the multiple ISPs at 100Mb, you don't need an especially beefy router. David Lang On Sun, 4 Jun 2023, John D via Bloat wrote: Date: Sun, 4 Jun 2023 09:48:21 +0100 From: John D via Bloat Reply-To: John D To: bloat@lists.bufferbloat.net Subject: [Bloat] Dual WAN home router with decent SQM? I want to set up a new home router with SQM support to alleviate (mostly upload) bloat. However I would also like dual ISP support, more for backup/failover than load sharing. I can see affordable multi wan "load balancer" routers eg TP-link, and affordable SQM routers such as ubiquity Edgerouter, but I'm struggling to find a single device providing both. Just after any advice on a) am I better having two separate components b) if so any suggestions which product/brand? Two devices means more configuration but possibly more flexible and cheaper. My internet speed is looking to max out around 100/20mbps so I don't need Gbit performance. Thanks for any help. ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Dual WAN home router with decent SQM?
The wizards for quick setup makes some assumptions about which ports is WAN/LAN IIRC, but you're free to use the ports for whatever purpose you wish So yes, all the models can be used with load balancing / failover Best Regards Nils On Sun, Jun 4, 2023, at 15:57, John D wrote: > I might be missing the very obvious but I'm struggling to see which ER models > support multiple WAN when I look on their own website. Unless ALL ports > support WAN... if the ER-X can do this already then that was what I was > looking at on the router side anyway. > > On Sun, Jun 4, 2023, 14:07 Nils Andreas Svee wrote: >> Hi >> >> The EdgeRouter line does have load balancing as well as SQM (htb + fq_codel), >> so one of those should cover your needs. >> >> Of note, CAKE is not built-in to the firmware if you care about it, >> but support can be compiled >> >> With your current bandwidth, I'd guess an EdgeRouter Lite would to weak, >> but the EdgeRouter X or more powerful would do the job well. >> >> Keep in mind though, that as far as I can tell, the EdgeMax line is >> basically on life support, >> and only receives some security fixes once in a blue moon. >> >> Mikrotik's RouterOS devices supports load balancing / failover and SQM I >> believe, >> so there might be an option for you there, but as I've never used them >> personally, >> I can't comment beyond that. >> >> Best Regards >> Nils >> >> On Sun, Jun 4, 2023, at 10:48, John D via Bloat wrote: >> > I want to set up a new home router with SQM support to alleviate (mostly >> > upload) bloat. >> > However I would also like dual ISP support, more for backup/failover than >> > load sharing. >> > I can see affordable multi wan "load balancer" routers eg TP-link, and >> > affordable SQM routers such as ubiquity Edgerouter, but I'm struggling to >> > find a single device providing both. >> > Just after any advice on a) am I better having two separate components b) >> > if so any suggestions which product/brand? >> > Two devices means more configuration but possibly more flexible and >> > cheaper. >> > >> > My internet speed is looking to max out around 100/20mbps so I don't need >> > Gbit performance. Thanks for any help. >> > ___ >> > Bloat mailing list >> > Bloat@lists.bufferbloat.net >> > https://lists.bufferbloat.net/listinfo/bloat >> >___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Dual WAN home router with decent SQM?
I might be missing the very obvious but I'm struggling to see which ER models support multiple WAN when I look on their own website. Unless ALL ports support WAN... if the ER-X can do this already then that was what I was looking at on the router side anyway. On Sun, Jun 4, 2023, 14:07 Nils Andreas Svee wrote: > Hi > > The EdgeRouter line does have load balancing as well as SQM (htb + > fq_codel), > so one of those should cover your needs. > > Of note, CAKE is not built-in to the firmware if you care about it, > but support can be compiled > > With your current bandwidth, I'd guess an EdgeRouter Lite would to weak, > but the EdgeRouter X or more powerful would do the job well. > > Keep in mind though, that as far as I can tell, the EdgeMax line is > basically on life support, > and only receives some security fixes once in a blue moon. > > Mikrotik's RouterOS devices supports load balancing / failover and SQM I > believe, > so there might be an option for you there, but as I've never used them > personally, > I can't comment beyond that. > > Best Regards > Nils > > On Sun, Jun 4, 2023, at 10:48, John D via Bloat wrote: > > I want to set up a new home router with SQM support to alleviate (mostly > upload) bloat. > > However I would also like dual ISP support, more for backup/failover > than load sharing. > > I can see affordable multi wan "load balancer" routers eg TP-link, and > affordable SQM routers such as ubiquity Edgerouter, but I'm struggling to > find a single device providing both. > > Just after any advice on a) am I better having two separate components > b) if so any suggestions which product/brand? > > Two devices means more configuration but possibly more flexible and > cheaper. > > > > My internet speed is looking to max out around 100/20mbps so I don't > need Gbit performance. Thanks for any help. > > ___ > > Bloat mailing list > > Bloat@lists.bufferbloat.net > > https://lists.bufferbloat.net/listinfo/bloat > > > ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] Dual WAN home router with decent SQM?
Hi The EdgeRouter line does have load balancing as well as SQM (htb + fq_codel), so one of those should cover your needs. Of note, CAKE is not built-in to the firmware if you care about it, but support can be compiled With your current bandwidth, I'd guess an EdgeRouter Lite would to weak, but the EdgeRouter X or more powerful would do the job well. Keep in mind though, that as far as I can tell, the EdgeMax line is basically on life support, and only receives some security fixes once in a blue moon. Mikrotik's RouterOS devices supports load balancing / failover and SQM I believe, so there might be an option for you there, but as I've never used them personally, I can't comment beyond that. Best Regards Nils On Sun, Jun 4, 2023, at 10:48, John D via Bloat wrote: > I want to set up a new home router with SQM support to alleviate (mostly > upload) bloat. > However I would also like dual ISP support, more for backup/failover than > load sharing. > I can see affordable multi wan "load balancer" routers eg TP-link, and > affordable SQM routers such as ubiquity Edgerouter, but I'm struggling to > find a single device providing both. > Just after any advice on a) am I better having two separate components b) if > so any suggestions which product/brand? > Two devices means more configuration but possibly more flexible and cheaper. > > My internet speed is looking to max out around 100/20mbps so I don't need > Gbit performance. Thanks for any help. > ___ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat > ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
[Bloat] Dual WAN home router with decent SQM?
I want to set up a new home router with SQM support to alleviate (mostly upload) bloat. However I would also like dual ISP support, more for backup/failover than load sharing. I can see affordable multi wan "load balancer" routers eg TP-link, and affordable SQM routers such as ubiquity Edgerouter, but I'm struggling to find a single device providing both. Just after any advice on a) am I better having two separate components b) if so any suggestions which product/brand? Two devices means more configuration but possibly more flexible and cheaper. My internet speed is looking to max out around 100/20mbps so I don't need Gbit performance. Thanks for any help. ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
[Bloat] Fwd: [iccrg] I-D Action: draft-irtf-iccrg-rledbat-04.txt
-- Forwarded message - From: Date: Sun, Jun 4, 2023 at 2:02 AM Subject: [iccrg] I-D Action: draft-irtf-iccrg-rledbat-04.txt To: Cc: A New Internet-Draft is available from the on-line Internet-Drafts directories. This Internet-Draft is a work item of the Internet Congestion Control (ICCRG) RG of the IRTF. Title : rLEDBAT: receiver-driven Low Extra Delay Background Transport for TCP Authors : Marcelo Bagnulo Alberto Garcia-Martinez Gabriel Montenegro Praveen Balasubramanian Filename: draft-irtf-iccrg-rledbat-04.txt Pages : 15 Date: 2023-06-04 Abstract: This document specifies the rLEDBAT, a set of mechanisms that enable the execution of a less-than-best-effort congestion control algorithm for TCP at the receiver end. The IETF datatracker status page for this Internet-Draft is: https://datatracker.ietf.org/doc/draft-irtf-iccrg-rledbat/ There is also an htmlized version available at: https://datatracker.ietf.org/doc/html/draft-irtf-iccrg-rledbat-04 A diff from the previous version is available at: https://author-tools.ietf.org/iddiff?url2=draft-irtf-iccrg-rledbat-04 Internet-Drafts are also available by rsync at rsync.ietf.org::internet-drafts ___ iccrg mailing list ic...@irtf.org https://www.irtf.org/mailman/listinfo/iccrg -- Podcast: https://www.linkedin.com/feed/update/urn:li:activity:7058793910227111937/ Dave Täht CSO, LibreQos ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
