Re: [Starlink] The "reasons" that bufferbloat isn't a problem

[Eugene Y Chang via Starlink]

I am appreciating this email thread.

It is really hard to “sell” the features ala carte. They need to be bundled 
into reference packages. This you know.

> The wisps totally got it with fq codel and cake arriving native for mikeotiks 
> entire product line and much of ubnts gear prior to that.


Now we need a demo (or packaged test suite) that shows a full package (aka 
Mikrotik) against a less capable implementation with missing features. Users 
need to visualize the difference to develop envy for the whole package. 
Hopefully, the demo is not lame.

Dave, did you indirectly point out that an eSports demo with Miriotik 
AP/routers could deliver visible improvement? What do I need to verify before 
trying it out? Any advice on what expectations to set?

Gene
--
Eugene Chang




> On May 7, 2024, at 3:48 PM, Dave Taht  wrote:
> 
> This was a wonderful post, rich!
> 
> I note that preseem, paraqum,  bequant and libreqos (a bufferbloat.net 
>  backed project) are in the fq codel or cake 
> Middlebox for isps *Qoe) market and all of us have made a substantial dent in 
> the problem for oh, call it 1000 isps worldwide total between us. Comcast 
> also has done a pretty good job but it seems yhe rest of the cable industry 
> is asleep at the switch.
> 
> The wisps totally got it with fq codel and cake arriving native for mikeotiks 
> entire product line and much of ubnts gear prior to that.
> 
> 
> Qoe is still a pretty hard sell. Libreqos has a ton of free users and we 
> think over a million devices managed by it but not enough paid users to 
> justify even 1/10th the investment we have made so far into it (something 
> that I hope turns around with the upcoming v1.5 lts release and some outputs 
> from the nlnet and Comcast funded cakemaint and nqb projects)
> 
> Thing is, at higher and fiber rates all the bloat moves to the wifi, and a 
> ton of that, like eero especially was long ago fq codeled and so I think 
> several major players have also (except for those stuck with broadcom).
> 
> That said there are a lot of defective wifi aps left to replace. Nearly every 
> coffee shop I have been in with the exception of Starbucks has really lousy 
> wifi.
> 
> I am so thrilled to see what starlink has accomplished so far with their 
> rollout of bufferbloat.net  stuff and look forward 
> to more. They are still missing a few tricks... but are aware of what tricks 
> they are missing...
> 
> Lack of knowledge of which regrettably remains the case for 97% of the market 
> and 99.99$ user base. Still ar apps will drive this rventually... I think 
> starlink is nicely positioned now to meet their demanding growth goals and 
> humanity's future in space assured, so there's that. ( i still would rather 
> like elone to send over a nice pair of tesla motors and battery pack for my 
> sailboat)
> 
> I did have a nice jam with ajit Pai last week who is now well on his way 
> towards getting it. (See my twitter for the pics)
> 
> On Mon, May 6, 2024, 4:25 AM Rich Brown via Starlink 
> mailto:starlink@lists.bufferbloat.net>> 
> wrote:
> Hi Gene,
> 
> I've been vacillating on whether to send this note, but have decided to pull 
> the trigger. I apologize in advance for the "Debbie Downer" nature of this 
> message. I also apologize for any errors, omissions, or over-simplifications 
> of the "birth of bufferbloat" story and its fixes. Corrections welcome.
> 
> Rich
> --
> 
> If we are going to take a shot at opening people's eyes to bufferbloat, we 
> should know some of the "objections" we'll run up against. Even though 
> there's terrific technical data to back it up, people seem especially 
> resistant to thinking that bufferbloat might affect their network, even when 
> they're seeing problems that sound exactly like bufferbloat symptoms. But 
> first, some history:
> 
> The very idea of bufferbloat is simply unbelievable. Jim Gettys in 2011 [1] 
> couldn't believe it, and he's a smart networking guy,. At the time, it seemed 
> incredible (that is "not credible" == impossible) that something could induce 
> 1.2 seconds of latency into his home network connection. He called in favors 
> from technical contacts at his ISP and at Bell Labs who went over everything 
> with a fine-toothed comb. It was all exactly as spec'd. But he still had the 
> latency.
> 
> This led Jim and Dave Täht to start the investigation into the phenomenon 
> known today as "bufferbloat" - the undesirable latency that comes from a 
> router or other network equipment buffering too much data. Over several 
> years, a group of smart people made huge improvements: fq_codel was released 
> 14 May 2012 [3]; it was incorporated into the Linux kernel shortly afterward. 
> CAKE came in 2015, and the fixes that minimize bufferbloat in Wi-Fi arrived 
> in 2018. In 2021 cake-autorate [4] arrived to handle varying speed ISP links. 
> All these techniques work 

Re: [Starlink] The "reasons" that bufferbloat isn't a problem

[David Fernández via Starlink]

I see that L4S is not really solving everything (I read about issues with
Wi-Fi), although it seems to be a step in the right direction, to be
improved, let's hope.

At least, Nokia is implementing it in its network gear (for mobile
operators), so the bufferbloat problem is somehow acknowledged by industry,
at least initially or partially.

I have seen two consecutive RFCs to 9330:
https://www.rfc-editor.org/info/rfc9331
https://www.rfc-editor.org/info/rfc9332

I suspect that optimal results require the bufferbloat to be addressed not
only at network layer (IP), but also with some pipelining or cross-layering
at link level (Ethernet, Wi-Fi or any other link technology, such as 5G,
SATCOM, VHF...)

Regards,

David F.

Date: Tue, 7 May 2024 08:46:03 -0400
From: Dave Collier-Brown 
To: starlink@lists.bufferbloat.net
Subject: Re: [Starlink] The "reasons" that bufferbloat isn't a problem
Message-ID: <3d6bdccf-e3d1-4f62-a029-25bfd1f45...@indexexchange.com>
Content-Type: text/plain; charset="utf-8"; Format="flowed"

It has an RFC at https://datatracker.ietf.org/doc/rfc9330/

I read it as a way to rapidly find the available bandwidth without the TCP
"sawtooth". The paper cites fc_codel and research based on it.

I suspect My Smarter Colleagues know more (;-))

--dave



On 2024-05-07 08:13, David Fernández via Starlink wrote:
Is L4S a solution to bufferbloat? I have read that gamers are happy with it.

Sorry, I read it here, in Spanish:
https://www.adslzone.net/noticias/operadores/retardo-videojuegos-nokia-vodafone

Regards,

David F.
___
Starlink mailing list
Starlink@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/starlink

Re: [Starlink] The "reasons" that bufferbloat isn't a problem

[Frantisek Borsik via Starlink]

Sorry - I meant ~ 4,5 % of the ISPs, not 2 :)

All the best,

Frank

Frantisek (Frank) Borsik



https://www.linkedin.com/in/frantisekborsik

Signal, Telegram, WhatsApp: +421919416714

iMessage, mobile: +420775230885

Skype: casioa5302ca

frantisek.bor...@gmail.com


On Wed, May 8, 2024 at 9:58 AM Frantisek Borsik 
wrote:

> Just to add the latest numbers from our (LibreQoS) ongoing "QoE
> competitive landscape research":
>
> Out of 66k plus ISPs worldwide, barely 3k use some QoE middle-box. Preseem
> is the market leader, with well over 400 T2 & T3 ISPs (number shared in
> their wonderful Fixed Wireless Network Report 2024 Q1 Edition
> ),
> Bequant seems to be 2nd, in terms of market penetration, claiming 500+ ISPs
> worldwide. Then we assume Cambium Networks and Paraqum - numbers of their
> users are not know, but we can expect something similar, in the Preseem and
> Bequant ballpark. Then there is LibreQoS. All in all, it's safe to say that
> somewhere between 2,5 and 3 thousand Internet Service Providers worldwide
> are using QoE middle-box of sort. So barely 2% of the ISPs worldwide are
> using it, we are still in the "innovator" stage of the whole "crossing the
> chasm" paradigm.
>
> We are all still very early on, working on it, and I'm lovin' it.
>
>
> All the best,
>
> Frank
>
> Frantisek (Frank) Borsik
>
>
>
> https://www.linkedin.com/in/frantisekborsik
>
> Signal, Telegram, WhatsApp: +421919416714
>
> iMessage, mobile: +420775230885
>
> Skype: casioa5302ca
>
> frantisek.bor...@gmail.com
>
>
> On Wed, May 8, 2024 at 4:16 AM Dave Taht via Starlink <
> starlink@lists.bufferbloat.net> wrote:
>
>> This was a wonderful post, rich!
>>
>> I note that preseem, paraqum,  bequant and libreqos (a bufferbloat.net
>> backed project) are in the fq codel or cake Middlebox for isps *Qoe) market
>> and all of us have made a substantial dent in the problem for oh, call it
>> 1000 isps worldwide total between us. Comcast also has done a pretty good
>> job but it seems yhe rest of the cable industry is asleep at the switch.
>>
>> The wisps totally got it with fq codel and cake arriving native for
>> mikeotiks entire product line and much of ubnts gear prior to that.
>>
>>
>> Qoe is still a pretty hard sell. Libreqos has a ton of free users and we
>> think over a million devices managed by it but not enough paid users to
>> justify even 1/10th the investment we have made so far into it (something
>> that I hope turns around with the upcoming v1.5 lts release and some
>> outputs from the nlnet and Comcast funded cakemaint and nqb projects)
>>
>> Thing is, at higher and fiber rates all the bloat moves to the wifi, and
>> a ton of that, like eero especially was long ago fq codeled and so I think
>> several major players have also (except for those stuck with broadcom).
>>
>> That said there are a lot of defective wifi aps left to replace. Nearly
>> every coffee shop I have been in with the exception of Starbucks has really
>> lousy wifi.
>>
>> I am so thrilled to see what starlink has accomplished so far with their
>> rollout of bufferbloat.net stuff and look forward to more. They are
>> still missing a few tricks... but are aware of what tricks they are
>> missing...
>>
>> Lack of knowledge of which regrettably remains the case for 97% of the
>> market and 99.99$ user base. Still ar apps will drive this rventually... I
>> think starlink is nicely positioned now to meet their demanding growth
>> goals and humanity's future in space assured, so there's that. ( i still
>> would rather like elone to send over a nice pair of tesla motors and
>> battery pack for my sailboat)
>>
>> I did have a nice jam with ajit Pai last week who is now well on his way
>> towards getting it. (See my twitter for the pics)
>>
>> On Mon, May 6, 2024, 4:25 AM Rich Brown via Starlink <
>> starlink@lists.bufferbloat.net> wrote:
>>
>>> Hi Gene,
>>>
>>> I've been vacillating on whether to send this note, but have decided to
>>> pull the trigger. I apologize in advance for the "Debbie Downer" nature of
>>> this message. I also apologize for any errors, omissions, or
>>> over-simplifications of the "birth of bufferbloat" story and its fixes.
>>> Corrections welcome.
>>>
>>> Rich
>>> --
>>>
>>> If we are going to take a shot at opening people's eyes to bufferbloat,
>>> we should know some of the "objections" we'll run up against. Even though
>>> there's terrific technical data to back it up, people seem especially
>>> resistant to thinking that bufferbloat might affect their network, even
>>> when they're seeing problems that sound exactly like bufferbloat symptoms.
>>> But first, some history:
>>>
>>> The very idea of bufferbloat is simply unbelievable. Jim Gettys in 2011
>>> [1] couldn't believe it, and he's a smart networking guy,. At the time, it
>>> seemed incredible (that is "not credible" == impossible) that something
>>> could induce 1.2 

Re: [Starlink] The "reasons" that bufferbloat isn't a problem

[Frantisek Borsik via Starlink]

Just to add the latest numbers from our (LibreQoS) ongoing "QoE competitive
landscape research":

Out of 66k plus ISPs worldwide, barely 3k use some QoE middle-box. Preseem
is the market leader, with well over 400 T2 & T3 ISPs (number shared in
their wonderful Fixed Wireless Network Report 2024 Q1 Edition
),
Bequant seems to be 2nd, in terms of market penetration, claiming 500+ ISPs
worldwide. Then we assume Cambium Networks and Paraqum - numbers of their
users are not know, but we can expect something similar, in the Preseem and
Bequant ballpark. Then there is LibreQoS. All in all, it's safe to say that
somewhere between 2,5 and 3 thousand Internet Service Providers worldwide
are using QoE middle-box of sort. So barely 2% of the ISPs worldwide are
using it, we are still in the "innovator" stage of the whole "crossing the
chasm" paradigm.

We are all still very early on, working on it, and I'm lovin' it.


All the best,

Frank

Frantisek (Frank) Borsik



https://www.linkedin.com/in/frantisekborsik

Signal, Telegram, WhatsApp: +421919416714

iMessage, mobile: +420775230885

Skype: casioa5302ca

frantisek.bor...@gmail.com


On Wed, May 8, 2024 at 4:16 AM Dave Taht via Starlink <
starlink@lists.bufferbloat.net> wrote:

> This was a wonderful post, rich!
>
> I note that preseem, paraqum,  bequant and libreqos (a bufferbloat.net
> backed project) are in the fq codel or cake Middlebox for isps *Qoe) market
> and all of us have made a substantial dent in the problem for oh, call it
> 1000 isps worldwide total between us. Comcast also has done a pretty good
> job but it seems yhe rest of the cable industry is asleep at the switch.
>
> The wisps totally got it with fq codel and cake arriving native for
> mikeotiks entire product line and much of ubnts gear prior to that.
>
>
> Qoe is still a pretty hard sell. Libreqos has a ton of free users and we
> think over a million devices managed by it but not enough paid users to
> justify even 1/10th the investment we have made so far into it (something
> that I hope turns around with the upcoming v1.5 lts release and some
> outputs from the nlnet and Comcast funded cakemaint and nqb projects)
>
> Thing is, at higher and fiber rates all the bloat moves to the wifi, and a
> ton of that, like eero especially was long ago fq codeled and so I think
> several major players have also (except for those stuck with broadcom).
>
> That said there are a lot of defective wifi aps left to replace. Nearly
> every coffee shop I have been in with the exception of Starbucks has really
> lousy wifi.
>
> I am so thrilled to see what starlink has accomplished so far with their
> rollout of bufferbloat.net stuff and look forward to more. They are still
> missing a few tricks... but are aware of what tricks they are missing...
>
> Lack of knowledge of which regrettably remains the case for 97% of the
> market and 99.99$ user base. Still ar apps will drive this rventually... I
> think starlink is nicely positioned now to meet their demanding growth
> goals and humanity's future in space assured, so there's that. ( i still
> would rather like elone to send over a nice pair of tesla motors and
> battery pack for my sailboat)
>
> I did have a nice jam with ajit Pai last week who is now well on his way
> towards getting it. (See my twitter for the pics)
>
> On Mon, May 6, 2024, 4:25 AM Rich Brown via Starlink <
> starlink@lists.bufferbloat.net> wrote:
>
>> Hi Gene,
>>
>> I've been vacillating on whether to send this note, but have decided to
>> pull the trigger. I apologize in advance for the "Debbie Downer" nature of
>> this message. I also apologize for any errors, omissions, or
>> over-simplifications of the "birth of bufferbloat" story and its fixes.
>> Corrections welcome.
>>
>> Rich
>> --
>>
>> If we are going to take a shot at opening people's eyes to bufferbloat,
>> we should know some of the "objections" we'll run up against. Even though
>> there's terrific technical data to back it up, people seem especially
>> resistant to thinking that bufferbloat might affect their network, even
>> when they're seeing problems that sound exactly like bufferbloat symptoms.
>> But first, some history:
>>
>> The very idea of bufferbloat is simply unbelievable. Jim Gettys in 2011
>> [1] couldn't believe it, and he's a smart networking guy,. At the time, it
>> seemed incredible (that is "not credible" == impossible) that something
>> could induce 1.2 seconds of latency into his home network connection. He
>> called in favors from technical contacts at his ISP and at Bell Labs who
>> went over everything with a fine-toothed comb. It was all exactly as
>> spec'd. But he still had the latency.
>>
>> This led Jim and Dave Täht to start the investigation into the phenomenon
>> known today as "bufferbloat" - the undesirable latency that comes from a
>> router or other network equipment buffering too much data. 

Re: [Starlink] The "reasons" that bufferbloat isn't a problem

[Dave Taht via Starlink]

This was a wonderful post, rich!

I note that preseem, paraqum,  bequant and libreqos (a bufferbloat.net
backed project) are in the fq codel or cake Middlebox for isps *Qoe) market
and all of us have made a substantial dent in the problem for oh, call it
1000 isps worldwide total between us. Comcast also has done a pretty good
job but it seems yhe rest of the cable industry is asleep at the switch.

The wisps totally got it with fq codel and cake arriving native for
mikeotiks entire product line and much of ubnts gear prior to that.


Qoe is still a pretty hard sell. Libreqos has a ton of free users and we
think over a million devices managed by it but not enough paid users to
justify even 1/10th the investment we have made so far into it (something
that I hope turns around with the upcoming v1.5 lts release and some
outputs from the nlnet and Comcast funded cakemaint and nqb projects)

Thing is, at higher and fiber rates all the bloat moves to the wifi, and a
ton of that, like eero especially was long ago fq codeled and so I think
several major players have also (except for those stuck with broadcom).

That said there are a lot of defective wifi aps left to replace. Nearly
every coffee shop I have been in with the exception of Starbucks has really
lousy wifi.

I am so thrilled to see what starlink has accomplished so far with their
rollout of bufferbloat.net stuff and look forward to more. They are still
missing a few tricks... but are aware of what tricks they are missing...

Lack of knowledge of which regrettably remains the case for 97% of the
market and 99.99$ user base. Still ar apps will drive this rventually... I
think starlink is nicely positioned now to meet their demanding growth
goals and humanity's future in space assured, so there's that. ( i still
would rather like elone to send over a nice pair of tesla motors and
battery pack for my sailboat)

I did have a nice jam with ajit Pai last week who is now well on his way
towards getting it. (See my twitter for the pics)

On Mon, May 6, 2024, 4:25 AM Rich Brown via Starlink <
starlink@lists.bufferbloat.net> wrote:

> Hi Gene,
>
> I've been vacillating on whether to send this note, but have decided to
> pull the trigger. I apologize in advance for the "Debbie Downer" nature of
> this message. I also apologize for any errors, omissions, or
> over-simplifications of the "birth of bufferbloat" story and its fixes.
> Corrections welcome.
>
> Rich
> --
>
> If we are going to take a shot at opening people's eyes to bufferbloat, we
> should know some of the "objections" we'll run up against. Even though
> there's terrific technical data to back it up, people seem especially
> resistant to thinking that bufferbloat might affect their network, even
> when they're seeing problems that sound exactly like bufferbloat symptoms.
> But first, some history:
>
> The very idea of bufferbloat is simply unbelievable. Jim Gettys in 2011
> [1] couldn't believe it, and he's a smart networking guy,. At the time, it
> seemed incredible (that is "not credible" == impossible) that something
> could induce 1.2 seconds of latency into his home network connection. He
> called in favors from technical contacts at his ISP and at Bell Labs who
> went over everything with a fine-toothed comb. It was all exactly as
> spec'd. But he still had the latency.
>
> This led Jim and Dave Täht to start the investigation into the phenomenon
> known today as "bufferbloat" - the undesirable latency that comes from a
> router or other network equipment buffering too much data. Over several
> years, a group of smart people made huge improvements: fq_codel was
> released 14 May 2012 [3]; it was incorporated into the Linux kernel shortly
> afterward. CAKE came in 2015, and the fixes that minimize bufferbloat in
> Wi-Fi arrived in 2018. In 2021 cake-autorate [4] arrived to handle varying
> speed ISP links. All these techniques work great: in 2014, my 7mbps DSL
> link was quite usable. And when the pandemic hit, fq_codel on my OpenWrt
> router allowed me to use that same 7mbps DSL line for two simultaneous zoom
> calls.
>
> As one of the authors of [2], I am part of the team that has tried over
> the years to explain bufferbloat and how to fix it. We've spoken with
> vendors. We've spent untold hours responding to posts on assorted boards
> and forums with the the bufferbloat story.
>
> With these technical fixes in hand, we cockily set about to tell the world
> about how to fix bufferbloat. Our efforts have been met with skepticism at
> best, or stony silence. What are the objections?
>
> - This is just the ordinary behavior: I would expect things to be slower
> when there's more traffic (Willfully ignoring orders of magnitude increase
> in delay.)
> - Besides, I'm the only one using the internet. (Except when my phone
> uploads photos. Or my computer kicks off some automated process. Or I
> browse the web. Or ...)
> - It only happens some of the time. (Exactly. That's probably when
> 

Re: [Starlink] The "reasons" that bufferbloat isn't a problem

[Eugene Y Chang via Starlink]

Thanks Frank,
So it is not the universal cure. Not surprising.
I don’t see a show-stopper for pushing adoption.

Gene
--
Eugene Chang
IEEE Life Senior Member
IEEE Communications Society & Signal Processing Society,
Hawaii Chapter Chair
IEEE Life Member Affinity Group Hawaii Chair
IEEE Entrepreneurship, Mentor
eugene.ch...@ieee.org
m 781-799-0233 (in Honolulu)



> On May 7, 2024, at 10:05 AM, Frantisek Borsik  
> wrote:
> 
> Here is a current view of it, IIRC:
> 
> https://forum.openwrt.org/t/rfc9330-rfc9331-rfc9332-for-lower-latency/180519/12
>  
> 
> 
> All the best,
> 
> Frank
> 
> Frantisek (Frank) Borsik
> 
> 
> 
> https://www.linkedin.com/in/frantisekborsik 
> 
> Signal, Telegram, WhatsApp: +421919416714
> 
> iMessage, mobile: +420775230885
> 
> Skype: casioa5302ca
> 
> frantisek.bor...@gmail.com 
> 
> On Tue, May 7, 2024 at 10:03 PM Eugene Y Chang via Starlink 
> mailto:starlink@lists.bufferbloat.net>> 
> wrote:
> I thought I saw a reference to an OpenWRT implementation with L4S. How well 
> does that work?
> 
> 
> 
> Gene
> --
> Eugene Chang
> 
> 
> 
>> On May 7, 2024, at 9:46 AM, Dave Taht > > wrote:
>> 
>> Pete heist, jon morton, and rod grimes published a TON of research as
>> to where l4s went wrong in these github repos:
>> 
>> https://github.com/heistp 
>> 
>> The last was: 
>> https://github.com/heistp/l4s-tests?tab=readme-ov-file#key-findings 
>> 
>> 
>> They were ignored. Me, I had taken one look at it 7+ years ago and
>> said this cannot possibly work with the installed base of wifi
>> properly and since 97E% of all home connections terminate in that it
>> was a dead horse which they kept flogging.
>> 
>> After the L4S RFCs were published they FINALLY took their brands of
>> wishful thinking to actually exploring what happeed on real wifi
>> networks, and... I have no idea where that stands today. Yes a custom
>> wifi7 AP and presumably wifi8 will be able to deal with it, but
>> everything from the backoff mechanisms in the e2e TCP Prague code and
>> the proposed implementations on routers just plain does not work
>> except in a testbed. Fq_codel outperforms it across the board with
>> perhaps, some increased sensivity to RFC3168 seems needed only. L4S
>> (all transports actually) benefits a lot from packet pacing, and...
>> wait for it... fq)
>> 
>> Slow start and convergence issues are problematic also with l4s.
>> 
>> Being backward incompatible with fq_codel's deployed treatment of RFC3168 
>> ECN.
>> is a huge barrier too.
>> 
>> The best use case I can think of for l4s is on a tightly controlled
>> docsis network, pure wires and short RTTs only. The one implementation
>> for 5G I have heard of was laughable in that they were only aiming for
>> 200ms of induced latency on that.
>> 
>> If on the other hand you look at fq (and also how well starlink is
>> performing nowadays) and ccs like bbr, well...
>> 
>> I do honestly think there is room for this sort of signalling
>> somewhere on the internet, and do plan to add what I think will work
>> to cake at some point in the future. I do wish SCE had won, as it was
>> backwards compatible.
>> 
>> 
>> On Tue, May 7, 2024 at 12:15 PM Jeremy Austin > > wrote:
>>> 
>>> 
>>> 
>>> On Tue, May 7, 2024 at 11:11 AM Dave Taht via Starlink 
>>> mailto:starlink@lists.bufferbloat.net>> 
>>> wrote:
 
 The RFC is very plausible but the methods break down in multiple ways,
 particularly with wifi.
>>> 
>>> 
>>> Dave, can you elaborate more on the failures? Are these being researched or 
>>> addressed in the current trials, in your opinion?
>>> 
>>> Jeremy
>> 
>> 
>> 
>> --
>> https://www.youtube.com/watch?v=BVFWSyMp3xg=1098s 
>>  Waves Podcast
>> Dave Täht CSO, LibreQos
> 
> ___
> Starlink mailing list
> Starlink@lists.bufferbloat.net 
> https://lists.bufferbloat.net/listinfo/starlink 
> 



signature.asc
Description: Message signed with OpenPGP
___
Starlink mailing list
Starlink@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/starlink

Re: [Starlink] The "reasons" that bufferbloat isn't a problem

[Frantisek Borsik via Starlink]

Here is a current view of it, IIRC:

https://forum.openwrt.org/t/rfc9330-rfc9331-rfc9332-for-lower-latency/180519/12

All the best,

Frank

Frantisek (Frank) Borsik



https://www.linkedin.com/in/frantisekborsik

Signal, Telegram, WhatsApp: +421919416714

iMessage, mobile: +420775230885

Skype: casioa5302ca

frantisek.bor...@gmail.com


On Tue, May 7, 2024 at 10:03 PM Eugene Y Chang via Starlink <
starlink@lists.bufferbloat.net> wrote:

> I thought I saw a reference to an OpenWRT implementation with L4S. How
> well does that work?
>
>
>
> Gene
> --
> Eugene Chang
>
>
>
> On May 7, 2024, at 9:46 AM, Dave Taht  wrote:
>
> Pete heist, jon morton, and rod grimes published a TON of research as
> to where l4s went wrong in these github repos:
>
> https://github.com/heistp
>
> The last was:
> https://github.com/heistp/l4s-tests?tab=readme-ov-file#key-findings
>
> They were ignored. Me, I had taken one look at it 7+ years ago and
> said this cannot possibly work with the installed base of wifi
> properly and since 97E% of all home connections terminate in that it
> was a dead horse which they kept flogging.
>
> After the L4S RFCs were published they FINALLY took their brands of
> wishful thinking to actually exploring what happeed on real wifi
> networks, and... I have no idea where that stands today. Yes a custom
> wifi7 AP and presumably wifi8 will be able to deal with it, but
> everything from the backoff mechanisms in the e2e TCP Prague code and
> the proposed implementations on routers just plain does not work
> except in a testbed. Fq_codel outperforms it across the board with
> perhaps, some increased sensivity to RFC3168 seems needed only. L4S
> (all transports actually) benefits a lot from packet pacing, and...
> wait for it... fq)
>
> Slow start and convergence issues are problematic also with l4s.
>
> Being backward incompatible with fq_codel's deployed treatment of RFC3168
> ECN.
> is a huge barrier too.
>
> The best use case I can think of for l4s is on a tightly controlled
> docsis network, pure wires and short RTTs only. The one implementation
> for 5G I have heard of was laughable in that they were only aiming for
> 200ms of induced latency on that.
>
> If on the other hand you look at fq (and also how well starlink is
> performing nowadays) and ccs like bbr, well...
>
> I do honestly think there is room for this sort of signalling
> somewhere on the internet, and do plan to add what I think will work
> to cake at some point in the future. I do wish SCE had won, as it was
> backwards compatible.
>
>
> On Tue, May 7, 2024 at 12:15 PM Jeremy Austin  wrote:
>
>
>
>
> On Tue, May 7, 2024 at 11:11 AM Dave Taht via Starlink <
> starlink@lists.bufferbloat.net> wrote:
>
>
> The RFC is very plausible but the methods break down in multiple ways,
> particularly with wifi.
>
>
>
> Dave, can you elaborate more on the failures? Are these being researched
> or addressed in the current trials, in your opinion?
>
> Jeremy
>
>
>
>
> --
> https://www.youtube.com/watch?v=BVFWSyMp3xg=1098s Waves Podcast
> Dave Täht CSO, LibreQos
>
>
> ___
> Starlink mailing list
> Starlink@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/starlink
>
___
Starlink mailing list
Starlink@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/starlink

Re: [Starlink] The "reasons" that bufferbloat isn't a problem

[Eugene Y Chang via Starlink]

I thought I saw a reference to an OpenWRT implementation with L4S. How well 
does that work?



Gene
--
Eugene Chang



> On May 7, 2024, at 9:46 AM, Dave Taht  wrote:
> 
> Pete heist, jon morton, and rod grimes published a TON of research as
> to where l4s went wrong in these github repos:
> 
> https://github.com/heistp
> 
> The last was: 
> https://github.com/heistp/l4s-tests?tab=readme-ov-file#key-findings
> 
> They were ignored. Me, I had taken one look at it 7+ years ago and
> said this cannot possibly work with the installed base of wifi
> properly and since 97E% of all home connections terminate in that it
> was a dead horse which they kept flogging.
> 
> After the L4S RFCs were published they FINALLY took their brands of
> wishful thinking to actually exploring what happeed on real wifi
> networks, and... I have no idea where that stands today. Yes a custom
> wifi7 AP and presumably wifi8 will be able to deal with it, but
> everything from the backoff mechanisms in the e2e TCP Prague code and
> the proposed implementations on routers just plain does not work
> except in a testbed. Fq_codel outperforms it across the board with
> perhaps, some increased sensivity to RFC3168 seems needed only. L4S
> (all transports actually) benefits a lot from packet pacing, and...
> wait for it... fq)
> 
> Slow start and convergence issues are problematic also with l4s.
> 
> Being backward incompatible with fq_codel's deployed treatment of RFC3168 ECN.
> is a huge barrier too.
> 
> The best use case I can think of for l4s is on a tightly controlled
> docsis network, pure wires and short RTTs only. The one implementation
> for 5G I have heard of was laughable in that they were only aiming for
> 200ms of induced latency on that.
> 
> If on the other hand you look at fq (and also how well starlink is
> performing nowadays) and ccs like bbr, well...
> 
> I do honestly think there is room for this sort of signalling
> somewhere on the internet, and do plan to add what I think will work
> to cake at some point in the future. I do wish SCE had won, as it was
> backwards compatible.
> 
> 
> On Tue, May 7, 2024 at 12:15 PM Jeremy Austin  wrote:
>> 
>> 
>> 
>> On Tue, May 7, 2024 at 11:11 AM Dave Taht via Starlink 
>>  wrote:
>>> 
>>> The RFC is very plausible but the methods break down in multiple ways,
>>> particularly with wifi.
>> 
>> 
>> Dave, can you elaborate more on the failures? Are these being researched or 
>> addressed in the current trials, in your opinion?
>> 
>> Jeremy
> 
> 
> 
> --
> https://www.youtube.com/watch?v=BVFWSyMp3xg=1098s Waves Podcast
> Dave Täht CSO, LibreQos



signature.asc
Description: Message signed with OpenPGP
___
Starlink mailing list
Starlink@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/starlink

Re: [Starlink] The "reasons" that bufferbloat isn't a problem

[Dave Taht via Starlink]

Pete heist, jon morton, and rod grimes published a TON of research as
to where l4s went wrong in these github repos:

https://github.com/heistp

The last was: 
https://github.com/heistp/l4s-tests?tab=readme-ov-file#key-findings

They were ignored. Me, I had taken one look at it 7+ years ago and
said this cannot possibly work with the installed base of wifi
properly and since 97E% of all home connections terminate in that it
was a dead horse which they kept flogging.

After the L4S RFCs were published they FINALLY took their brands of
wishful thinking to actually exploring what happeed on real wifi
networks, and... I have no idea where that stands today. Yes a custom
wifi7 AP and presumably wifi8 will be able to deal with it, but
everything from the backoff mechanisms in the e2e TCP Prague code and
the proposed implementations on routers just plain does not work
except in a testbed. Fq_codel outperforms it across the board with
perhaps, some increased sensivity to RFC3168 seems needed only. L4S
(all transports actually) benefits a lot from packet pacing, and...
wait for it... fq)

Slow start and convergence issues are problematic also with l4s.

Being backward incompatible with fq_codel's deployed treatment of RFC3168 ECN.
 is a huge barrier too.

The best use case I can think of for l4s is on a tightly controlled
docsis network, pure wires and short RTTs only. The one implementation
for 5G I have heard of was laughable in that they were only aiming for
200ms of induced latency on that.

If on the other hand you look at fq (and also how well starlink is
performing nowadays) and ccs like bbr, well...

I do honestly think there is room for this sort of signalling
somewhere on the internet, and do plan to add what I think will work
to cake at some point in the future. I do wish SCE had won, as it was
backwards compatible.


On Tue, May 7, 2024 at 12:15 PM Jeremy Austin  wrote:
>
>
>
> On Tue, May 7, 2024 at 11:11 AM Dave Taht via Starlink 
>  wrote:
>>
>> The RFC is very plausible but the methods break down in multiple ways,
>> particularly with wifi.
>
>
> Dave, can you elaborate more on the failures? Are these being researched or 
> addressed in the current trials, in your opinion?
>
> Jeremy



--
https://www.youtube.com/watch?v=BVFWSyMp3xg=1098s Waves Podcast
Dave Täht CSO, LibreQos
___
Starlink mailing list
Starlink@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/starlink

Re: [Starlink] The "reasons" that bufferbloat isn't a problem

[Jeremy Austin via Starlink]

On Tue, May 7, 2024 at 11:11 AM Dave Taht via Starlink <
starlink@lists.bufferbloat.net> wrote:

> The RFC is very plausible but the methods break down in multiple ways,
> particularly with wifi.
>

Dave, can you elaborate more on the failures? Are these being researched or
addressed in the current trials, in your opinion?

Jeremy
___
Starlink mailing list
Starlink@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/starlink

Re: [Starlink] The "reasons" that bufferbloat isn't a problem

[Dave Taht via Starlink]

The RFC is very plausible but the methods break down in multiple ways,
particularly with wifi.

On Tue, May 7, 2024 at 12:10 PM Eugene Y Chang via Starlink
 wrote:
>
> Dave,
> Thank you for calling attention to the RFC. I took a quick peek, and I need 
> to put more time into reading the whole doc. It feels very intuitive.
>
> What I like is that it is written for incremental adoption. I will focus on 
> that in my next pass. It opens the door to be incrementally deployed to 
> pacify an influential squeaky wheel. I like the possibility that a happy 
> squeaky wheel becomes a role model attracting more squeaky wheels until it 
> makes more sense to just adopt broad deployment. If you read my earlier 
> emails, you know I am in the hunt for an influential squeaky wheel. :P
>
> Anticipating more discussion in this direction, are there core router vendors 
> that have a favorable view of  L4S? Are there router implementations just 
> waiting to be turned on?
>
> Gene
> --
> Eugene Chang
>
>
>
>
> On May 7, 2024, at 2:46 AM, Dave Collier-Brown via Starlink 
>  wrote:
>
> It has an RFC at https://datatracker.ietf.org/doc/rfc9330/
>
> I read it as a way to rapidly find the available bandwidth without the TCP 
> "sawtooth". The paper cites fc_codel and research based on it.
>
> I suspect My Smarter Colleagues know more (;-))
>
> --dave
>
>
> On 2024-05-07 08:13, David Fernández via Starlink wrote:
>
> Is L4S a solution to bufferbloat? I have read that gamers are happy with it.
>
> Sorry, I read it here, in Spanish:
> https://www.adslzone.net/noticias/operadores/retardo-videojuegos-nokia-vodafone
>
> Regards,
>
> David F.
>
> Date: Tue, 7 May 2024 06:50:41 -0400
> From: Rich Brown 
> To: Eugene Y Chang 
> Cc: Sebastian Moeller , Colin_Higbie
> , Dave Taht via Starlink
> 
> Subject: Re: [Starlink] The "reasons" that bufferbloat isn't a problem
> Message-ID: <175cc5c3-f70a-49e8-a84d-87e24c04e...@gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hi Gene,
>
> > On May 6, 2024, at 8:38 PM, Eugene Y Chang  wrote:
> >
> > It seems like you signed off on this challenge. Don’t do that. Help give me 
> > the tools to push this to the next level.
>
> Not at all - I'm definitely signed up for this. But I collected all these 
> points so we can be clear-eyed about the objections that people cite.
>
> Bufferbloat definitely exists. And there are straightforward technical 
> solutions. And as you say, our challenge is to find a way to build the case 
> for broad adoption of these techniques.
>
> Best regards,
>
> Rich
> -- next part --
> An HTML attachment was scrubbed...
> URL: 
> 
>
> ___
> Starlink mailing list
> Starlink@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/starlink
>
> --
> David Collier-Brown, | Always do right. This will gratify
> System Programmer and Author | some people and astonish the rest
> dave.collier-br...@indexexchange.com |  -- Mark Twain
>
>
> CONFIDENTIALITY NOTICE AND DISCLAIMER : This telecommunication, including any 
> and all attachments, contains confidential information intended only for the 
> person(s) to whom it is addressed. Any dissemination, distribution, copying 
> or disclosure is strictly prohibited and is not a waiver of confidentiality. 
> If you have received this telecommunication in error, please notify the 
> sender immediately by return electronic mail and delete the message from your 
> inbox and deleted items folders. This telecommunication does not constitute 
> an express or implied agreement to conduct transactions by electronic means, 
> nor does it constitute a contract offer, a contract amendment or an 
> acceptance of a contract offer. Contract terms contained in this 
> telecommunication are subject to legal review and the completion of formal 
> documentation and are not binding until same is confirmed in writing and has 
> been signed by an authorized signatory.
>
> ___
> Starlink mailing list
> Starlink@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/starlink
>
>
> ___
> Starlink mailing list
> Starlink@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/starlink



-- 
https://www.youtube.com/watch?v=BVFWSyMp3xg=1098s Waves Podcast
Dave Täht CSO, LibreQos
___
Starlink mailing list
Starlink@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/starlink

Re: [Starlink] The "reasons" that bufferbloat isn't a problem

[Eugene Y Chang via Starlink]

Dave,
Thank you for calling attention to the RFC. I took a quick peek, and I need to 
put more time into reading the whole doc. It feels very intuitive.

What I like is that it is written for incremental adoption. I will focus on 
that in my next pass. It opens the door to be incrementally deployed to pacify 
an influential squeaky wheel. I like the possibility that a happy squeaky wheel 
becomes a role model attracting more squeaky wheels until it makes more sense 
to just adopt broad deployment. If you read my earlier emails, you know I am in 
the hunt for an influential squeaky wheel. :P

Anticipating more discussion in this direction, are there core router vendors 
that have a favorable view of  L4S? Are there router implementations just 
waiting to be turned on?

Gene
--
Eugene Chang




> On May 7, 2024, at 2:46 AM, Dave Collier-Brown via Starlink 
>  wrote:
> 
> It has an RFC at https://datatracker.ietf.org/doc/rfc9330/ 
> 
> I read it as a way to rapidly find the available bandwidth without the TCP 
> "sawtooth". The paper cites fc_codel and research based on it.
> 
> I suspect My Smarter Colleagues know more (;-))
> 
> --dave
> 
> 
> On 2024-05-07 08:13, David Fernández via Starlink wrote:
>> Is L4S a solution to bufferbloat? I have read that gamers are happy with it.
>> 
>> Sorry, I read it here, in Spanish:
>> https://www.adslzone.net/noticias/operadores/retardo-videojuegos-nokia-vodafone
>>  
>> 
>> 
>> Regards,
>> 
>> David F.
>> 
>> Date: Tue, 7 May 2024 06:50:41 -0400
>> From: Rich Brown mailto:richb.hano...@gmail.com>>
>> To: Eugene Y Chang mailto:eugene.ch...@ieee.org>>
>> Cc: Sebastian Moeller mailto:moell...@gmx.de>>, 
>> Colin_Higbie
>>  , Dave Taht via 
>> Starlink
>> > >
>> Subject: Re: [Starlink] The "reasons" that bufferbloat isn't a problem
>> Message-ID: <175cc5c3-f70a-49e8-a84d-87e24c04e...@gmail.com 
>> >
>> Content-Type: text/plain; charset="utf-8"
>> 
>> Hi Gene,
>> 
>> > On May 6, 2024, at 8:38 PM, Eugene Y Chang > > > wrote:
>> >
>> > It seems like you signed off on this challenge. Don’t do that. Help give 
>> > me the tools to push this to the next level.
>> 
>> Not at all - I'm definitely signed up for this. But I collected all these 
>> points so we can be clear-eyed about the objections that people cite.
>> 
>> Bufferbloat definitely exists. And there are straightforward technical 
>> solutions. And as you say, our challenge is to find a way to build the case 
>> for broad adoption of these techniques.
>> 
>> Best regards,
>> 
>> Rich
>> -- next part --
>> An HTML attachment was scrubbed...
>> URL: 
>> >  
>> >
>> 
>> 
>> ___
>> Starlink mailing list
>> Starlink@lists.bufferbloat.net 
>> https://lists.bufferbloat.net/listinfo/starlink 
>> 
> --
> David Collier-Brown, | Always do right. This will gratify
> System Programmer and Author | some people and astonish the rest
> dave.collier-br...@indexexchange.com 
>  |  -- Mark Twain
> 
> CONFIDENTIALITY NOTICE AND DISCLAIMER : This telecommunication, including any 
> and all attachments, contains confidential information intended only for the 
> person(s) to whom it is addressed. Any dissemination, distribution, copying 
> or disclosure is strictly prohibited and is not a waiver of confidentiality. 
> If you have received this telecommunication in error, please notify the 
> sender immediately by return electronic mail and delete the message from your 
> inbox and deleted items folders. This telecommunication does not constitute 
> an express or implied agreement to conduct transactions by electronic means, 
> nor does it constitute a contract offer, a contract amendment or an 
> acceptance of a contract offer. Contract terms contained in this 
> telecommunication are subject to legal review and the completion of formal 
> documentation and are not binding until same is confirmed in writing and has 
> been signed by an authorized signatory.
> 
> ___
> Starlink mailing list
> Starlink@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/starlink



signature.asc
Description: Message signed with OpenPGP
___
Starlink mailing list
Starlink@lists.bufferbloat.net

Re: [Starlink] The "reasons" that bufferbloat isn't a problem

[Dave Collier-Brown via Starlink]

It has an RFC at https://datatracker.ietf.org/doc/rfc9330/

I read it as a way to rapidly find the available bandwidth without the TCP 
"sawtooth". The paper cites fc_codel and research based on it.

I suspect My Smarter Colleagues know more (;-))

--dave



On 2024-05-07 08:13, David Fernández via Starlink wrote:
Is L4S a solution to bufferbloat? I have read that gamers are happy with it.

Sorry, I read it here, in Spanish:
https://www.adslzone.net/noticias/operadores/retardo-videojuegos-nokia-vodafone

Regards,

David F.

Date: Tue, 7 May 2024 06:50:41 -0400
From: Rich Brown mailto:richb.hano...@gmail.com>>
To: Eugene Y Chang mailto:eugene.ch...@ieee.org>>
Cc: Sebastian Moeller mailto:moell...@gmx.de>>, Colin_Higbie
   , Dave Taht via 
Starlink
   mailto:starlink@lists.bufferbloat.net>>
Subject: Re: [Starlink] The "reasons" that bufferbloat isn't a problem
Message-ID: 
<175cc5c3-f70a-49e8-a84d-87e24c04e...@gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi Gene,


On May 6, 2024, at 8:38 PM, Eugene Y Chang 
mailto:eugene.ch...@ieee.org>> wrote:

It seems like you signed off on this challenge. Don’t do that. Help give me the 
tools to push this to the next level.


Not at all - I'm definitely signed up for this. But I collected all these 
points so we can be clear-eyed about the objections that people cite.

Bufferbloat definitely exists. And there are straightforward technical 
solutions. And as you say, our challenge is to find a way to build the case for 
broad adoption of these techniques.

Best regards,

Rich
-- next part --
An HTML attachment was scrubbed...
URL: 




___
Starlink mailing list
Starlink@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/starlink


--
David Collier-Brown, | Always do right. This will gratify
System Programmer and Author | some people and astonish the rest
dave.collier-br...@indexexchange.com
 |  -- Mark Twain


CONFIDENTIALITY NOTICE AND DISCLAIMER : This telecommunication, including any 
and all attachments, contains confidential information intended only for the 
person(s) to whom it is addressed. Any dissemination, distribution, copying or 
disclosure is strictly prohibited and is not a waiver of confidentiality. If 
you have received this telecommunication in error, please notify the sender 
immediately by return electronic mail and delete the message from your inbox 
and deleted items folders. This telecommunication does not constitute an 
express or implied agreement to conduct transactions by electronic means, nor 
does it constitute a contract offer, a contract amendment or an acceptance of a 
contract offer. Contract terms contained in this telecommunication are subject 
to legal review and the completion of formal documentation and are not binding 
until same is confirmed in writing and has been signed by an authorized 
signatory.
___
Starlink mailing list
Starlink@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/starlink

Re: [Starlink] The "reasons" that bufferbloat isn't a problem

[David Fernández via Starlink]

Is L4S a solution to bufferbloat? I have read that gamers are happy with it.

Sorry, I read it here, in Spanish:
https://www.adslzone.net/noticias/operadores/retardo-videojuegos-nokia-vodafone

Regards,

David F.

Date: Tue, 7 May 2024 06:50:41 -0400
From: Rich Brown 
To: Eugene Y Chang 
Cc: Sebastian Moeller , Colin_Higbie
, Dave Taht via Starlink

Subject: Re: [Starlink] The "reasons" that bufferbloat isn't a problem
Message-ID: <175cc5c3-f70a-49e8-a84d-87e24c04e...@gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi Gene,

> On May 6, 2024, at 8:38 PM, Eugene Y Chang  wrote:
>
> It seems like you signed off on this challenge. Don’t do that. Help give
me the tools to push this to the next level.

Not at all - I'm definitely signed up for this. But I collected all these
points so we can be clear-eyed about the objections that people cite.

Bufferbloat definitely exists. And there are straightforward technical
solutions. And as you say, our challenge is to find a way to build the case
for broad adoption of these techniques.

Best regards,

Rich
-- next part --
An HTML attachment was scrubbed...
URL: <
https://lists.bufferbloat.net/pipermail/starlink/attachments/20240507/ecb7b91e/attachment-0001.html>
___
Starlink mailing list
Starlink@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/starlink

Re: [Starlink] The "reasons" that bufferbloat isn't a problem

[Dave Collier-Brown via Starlink]

Oh goodness, I wasn't suggesting that we had a total solution, I was 
pointing out that the gaming community was missing the point, even with 
evidence in their hands.


That suggests we have not made the point to a technically aware part of 
our community.


--dave

On 2024-05-06 20:43, Eugene Y Chang via Starlink wrote:

Dave,
We just can’t represent that we have the total solution.
We need to show the problem can be reduced.
We need to show that latency is a significant negative phenomena.
Take out one contributor and sic the users to the next contributor.

If we expect to solve the whole problem in one step, we end up where 
we are and effectively say the problem is too complex to solve.



Gene
--
Eugene Chang
IEEE Life Senior Member
IEEE Communications Society & Signal Processing Society,
    Hawaii Chapter Chair
IEEE Life Member Affinity Group Hawaii Chair
IEEE Entrepreneurship, Mentor
eugene.ch...@ieee.org
m 781-799-0233 (in Honolulu)



On May 6, 2024, at 2:11 AM, Dave Collier-Brown via Starlink 
 wrote:


I think that gamer experience doing simple (over-simple) tests with 
CAKE is a booby-trap. This discussion suggests that the real 
performance of their link is horrid, and that they turn off CAKE to 
get what they think is full performance... but isn't.


https://www.reddit.com/r/HomeNetworking/comments/174k0ko/low_latency_gaming_and_bufferbloat/#:~:text=If%20there's%20any%20chance%20that,out%20any%20intermittent%20latency%20spikes. 



 (I used to work for World Gaming, and follow the game commentators 
more that I do now)


--dave


On 2024-05-06 07:25, Rich Brown via Starlink wrote:

Hi Gene,

I've been vacillating on whether to send this note, but have decided 
to pull the trigger. I apologize in advance for the "Debbie Downer" 
nature of this message. I also apologize for any errors, omissions, 
or over-simplifications of the "birth of bufferbloat" story and its 
fixes. Corrections welcome.


Rich
--

If we are going to take a shot at opening people's eyes to 
bufferbloat, we should know some of the "objections" we'll run up 
against. Even though there's terrific technical data to back it up, 
people seem especially resistant to thinking that bufferbloat might 
affect their network, even when they're seeing problems that sound 
exactly like bufferbloat symptoms. But first, some history:


The very idea of bufferbloat is simply unbelievable. Jim Gettys in 
2011 [1] couldn't believe it, and he's a smart networking guy,. At 
the time, it seemed incredible (that is "not credible" == 
impossible) that something could induce 1.2 seconds of latency into 
his home network connection. He called in favors from technical 
contacts at his ISP and at Bell Labs who went over everything with a 
fine-toothed comb. It was all exactly as spec'd. But he still had 
the latency.


This led Jim and Dave Täht to start the investigation into the 
phenomenon known today as "bufferbloat" - the undesirable latency 
that comes from a router or other network equipment buffering too 
much data. Over several years, a group of smart people made huge 
improvements: fq_codel was released 14 May 2012 [3]; it was 
incorporated into the Linux kernel shortly afterward. CAKE came in 
2015, and the fixes that minimize bufferbloat in Wi-Fi arrived in 
2018. In 2021 cake-autorate [4] arrived to handle varying speed ISP 
links. All these techniques work great: in 2014, my 7mbps DSL link 
was quite usable. And when the pandemic hit, fq_codel on my OpenWrt 
router allowed me to use that same 7mbps DSL line for two 
simultaneous zoom calls.


As one of the authors of [2], I am part of the team that has tried 
over the years to explain bufferbloat and how to fix it. We've 
spoken with vendors. We've spent untold hours responding to posts on 
assorted boards and forums with the the bufferbloat story.


With these technical fixes in hand, we cockily set about to tell the 
world about how to fix bufferbloat. Our efforts have been met with 
skepticism at best, or stony silence. What are the objections?


- This is just the ordinary behavior: I would expect things to be 
slower when there's more traffic (Willfully ignoring orders of 
magnitude increase in delay.)
- Besides, I'm the only one using the internet. (Except when my 
phone uploads photos. Or my computer kicks off some automated 
process. Or I browse the web. Or ...)
- It only happens some of the time. (Exactly. That's probably when 
something's uploading photos, or your computer is doing stuff in the 
background.)
- Those bufferbloat tests you hear about are bogus. They 
artificially add load, which isn't a realistic test. (...and if you 
actually are downloading a file?)
- Bufferbloat only happens when the network is 100% loaded. (True. 
But when you open a web page, your browser briefly uses 100% of the 
link. Is this enough to cause momentary lag?)
- It's OK. I just tell my kids/spouse not to use the internet when 
I'm gaming. 

Re: [Starlink] The "reasons" that bufferbloat isn't a problem

[Rich Brown via Starlink]

Hi Gene,

> On May 6, 2024, at 8:38 PM, Eugene Y Chang  wrote:
> 
> It seems like you signed off on this challenge. Don’t do that. Help give me 
> the tools to push this to the next level.

Not at all - I'm definitely signed up for this. But I collected all these 
points so we can be clear-eyed about the objections that people cite. 

Bufferbloat definitely exists. And there are straightforward technical 
solutions. And as you say, our challenge is to find a way to build the case for 
broad adoption of these techniques. 

Best regards,

Rich___
Starlink mailing list
Starlink@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/starlink

Re: [Starlink] The "reasons" that bufferbloat isn't a problem

[Eugene Y Chang via Starlink]

Dave,
We just can’t represent that we have the total solution.
We need to show the problem can be reduced.
We need to show that latency is a significant negative phenomena.
Take out one contributor and sic the users to the next contributor.

If we expect to solve the whole problem in one step, we end up where we are and 
effectively say the problem is too complex to solve.


Gene
--
Eugene Chang
IEEE Life Senior Member
IEEE Communications Society & Signal Processing Society,
Hawaii Chapter Chair
IEEE Life Member Affinity Group Hawaii Chair
IEEE Entrepreneurship, Mentor
eugene.ch...@ieee.org
m 781-799-0233 (in Honolulu)



> On May 6, 2024, at 2:11 AM, Dave Collier-Brown via Starlink 
>  wrote:
> 
> I think that gamer experience doing simple (over-simple) tests with CAKE is a 
> booby-trap. This discussion suggests that the real performance of their link 
> is horrid, and that they turn off CAKE to get what they think is full 
> performance... but isn't.
> 
> https://www.reddit.com/r/HomeNetworking/comments/174k0ko/low_latency_gaming_and_bufferbloat/#:~:text=If%20there's%20any%20chance%20that,out%20any%20intermittent%20latency%20spikes
>  
> .
>  (I used to work for World Gaming, and follow the game commentators more that 
> I do now)
> 
> --dave
> 
> 
> On 2024-05-06 07:25, Rich Brown via Starlink wrote:
>> Hi Gene,
>> 
>> I've been vacillating on whether to send this note, but have decided to pull 
>> the trigger. I apologize in advance for the "Debbie Downer" nature of this 
>> message. I also apologize for any errors, omissions, or over-simplifications 
>> of the "birth of bufferbloat" story and its fixes. Corrections welcome.
>> 
>> Rich
>> --
>> 
>> If we are going to take a shot at opening people's eyes to bufferbloat, we 
>> should know some of the "objections" we'll run up against. Even though 
>> there's terrific technical data to back it up, people seem especially 
>> resistant to thinking that bufferbloat might affect their network, even when 
>> they're seeing problems that sound exactly like bufferbloat symptoms. But 
>> first, some history:
>> 
>> The very idea of bufferbloat is simply unbelievable. Jim Gettys in 2011 [1] 
>> couldn't believe it, and he's a smart networking guy,. At the time, it 
>> seemed incredible (that is "not credible" == impossible) that something 
>> could induce 1.2 seconds of latency into his home network connection. He 
>> called in favors from technical contacts at his ISP and at Bell Labs who 
>> went over everything with a fine-toothed comb. It was all exactly as spec'd. 
>> But he still had the latency.
>> 
>> This led Jim and Dave Täht to start the investigation into the phenomenon 
>> known today as "bufferbloat" - the undesirable latency that comes from a 
>> router or other network equipment buffering too much data. Over several 
>> years, a group of smart people made huge improvements: fq_codel was released 
>> 14 May 2012 [3]; it was incorporated into the Linux kernel shortly 
>> afterward. CAKE came in 2015, and the fixes that minimize bufferbloat in 
>> Wi-Fi arrived in 2018. In 2021 cake-autorate [4] arrived to handle varying 
>> speed ISP links. All these techniques work great: in 2014, my 7mbps DSL link 
>> was quite usable. And when the pandemic hit, fq_codel on my OpenWrt router 
>> allowed me to use that same 7mbps DSL line for two simultaneous zoom calls.
>> 
>> As one of the authors of [2], I am part of the team that has tried over the 
>> years to explain bufferbloat and how to fix it. We've spoken with vendors. 
>> We've spent untold hours responding to posts on assorted boards and forums 
>> with the the bufferbloat story.
>> 
>> With these technical fixes in hand, we cockily set about to tell the world 
>> about how to fix bufferbloat. Our efforts have been met with skepticism at 
>> best, or stony silence. What are the objections?
>> 
>> - This is just the ordinary behavior: I would expect things to be slower 
>> when there's more traffic (Willfully ignoring orders of magnitude increase 
>> in delay.)
>> - Besides, I'm the only one using the internet. (Except when my phone 
>> uploads photos. Or my computer kicks off some automated process. Or I browse 
>> the web. Or ...)
>> - It only happens some of the time. (Exactly. That's probably when 
>> something's uploading photos, or your computer is doing stuff in the 
>> background.)
>> - Those bufferbloat tests you hear about are bogus. They artificially add 
>> load, which isn't a realistic test. (...and if you actually are downloading 
>> a file?)
>> - Bufferbloat only happens when the network is 100% loaded. (True. But when 
>> you open a web page, your browser briefly uses 100% of the link. Is this 
>> enough to cause momentary lag?)
>> - It's OK. I just tell 

Re: [Starlink] The "reasons" that bufferbloat isn't a problem

[Eugene Y Chang via Starlink]

Rich,
Thanks for the recap in the email.
I have seen all of those bits.

I will help with the marketing magic needed.
We need a team of smart people engaged to help vouch for the technical 
integrity.

We need a simple case (call it a special case, if you must) that shows the 
problem can be fixed.
Never mind if it is not a universal fix.
We only need to show one happy, very visible community.
Give me something to work with that we can defend from the list of do-nothing 
reasons you list.

It seems like you signed off on this challenge. Don’t do that. Help give me the 
tools to push this to the next level.
An energetic, vocal community is very valuable. They aren’t satisfying if we 
want to debate the technology. We shouldn’t care. We just want to win adoption.

Gene
--
Eugene Chang
IEEE Life Senior Member
IEEE Communications Society & Signal Processing Society,
Hawaii Chapter Chair
IEEE Life Member Affinity Group Hawaii Chair
IEEE Entrepreneurship, Mentor
eugene.ch...@ieee.org
m 781-799-0233 (in Honolulu)



> On May 6, 2024, at 1:25 AM, Rich Brown  wrote:
> 
> Hi Gene,
> 
> I've been vacillating on whether to send this note, but have decided to pull 
> the trigger. I apologize in advance for the "Debbie Downer" nature of this 
> message. I also apologize for any errors, omissions, or over-simplifications 
> of the "birth of bufferbloat" story and its fixes. Corrections welcome.
> 
> Rich
> --
> 
> If we are going to take a shot at opening people's eyes to bufferbloat, we 
> should know some of the "objections" we'll run up against. Even though 
> there's terrific technical data to back it up, people seem especially 
> resistant to thinking that bufferbloat might affect their network, even when 
> they're seeing problems that sound exactly like bufferbloat symptoms. But 
> first, some history:
> 
> The very idea of bufferbloat is simply unbelievable. Jim Gettys in 2011 [1] 
> couldn't believe it, and he's a smart networking guy,. At the time, it seemed 
> incredible (that is "not credible" == impossible) that something could induce 
> 1.2 seconds of latency into his home network connection. He called in favors 
> from technical contacts at his ISP and at Bell Labs who went over everything 
> with a fine-toothed comb. It was all exactly as spec'd. But he still had the 
> latency.
> 
> This led Jim and Dave Täht to start the investigation into the phenomenon 
> known today as "bufferbloat" - the undesirable latency that comes from a 
> router or other network equipment buffering too much data. Over several 
> years, a group of smart people made huge improvements: fq_codel was released 
> 14 May 2012 [3]; it was incorporated into the Linux kernel shortly afterward. 
> CAKE came in 2015, and the fixes that minimize bufferbloat in Wi-Fi arrived 
> in 2018. In 2021 cake-autorate [4] arrived to handle varying speed ISP links. 
> All these techniques work great: in 2014, my 7mbps DSL link was quite usable. 
> And when the pandemic hit, fq_codel on my OpenWrt router allowed me to use 
> that same 7mbps DSL line for two simultaneous zoom calls.
> 
> As one of the authors of [2], I am part of the team that has tried over the 
> years to explain bufferbloat and how to fix it. We've spoken with vendors. 
> We've spent untold hours responding to posts on assorted boards and forums 
> with the the bufferbloat story.
> 
> With these technical fixes in hand, we cockily set about to tell the world 
> about how to fix bufferbloat. Our efforts have been met with skepticism at 
> best, or stony silence. What are the objections?
> 
> - This is just the ordinary behavior: I would expect things to be slower when 
> there's more traffic (Willfully ignoring orders of magnitude increase in 
> delay.)
> - Besides, I'm the only one using the internet. (Except when my phone uploads 
> photos. Or my computer kicks off some automated process. Or I browse the web. 
> Or ...)
> - It only happens some of the time. (Exactly. That's probably when 
> something's uploading photos, or your computer is doing stuff in the 
> background.)
> - Those bufferbloat tests you hear about are bogus. They artificially add 
> load, which isn't a realistic test. (...and if you actually are downloading a 
> file?)
> - Bufferbloat only happens when the network is 100% loaded. (True. But when 
> you open a web page, your browser briefly uses 100% of the link. Is this 
> enough to cause momentary lag?)
> - It's OK. I just tell my kids/spouse not to use the internet when I'm 
> gaming. (Huh?)
> - I have gigabit service from my ISP. (That helps, but if you're complaining 
> about "slowness" you still need to rule out bufferbloat in your router.)
> - I can't believe that router manufacturers would ever allow such a thing to 
> happen in their gear. (See the Jim Gettys story above.)
> - I mean... wouldn't router vendors want to provide the best for their 
> customers? (No - implementing this (new-ish) code 

Re: [Starlink] The "reasons" that bufferbloat isn't a problem

[Rich Brown via Starlink]

Thanks! I just posted to: 
https://randomneuronsfiring.com/all-the-reasons-that-bufferbloat-isnt-a-problem/
 

It has mild edits from the original to address a broader audience. Also posted 
to the bloat list.

Rich

> On May 6, 2024, at 3:05 PM, Frantisek Borsik  
> wrote:
> 
> Hey Rich,
> 
> This was really great trip down the memory lane.
> 
> Could you please publish it somewhere, like on your blog?
> 
> Would be great to share it with the world!
> 
> Greetings from Prague.
> 
> All the best,
> 
> Frank
> 
> Frantisek (Frank) Borsik
> 

___
Starlink mailing list
Starlink@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/starlink

Re: [Starlink] The "reasons" that bufferbloat isn't a problem

[Dave Collier-Brown via Starlink]

I think that gamer experience doing simple (over-simple) tests with CAKE is a 
booby-trap. This discussion suggests that the real performance of their link is 
horrid, and that they turn off CAKE to get what they think is full 
performance... but isn't.

https://www.reddit.com/r/HomeNetworking/comments/174k0ko/low_latency_gaming_and_bufferbloat/#:~:text=If%20there's%20any%20chance%20that,out%20any%20intermittent%20latency%20spikes.

(I used to work for World Gaming, and follow the game commentators more that I 
do now)

--dave

On 2024-05-06 07:25, Rich Brown via Starlink wrote:
Hi Gene,

I've been vacillating on whether to send this note, but have decided to pull the trigger. I 
apologize in advance for the "Debbie Downer" nature of this message. I also apologize for 
any errors, omissions, or over-simplifications of the "birth of bufferbloat" story and 
its fixes. Corrections welcome.

Rich
--

If we are going to take a shot at opening people's eyes to bufferbloat, we should know 
some of the "objections" we'll run up against. Even though there's terrific 
technical data to back it up, people seem especially resistant to thinking that 
bufferbloat might affect their network, even when they're seeing problems that sound 
exactly like bufferbloat symptoms. But first, some history:

The very idea of bufferbloat is simply unbelievable. Jim Gettys in 2011 [1] couldn't 
believe it, and he's a smart networking guy,. At the time, it seemed incredible (that is 
"not credible" == impossible) that something could induce 1.2 seconds of 
latency into his home network connection. He called in favors from technical contacts at 
his ISP and at Bell Labs who went over everything with a fine-toothed comb. It was all 
exactly as spec'd. But he still had the latency.

This led Jim and Dave Täht to start the investigation into the phenomenon known today as 
"bufferbloat" - the undesirable latency that comes from a router or other 
network equipment buffering too much data. Over several years, a group of smart people 
made huge improvements: fq_codel was released 14 May 2012 [3]; it was incorporated into 
the Linux kernel shortly afterward. CAKE came in 2015, and the fixes that minimize 
bufferbloat in Wi-Fi arrived in 2018. In 2021 cake-autorate [4] arrived to handle varying 
speed ISP links. All these techniques work great: in 2014, my 7mbps DSL link was quite 
usable. And when the pandemic hit, fq_codel on my OpenWrt router allowed me to use that 
same 7mbps DSL line for two simultaneous zoom calls.

As one of the authors of [2], I am part of the team that has tried over the 
years to explain bufferbloat and how to fix it. We've spoken with vendors. 
We've spent untold hours responding to posts on assorted boards and forums with 
the the bufferbloat story.

With these technical fixes in hand, we cockily set about to tell the world 
about how to fix bufferbloat. Our efforts have been met with skepticism at 
best, or stony silence. What are the objections?

- This is just the ordinary behavior: I would expect things to be slower when 
there's more traffic (Willfully ignoring orders of magnitude increase in delay.)
- Besides, I'm the only one using the internet. (Except when my phone uploads 
photos. Or my computer kicks off some automated process. Or I browse the web. 
Or ...)
- It only happens some of the time. (Exactly. That's probably when something's 
uploading photos, or your computer is doing stuff in the background.)
- Those bufferbloat tests you hear about are bogus. They artificially add load, 
which isn't a realistic test. (...and if you actually are downloading a file?)
- Bufferbloat only happens when the network is 100% loaded. (True. But when you 
open a web page, your browser briefly uses 100% of the link. Is this enough to 
cause momentary lag?)
- It's OK. I just tell my kids/spouse not to use the internet when I'm gaming. 
(Huh?)
- I have gigabit service from my ISP. (That helps, but if you're complaining about 
"slowness" you still need to rule out bufferbloat in your router.)
- I can't believe that router manufacturers would ever allow such a thing to 
happen in their gear. (See the Jim Gettys story above.)
- I mean... wouldn't router vendors want to provide the best for their customers? (No - 
implementing this (new-ish) code requires engineering effort. They're selling plenty of 
routers with decade-old software. The Boss says, "would we sell more if they made 
these changes? Probably not.")
- Why would my ISP provision/sell me a router that gave crappy service? They're 
a big company, they must know about this stuff. (Maybe. We have reached out to 
all the vendors. But remember they profit if you decide your network is too 
slow and you upgrade to a faster device/plan.)
- But couldn't I just tweak the QoS on my router? (Maybe. But see [5])
- Besides, I just spent $300 on a "gaming router". Obviously, I bought the most 
expensive/best possible solution on the market (But I still have lag...)
- 

[Starlink] The "reasons" that bufferbloat isn't a problem

[Rich Brown via Starlink]

Hi Gene,

I've been vacillating on whether to send this note, but have decided to pull 
the trigger. I apologize in advance for the "Debbie Downer" nature of this 
message. I also apologize for any errors, omissions, or over-simplifications of 
the "birth of bufferbloat" story and its fixes. Corrections welcome.

Rich
--

If we are going to take a shot at opening people's eyes to bufferbloat, we 
should know some of the "objections" we'll run up against. Even though there's 
terrific technical data to back it up, people seem especially resistant to 
thinking that bufferbloat might affect their network, even when they're seeing 
problems that sound exactly like bufferbloat symptoms. But first, some history:

The very idea of bufferbloat is simply unbelievable. Jim Gettys in 2011 [1] 
couldn't believe it, and he's a smart networking guy,. At the time, it seemed 
incredible (that is "not credible" == impossible) that something could induce 
1.2 seconds of latency into his home network connection. He called in favors 
from technical contacts at his ISP and at Bell Labs who went over everything 
with a fine-toothed comb. It was all exactly as spec'd. But he still had the 
latency. 

This led Jim and Dave Täht to start the investigation into the phenomenon known 
today as "bufferbloat" - the undesirable latency that comes from a router or 
other network equipment buffering too much data. Over several years, a group of 
smart people made huge improvements: fq_codel was released 14 May 2012 [3]; it 
was incorporated into the Linux kernel shortly afterward. CAKE came in 2015, 
and the fixes that minimize bufferbloat in Wi-Fi arrived in 2018. In 2021 
cake-autorate [4] arrived to handle varying speed ISP links. All these 
techniques work great: in 2014, my 7mbps DSL link was quite usable. And when 
the pandemic hit, fq_codel on my OpenWrt router allowed me to use that same 
7mbps DSL line for two simultaneous zoom calls. 

As one of the authors of [2], I am part of the team that has tried over the 
years to explain bufferbloat and how to fix it. We've spoken with vendors. 
We've spent untold hours responding to posts on assorted boards and forums with 
the the bufferbloat story. 

With these technical fixes in hand, we cockily set about to tell the world 
about how to fix bufferbloat. Our efforts have been met with skepticism at 
best, or stony silence. What are the objections? 

- This is just the ordinary behavior: I would expect things to be slower when 
there's more traffic (Willfully ignoring orders of magnitude increase in delay.)
- Besides, I'm the only one using the internet. (Except when my phone uploads 
photos. Or my computer kicks off some automated process. Or I browse the web. 
Or ...)
- It only happens some of the time. (Exactly. That's probably when something's 
uploading photos, or your computer is doing stuff in the background.)
- Those bufferbloat tests you hear about are bogus. They artificially add load, 
which isn't a realistic test. (...and if you actually are downloading a file?)
- Bufferbloat only happens when the network is 100% loaded. (True. But when you 
open a web page, your browser briefly uses 100% of the link. Is this enough to 
cause momentary lag?)
- It's OK. I just tell my kids/spouse not to use the internet when I'm gaming. 
(Huh?)
- I have gigabit service from my ISP. (That helps, but if you're complaining 
about "slowness" you still need to rule out bufferbloat in your router.)
- I can't believe that router manufacturers would ever allow such a thing to 
happen in their gear. (See the Jim Gettys story above.)
- I mean... wouldn't router vendors want to provide the best for their 
customers? (No - implementing this (new-ish) code requires engineering effort. 
They're selling plenty of routers with decade-old software. The Boss says, 
"would we sell more if they made these changes? Probably not.")
- Why would my ISP provision/sell me a router that gave crappy service? They're 
a big company, they must know about this stuff. (Maybe. We have reached out to 
all the vendors. But remember they profit if you decide your network is too 
slow and you upgrade to a faster device/plan.)
- But couldn't I just tweak the QoS on my router? (Maybe. But see [5])
- Besides, I just spent $300 on a "gaming router". Obviously, I bought the most 
expensive/best possible solution on the market (But I still have lag...)
- You're telling me that a bunch of pointy-headed academics are smarter than 
commercial router developers - who sold me that $300 router? (I can't believe 
it.)
- And then you say that I should throw away that gaming router and install some 
"open source firmware"? (What the heck is that? And why should I believe you?) 
- What if it doesn't solve the problem? Who will give me support? And how will 
I get back to a vendor-supported system? (Valid point - the first valid point)
- Aren't there any commercial solutions I can just buy? (Not at the moment. 
IQrouter was a shining light