[Bloat] fq_codel Day - it's twelve years old!

[Rich Brown via Bloat]

My calendar reminds me that fq_codel is TWELVE YEARS OLD today! Dave Täht 
announced a testable version on the CeroWrt list at: 
https://lists.bufferbloat.net/pipermail/cerowrt-devel/2012-May/000233.html

Since then, fq_codel, CAKE, TX Queues, AQL, ATF, and cake-autorate have all 
made their way into good products. (See 
https://forum.openwrt.org/t/how-openwrt-vanquishes-bufferbloat/189381 for 
details.)

Congratulations, everyone!

Rich
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] [Starlink] L4S

[Rich Brown via Bloat]

Let's split this thread and use this message to continue the discussion of L4S. 
Thanks

> On May 8, 2024, at 5:31 AM, David Fernández via Starlink 
>  wrote:
> 
> I see that L4S is not really solving everything (I read about issues with 
> Wi-Fi), although it seems to be a step in the right direction, to be 
> improved, let's hope.
> 
> At least, Nokia is implementing it in its network gear (for mobile 
> operators), so the bufferbloat problem is somehow acknowledged by industry, 
> at least initially or partially.
> 
> I have seen two consecutive RFCs to 9330:
> https://www.rfc-editor.org/info/rfc9331 
> 
> https://www.rfc-editor.org/info/rfc9332 
> 
> 
> I suspect that optimal results require the bufferbloat to be addressed not 
> only at network layer (IP), but also with some pipelining or cross-layering 
> at link level (Ethernet, Wi-Fi or any other link technology, such as 5G, 
> SATCOM, VHF...)
> 
> Regards,
> 
> David F.
> 
> Date: Tue, 7 May 2024 08:46:03 -0400
> From: Dave Collier-Brown  >
> To: starl...@lists.bufferbloat.net 
> Subject: Re: [Starlink] The "reasons" that bufferbloat isn't a problem
> Message-ID: <3d6bdccf-e3d1-4f62-a029-25bfd1f45...@indexexchange.com 
> >
> Content-Type: text/plain; charset="utf-8"; Format="flowed"
> 
> It has an RFC at https://datatracker.ietf.org/doc/rfc9330/ 
> 
> 
> I read it as a way to rapidly find the available bandwidth without the TCP 
> "sawtooth". The paper cites fc_codel and research based on it.
> 
> I suspect My Smarter Colleagues know more (;-))
> 
> --dave
> 
> 
> 
> On 2024-05-07 08:13, David Fernández via Starlink wrote:
> Is L4S a solution to bufferbloat? I have read that gamers are happy with it.
> 
> Sorry, I read it here, in Spanish:
> https://www.adslzone.net/noticias/operadores/retardo-videojuegos-nokia-vodafone
>  
> 
> 
> Regards,
> 
> David F.
> ___
> Starlink mailing list
> starl...@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/starlink

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] [Starlink] The "reasons" that bufferbloat isn't a problem

[Rich Brown via Bloat]

Thanks! I just posted to: 
https://randomneuronsfiring.com/all-the-reasons-that-bufferbloat-isnt-a-problem/
 

It has mild edits from the original to address a broader audience. Also posted 
to the bloat list.

Rich

> On May 6, 2024, at 3:05 PM, Frantisek Borsik  
> wrote:
> 
> Hey Rich,
> 
> This was really great trip down the memory lane.
> 
> Could you please publish it somewhere, like on your blog?
> 
> Would be great to share it with the world!
> 
> Greetings from Prague.
> 
> All the best,
> 
> Frank
> 
> Frantisek (Frank) Borsik
> 

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] Need help with netperf.bufferbloat.net server

[Rich Brown via Bloat]

Hi folks,

This note was prompted by a question from the crusader github repo [1] where I 
wrote the following:

>> It seems to me that the server netperf.bufferbloat.net (also called 
>> netperf-east.bufferbloat.net) has been down for quite a while.
>
> Yes. I have been stymied by heavy abuse of the server. In addition to 
> legitimate researchers or occasional users,
> I see people running a speed test every five minutes, 24x7.
>
> I created a bunch of scripts [2] to review the netperf server logs and use 
> iptables to shut off people who abuse the server.
> Even with those scripts running, I have been unable to keep the traffic 
> sent/received below the 4TB/month cap at my VPS.

Does anyone have thoughts about how to continue providing a netperf server at 
the name "netperf.bufferbloat.net" while not overwhelming any particular 
server? Many thanks.

Rich

[1] https://github.com/Zoxc/crusader/issues/14#issuecomment-2028273112
[2] https://github.com/richb-hanover/netperfclean
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] mDNS

[Rich Brown via Bloat]

I'm not advocating that we change the default OpenWrt address from 192.168.1.1 
That's welded too deeply into our synapses (and documentation). But this 
proposal will benefit newcomers for the reasons described below.

> On Feb 28, 2024, at 7:17 AM, David Lang  wrote:
> 
> remember, you don't need to randomly pick something, you just need to have a 
> couple networks, figure out if one is in use by the WAN and if so pick the 
> other.

Remember, too, that this proposal is designed to solve problems for first-time 
users. This lets them avoid reading an entire page of documentation that 
explains how to find their ISP's assigned subnet, and then set their new 
OpenWrt device LAN to use a different subnet.

This won't affect experienced OpenWrt users. On an initial flash/configuration, 
they'll know they can use 192.168.1.0/24 because they know their upstream 
device configuration. Or they can log in using the mDNS name and configure it 
themselves.

> I will say that 192.168.0 and 192.168.1 are very commonly used, so anything 
> other than those a better default
> 
> personally, I like 192.168.255 as people tend to forget that's a valid 
> network.

So... Is there any reason not to incorporate this into the OpenWrt default 
build? Thanks again.

-- Rich___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] mDNS

[Rich Brown via Bloat]

> Avoid the WAN port's DHCP assigned subnet (what if the ISP uses 
> 192.168.1.0/24?)
> 
> ...
> 
> Exactly! There are no rules about what subnet range an ISP's gear will assign 
> to DHCP devices.


@Mark Andrews and @Kenneth Porter... I apologize for any confusion caused by my 
slightly ambiguous language. In both cases, I was referring to the DHCP 
address/subnet assigned to the OpenWrt router's WAN port, by the ISP's gear. 
This will be an RFC1918 address by default.

But my point is that the OpenWrt router has no way to predict what 
address/subnet will be assigned to its WAN port. Consequently, at boot-time, 
OpenWrt should simply choose some different subnet for its LAN subnet(s), and 
then advertise an mDNS name.

Does this make sense? Thanks.

Rich___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] mDNS

[Rich Brown via Bloat]

> On Feb 27, 2024, at 12:00 PM, bloat-requ...@lists.bufferbloat.net wrote:
> 
> On 2/26/2024 6:28 AM, Rich Brown via Bloat wrote:
>> - Avoid the WAN port's DHCP assigned subnet (what if the ISP uses 
>> 192.168.1.0/24?)
> 
> I recently got ATT fiber and its modem won't let me assign from 
> 10.0.0.0/8! So I put a Raspberry Pi 4 in front of it.

Exactly! There are no rules about what subnet range an ISP's gear will assign 
to DHCP devices.

So (I believe) it becomes incumbent on OpenWrt to be smarter than the ISP's 
router (shouldn't be hard) and pick a separate subnet for its LAN & wireless 
interface. (Clearly, OpenWrt could default to 192.168.1.0/24, but if that's 
that range the ISP is using, it could switch to 192.168.2.0/24. I think that's 
all the flexibility that's required...)

And then advertise a mDNS name to make it easy for humans to connect. Who would 
notice?

- Newcomers wouldn't - they'd just connect and configure as described in the 
Wiki
- Grizzled OpenWrt old-timers wouldn't notice either, because they will have 
set their ISP device to use some other address range.

Any reason not to build this into OpenWrt? Thanks.

Rich___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] mDNS (was Disappointment on "Best Newcomer Router" front)

[Rich Brown via Bloat]

Thanks for the observations re: "zero configuration OpenWrt install" 

> On Feb 26, 2024, at 4:47 AM, Sebastian Moeller  wrote:
> 
>> 
>> How about "openwrt.local" to conform to mDNS (RFC 6762) and IANA's
>> [special-use domain names][1]?  Or perhaps "openwrt.internal" in the
>> future?
> 
> Or perhaps openwrt.home.arpa...? I think openwrt.local shouldd not be used, 
> because .local is reserved for mDNS, no?
> 
> OpenWrt defaults to .lan, and .lan is one of the top ten 'leaking' TLDs 
> already, we should simply< agree never to assign the 10 (or 15 or whatever) 
> highest leaking fake TLD as true TLDs, and problem solved, without requiring 
> any more bike shedding from IANA, ICAN, ARPA, ...
> I am sure that, as pragmatic as it might be, is not going to happen... 
> Perhaps because someone intends to convert one/more of the leaking fake TLDs 
> into real TLDs?

Sure. Any of these make sense - use the best practice. The important points of 
the proposal (original link at 
https://forum.openwrt.org/t/cerowrt-ii-would-anyone-care/110554/106 
) are:

- Avoid the WAN port's DHCP assigned subnet (what if the ISP uses 
192.168.1.0/24?)
- Don't require ANY IP address for default configuration
- Instead, use a single well-known mDNS address for initial configuration by 
newcomers

(Why do I care? I tried to write a one-page How to Install OpenWrt guide, and 
wound spending an entire new page describing how to set the router's LAN 
address so that it doesn't conflict with the ISP's assigned range... Sigh.) 

Thanks.

Rich___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Disappointment on "Best Newcomer Router" front

[Rich Brown via Bloat]

> the problem is the 'easy to install' requirement. Too many vendors (including 
> ones that used to be easy to install) are adding extra hoops to jump through 
> on the newer models.

Yes. My fantasy (I can still have fantasies, right? :-) is that people buy one 
of our recommended routers for a modest price, download our recommended 
firmware, go to the vendor's "update firmware" page, upload it and reboot.

Then they plug the new router's WAN port into their ISP modem's port, connect 
their laptop's Ethernet to the LAN port, point their browser to "openwrt.lan" 
(see https://forum.openwrt.org/t/cerowrt-ii-would-anyone-care/110554/106) and 
set the router password, configure SQM, [configure Wi-Fi], and go back to 
fishing, reading, or other productive work.

Simple, eh? (I wish...)

Rich___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Disappointment on "Best Newcomer Router" front

[Rich Brown via Bloat]

> On Feb 25, 2024, at 5:28 PM, Sebastian Moeller  wrote:
> 
> the coming OpenWrt One, which I am quite excited about, way mote excited that 
> I should).

Me too.

> Is that what you asked for, hell no. But it might do as a replacement in a 
> pinch...

I suspect that's what I'll have to wait for...

Thanks___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] Disappointment on "Best Newcomer Router" front

[Rich Brown via Bloat]

For the last several years, I have responded on various boards to complaints 
about bad latency with a happy-go-lucky "Or just try OpenWrt!". [recent 
example: 
https://www.reddit.com/r/HomeNetworking/comments/1ay509v/comment/krx120h/?context=3]
 I used to mention IQrouter, but that's no longer a possibility.

Not wanting to give hollow advice, I decided to gather information about router 
models that support OpenWrt without much risk to a newcomer's: 

- home/family network
- free time
- wallet

My hope was to collect a few recommendations for units that might permit a 
someone who was intrigued by our "Just try OpenWrt!" enthusiasm to spend a few 
bucks and a few hours and see if it made things better. So I threw this note 
over the wall to the OpenWrt forum: 
https://forum.openwrt.org/t/best-newcomer-router-2024/189050 

Along with two options (only available on eBay), I got back a whole lot of 
unsatisfactory responses. (See 
https://forum.openwrt.org/t/best-newcomer-router-2024/189050/21)

But then thoughtful messages started arriving, discussing the state of the 
world for modern OpenWrt routers, and why the firmware really isn't as easy to 
install as one might wish, how capable routers aren't as inexpensive as one 
might hope, etc.

Am I right to despair? Are there no routers out there that we can recommend in 
good faith that are easy to install, powerful enough (for SQM) at ordinary 
speeds, and won't break the bank?

Many thanks.

Rich


___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] Best approach for debloating Airbnb host?

[Rich Brown via Bloat]

I stayed in an Airbnb rental last week. It was nicely appointed with a very 
gracious host who lived in the other half of the home. They had decent internet 
from xfinity - I was getting 20mbps/5mpbs.

But.. they have bad bufferbloat. I was on a Zoom call and occasionally people 
would sound like Darth Vader. I busted out a ping test to 8.8.8.8 and sure 
enough, latency spiked from a nominal 10-20 msec to 2500 msec and occasionally 
over 4000 msec.

I had to check out before I had a chance to mention it to the Airbnb host. And 
I'll probably leave it alone. But I'm still wondering - if I wanted to 
evangelize:

1. What would I say? I know I don't want to blurt out, "your network has 
bufferbloat". That sounds worse than the cooties :-) I imagine that I'd mumble 
something about the Zoom call occasionally sounding like Darth Vader, and that 
I'm a network professional and recognize the symptom, and that there's a 
technical fix for it. I'd probably pause to see if their eyes lit up ("Oh, that 
happens to us all the time...") before proceeding. And then...

2. What would I recommend? Obviously, inserting something with cake into the 
mix would help a lot. Even if they were willing to let me examine their entire 
network (Comcast router, Apple Airport in our Airbnb unit, other router?) I 
have no idea what kind of tar baby I would be touching. I don't want to become 
their network admin for the rest of time.

I know Dave Täht recommends that you help your local coffee shop debloat their 
network. But that's a place that you develop a personal relationship and you 
visit often enough to answer questions during a shake-down period. And they'll 
probably "let you in the back" to see what's there.

Anyone have good ideas about handling this? Or should I give it up?Thanks!

Rich


___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] [Rpm] [Starlink] [LibreQoS] net neutrality back in the news

[Rich Brown via Bloat]

Thank you Jonathan for this clear description of the issues and their history. 
I wonder if there's a fourth one - privacy. 

Rosenworcel's talk https://docs.fcc.gov/public/attachments/DOC-397257A1.pdf 
also points out that ISPs might want to monetize our traffic patterns and 
location data. (This is less of an issue in the EU, but the US remains a Wild 
West in this regard.) 

I am hopeful that the FCC will include this in their NPRM (which must be 
available by now but I haven't looked...)

- Rich Brown

> On Sep 29, 2023, at 12:54 AM, Jonathan Morton via Rpm 
>  wrote:
> 
>> On 29 Sep, 2023, at 1:19 am, David Lang via Bloat 
>>  wrote:
>> 
>> Dave T called out earlier that the rise of bittorrent was a large part of 
>> the inital NN discussion here in the US. But a second large portion was a 
>> money grab from ISPs thinking that they could hold up large paid websites 
>> (netflix for example) for additional fees by threatening to make their 
>> service less useful to their users (viewing their users as an asset to be 
>> marketed to the websites rather than customers to be satisfied by providing 
>> them access to the websites)
>> 
>> I don't know if a new round of "it's not fair that Netflix doesn't pay us 
>> for the bandwidth to service them" would fall flat at this point or not.
> 
> I think there were three more-or-less separate concerns which have, over 
> time, fallen under the same umbrella:
> 
> 
> 1:  Capacity-seeking flows tend to interfere with latency-sensitive flows, 
> and the "induced demand" phenomenon means that increases in link rate do not 
> in themselves solve this problem, even though they may be sold as doing so.
> 
> This is directly addressed by properly-sized buffers and/or AQM, and even 
> better by FQ and SQM.  It's a solved problem, so long as the solutions are 
> deployed.  It's not usually necessary, for example, to specifically enhance 
> service for latency-sensitive traffic, if FQ does a sufficiently good job.  
> An increased link rate *does* enhance service quality for both 
> latency-sensitive and capacity-seeking traffic, provided FQ is in use.
> 
> 
> 2:  Swarm traffic tends to drown out conventional traffic, due to congestion 
> control algorithms which try to be more-or-less fair on a per-flow basis, and 
> the substantially larger number of parallel flows used by swarm traffic.  
> This also caused subscribers using swarm traffic to impair the service of 
> subscribers who had nothing to do with it.
> 
> FQ on a per-flow basis (see problem 1) actually amplifies this effect, and I 
> think it was occasionally used as an argument for *not* deploying FQ.  ISPs' 
> initial response was to outright block swarm traffic where they could 
> identify it, which was then softened to merely throttling it heavily, before 
> NN regulations intervened.  Usage quotas also showed up around this time, and 
> were probably related to this problem.
> 
> This has since been addressed by several means.  ISPs may use FQ on a 
> per-subscriber basis to prevent one subscriber's heavy traffic from degrading 
> service for another.  Swarm applications nowadays tend to employ altruistic 
> congestion control which deliberately compensates for the large number of 
> flows, and/or mark them with one or more of the Least Effort class DSCPs.  
> Hence, swarm applications are no longer as damaging to service quality as 
> they used to be.  Usage quotas, however, still remain in use as a profit 
> centre, to the point where an "unlimited" service is a rare and precious 
> specimen in many jurisdictions.
> 
> 
> 3:  ISPs merged with media distribution companies, creating a conflict of 
> interest in which the media side of the business wanted the internet side to 
> actively favour "their own" media traffic at the expense of "the 
> competition".  Some ISPs began to actively degrade Netflix traffic, in 
> particular by refusing to provision adequate peering capacity at the nodes 
> through which Netflix traffic predominated, or by zero-rating (for the 
> purpose of usage quotas) traffic from their own media empire while refusing 
> to do the same for Netflix traffic.
> 
> **THIS** was the true core of Net Neutrality.  NN regulations forced ISPs to 
> carry Netflix traffic with reasonable levels of service, even though they 
> didn't want to for purely selfish and greedy commercial reasons.  NN 
> succeeded in curbing an anti-competitive and consumer-hostile practice, which 
> I am perfectly sure would resume just as soon as NN regulations were repealed.
> 
> And this type of practice is just the sort of thing that technologies like 
> L4S are designed to support.  The ISPs behind L4S a

Re: [Bloat] Open Source Needs You to Act - Help Fight Proposed Patent Rules

[Rich Brown via Bloat]

> On May 31, 2023, at 9:08 AM, bloat-requ...@lists.bufferbloat.net wrote:
> 
> just when I thought it was safe to stop worrying about patents

Hi Dave,

The letter you forwarded has really long URLs that made it hard to see the 
essentials of the argument.

Do you have a link to the heart of the matter? Thanks.

Rich___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] [Starlink] [LibreQoS] Enabling a production model

[Rich Brown via Bloat]

> On Mar 29, 2023, at 1:13 PM, David Lang via Starlink 
>  wrote:
> 
> The problem is that laying cable (or provisioning wifi access to cover the 
> area) is expensive, and if you try to have multiple different companies doing 
> it, they each need a minimum density of users to make it worth their while.

Yes, this stuff is expensive, Here is reasonably current order-of-magnitude 
cost breakdown for a rural NH town nearby:

1) $55,000 per road-mile to design the system, get licenses to install on the 
utility poles, "make ready" (to check that the poles are ready for new 
facilities) and to hang the fiber on the pole. Installing coax would save $5K 
to $8K per mile.

2) $2,000 to $4,000 per premise to install the drop from the utility pole to 
the building, bring the fiber into the building and install the router. 

3) Pole rental (in NH) is about $10/pole/year. Divide miles of road by 200 feet 
between poles to get an estimate of the number of poles.

So density of customers is critical for the business case. That's why there are 
so many monopoly providers - it's costly to overbuild an already served area.

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] [Starlink] Annoyed at 5/1 Mbps...

[Rich Brown via Bloat]

> On Mar 21, 2023, at 8:31 AM, Sebastian Moeller  wrote:
> 
> I have to push back gently on this...
> 
...

> So I am not su sure I would prefer the 5/1 (A)DSL over a PON... 
> 
> That however is orthogonal to me preferring a competent ISP that takes care 
> of keeping latency under load at bay.

OK. I concede. PON (or even a 25/25mbps connection) is way better than DSL. As 
long as I can use a router with SQM :-)

Rich___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Annoyed at 5/1 Mbps...

[Rich Brown via Bloat]

> On Mar 21, 2023, at 1:21 AM, Frantisek Borsik via Rpm 
>  wrote:
> 
> Now, I hope to really piss You off with the following statement  :-P but:
> 
> even sub 5/1 Mbps “broadband” in Africa with bufferbloat fixed on as many 
> hops along the internet journey from a data center to the customers mobile 
> device (or with just LibreQoS middle box in the ISP’s network) is feeling way 
> better than 25Gbps XG-PON. The only time the XG-PON guy could really feel 
> like a king of the world would be during his speedtest.

Nope. Sorry - this doesn't piss me off :-) It's just true. 

- 7mbps/768kbps DSL with an IQrouter works fine for two simultaneous Zoom 
conferences. (Even though no one would think that it's fast.)
- I recommend people on a budget drop their ISP speed so they can afford a 
router that does SQM 
https://forum.openwrt.org/t/so-you-have-500mbps-1gbps-fiber-and-need-a-router-read-this-first/90305/40
 


The people that get annoyed are those who just upgraded to 1Gbps service and 
still are getting fragged in their games.

Rich___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] On metrics

[Rich Brown via Bloat]

> On Mar 19, 2023, at 11:03 PM, bloat-requ...@lists.bufferbloat.net wrote:
> 
>> Consumers really need things like published performance specs so they can
>> assemble their needs like an a la carte menu.  What do you do, what’s
>> important to you, what details support that need, and they need that in a
>> simple way.   Like a little app that says “how many 1080p TVs or 4K TVs,
>> how many gaming consoles, do you take zoom calls or VoIP/phone calls.  Do
>> you send large emails, videos, or pictures.”
> 
> The problem is that these needs really are not that heavy. Among my ISP 
> connections, I have a 8/1 dsl connection, even when I fail over to that, I 
> can 
> run my 4k tv + a couple other HD TVs + email (although it's at the ragged 
> edge, 
> trying to play 4k at 2x speed can hiccup, and zoom calls can stutter when 
> large 
> emails/downloads flow)

I want to second David Lang's comment. I live in a small rural NH town that was 
stuck at DSL prior to a local company raising the money to install fiber to all 
premises. 

Before the fiber came in, I had 7mbps/768kbps service. If I wanted to bond two 
circuits, I could get 15/1mbps. But many neighbors had 3mbps/768kbps - or worse 
- so they were basically unserved. We frequently saw people parked outside our 
public library after hours to get internet. (And yes, a good router improved 
things. I told a lot of people about the IQrouter that turned the unusable 
service into merely slow.)

But there is a huge swath of rural US that is in the same situation, with zero 
or one provider of dreadful service.

What's the value of a "nutrition label"?

a) It's meaningless for those rural customers. They have no choice beyond "take 
it or leave it." 

b) For the lucky ones where alternative providers compete, the proposed label 
does provide a standardized format that lays out purported speeds and the the 
pricing tiers (including overage charges). I don't think I'd ever believe the 
latency numbers.

c) Coming back to metrics: we can't look to a federally-agreed-to Nutrition 
Label to give guidance for which provider offers the right choice for your mix 
of gadgets. The most important advice I can imagine is "get a router that 
manages your latency", and your problems will go away, or at least be *much* 
better.

Rich
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] Review of Delta's in-flight wifi mentions latency!

[Rich Brown via Bloat]

In this ZDnet article, they showed a screenshot of speedtest.net while using 
Delta's in-flight wifi. It showed pretty big latency.
 
https://www.zdnet.com/home-and-office/networking/i-tried-deltas-new-free-inflight-wi-fi-heres-how-fast-it-was/

BUT... the good news is that the reviewer commented on it. 

> With ping rates of 600 to 1,000 milliseconds, however, it's absolutely 
> unusable for videoconferencing with Zoom and the like, 
> and only a glutton for punishment would use it for gaming.

They said other services "worked as well as in the lobby of a trade-show 
hotel". (That's damning with faint praise...)

Nonetheless, this is PROGRESS! Someone in the main stream press acknowledged 
that latency might be a thing. And that it could harm your network access!

Rich
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Dave's wonderful rant (was: grinch...)

[Rich Brown via Bloat]

Thanks Dave for summarizing the current state of speedtests at 
https://blog.cerowrt.org/post/speedtests/. (Perhaps this post  should be linked 
from the Bufferbloat.net home page?)

I really enjoyed Jim Roskind's presentation when I watched it on Youtube: 
https://youtu.be/_uaaCiyJCFA?t=499 I found the part about outliers and why p50 
and p90 are problematic to be delightfully intuitive. (This Youtube link is 
queued up at that point, but start at the beginning to see the full 
presentation.) 

Rich

> Message: 2
> Date: Tue, 10 Jan 2023 21:07:00 -0800
> From: Dave Taht 
> To: "Luis A. Cornejo" 
> Cc: dick...@alum.mit.edu, Cake List ,
>   "MORTON JR., AL" , IETF IPPM WG ,
>   libreqos , Rpm
>   ,  bloat 
> Subject: Re: [Bloat] [Rpm] [Starlink] [LibreQoS] the grinch meets
>   cloudflare'schristmas present
> Message-ID:
>   
> Content-Type: text/plain; charset="UTF-8"
> 
> Dear Luis:
> 
> You hit 17 seconds of delay on your test.
> 
> I got you beat, today, on my LTE connection, I cracked 182 seconds.
> 
> I'd like to thank Verizon for making it possible for me to spew 4000
> words on my kvetches about the current speedtest regimes of speedtest,
> cloudflare, and so on, by making my network connection so lousy today
> that I sat in front of emacs to rant - and y'all for helping tone
> down, a little, this blog entry:
> 
> https://blog.cerowrt.org/post/speedtests/
> 

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] really lovely live plots of bandwidth, queue depth, marks and drops in libreqos

[Rich Brown via Bloat]

> On Jan 6, 2023, at 11:14 PM, bloat-requ...@lists.bufferbloat.net wrote:
> 
> Wearing my theorist hat... and looking at all the ISP plans
> https://payne.taht.net  is now emulating (click 
> on bandwidth test,
> then a plan)

This is really cool!

I see the "Bandwidth Test", but don't see a place to look at a Plan...

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] FCC requires broadband "Nutritional Label"

[Rich Brown via Bloat]

Forwarded from NANOG list:

> The effective date will be determined later, after publication in the 
> Federal Register and OMB review under PRA.
> 
> November 17, 2022?The Federal Communications Commission today
> unveiled new rules that will for the first time require broadband 
> providers to display easy-tounderstand labels to allow consumers to 
> comparison shop for broadband services. The Report and Order approved by 
> the Commission creates rules that require broadband providers to
> display, at the point of sale, labels that show key information consumers 
> want?prices, speeds,fees, data allowances, and other critical 
> information. The labels resemble the well-known nutrition labels that 
> appear on food products.
> 
> https://www.fcc.gov/document/fcc-requires-broadband-providers-display-labels-help-consumers-0

Even though the label only specifies "typical latency", I have a sense that 
this is a good step forward. If your ISP specifies 5 msec as "typical" and 
their crummy router is bloated, can you get a repair if you call them and say 
that it's averaging 150msec? 

At what point does the expense of handling all the tech support calls outweigh 
the expense of actually making (and deploying) good routers?


___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] [Rpm] [Make-wifi-fast] Traffic analogies (was: Wonderful video)

[Rich Brown via Bloat]

> On Oct 19, 2022, at 7:36 PM, Stephen Hemminger via Rpm 
>  wrote:
> 
> Grocery store analogies also breakdown because packets are not "precious"
> it is okay to drop packets. A lot of AQM works by doing "drop early and often"
> instead of "drop late and collapse".

Another problem is that grocery store customers are individual flows in their 
own right - not correlated with each other. Why is my grocery cart any more (or 
less) important than all the others who're waiting?

I continue to cast about for intuitive analogies (and getting skunked each 
time). But I'm going to try again...

Imagine a company with a bunch of employees. (Or a sports venue, or a UPS depot 
- any location where a bunch of vehicles with similar interests all decide to 
travel at once.) At quitting time, everyone leaves the parking lot where a 
traffic cop controls entry onto a two-lane road. 

If there isn't any traffic on that road, the traffic cop keeps people coming 
out of the driveway "at the maximum rate".

If a car approaches on the road, what's the fair strategy for letting that 
single car pass? Wait 'til the parking lot empties? Make them wait 5 minutes? 
Make them wait one minute? It seems clear to me that it's fairest to stop 
traffic right away, let the car pass, then resume the driveway traffic.

This has the advantage of distinguishing between new flows (the single car) and 
bulk flows (treating vehicles in the driveway as a single flow). But it also 
feels like QoS prioritization or a simple two-queue model, neither of which 
lead to the proper intuition. 

Any "traffic" analogy also ignores people's very real (and correct) intuition 
that "cars have mass". They can't stop in an instant and need to maintain space 
between them. This also ignores the recently-stated reality (for routers, at 
least) that "The best queue is no queue at all..."

Is there any hope of tweaking this analogy? :-)

Thanks.

Rich___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] [Rpm] [Make-wifi-fast] [Cake] The most wonderful video ever about bufferbloat

[Rich Brown via Bloat]

> On Oct 11, 2022, at 1:05 PM, Bob McMahon  wrote:
> 
> I agree that bufferbloat awareness is a good thing. The issue I have is the 
> approach - ask consumers to "detect it" and replace a device with a new one, 
> that may or may not, meet all the needs of the users.
> 
> Better is that network engineers "design bloat out" from the beginning 
> starting by properly sizing queues to service jitter, and for WiFi, to also 
> enable aggregation techniques that minimize TXOP consumption.

The Yes, but... part of my answer emphasizes awareness. How are the network 
engineers going to know it's worth the (minor) effort of creating 
properly-sized queues?

There are two fronts to attack:

- Manufacturers - This video is a start on getting their customers to use these 
responsiveness test tools and call the support lines.

- Hardware (especially router) reviewers - It kills me that there is radio 
silence whenever I ask a reviewer if they have ever measured 
latency/responsiveness.  (BTW: Has anyone heard from Ben Moskowitz from 
Consumer Reports? We had a very encouraging phone call about a year ago, and 
they were going to get back to us...)

Rich


> Bob
> 
> On Tue, Oct 11, 2022 at 6:57 AM Rich Brown  <mailto:richb.hano...@gmail.com>> wrote:
> 
> 
>> On Oct 10, 2022, at 8:05 PM, Bob McMahon via Rpm > <mailto:r...@lists.bufferbloat.net>> wrote:
>> 
>> > I think conflating bufferbloat with latency misses the subtle point in that
>> > bufferbloat is a measurement in memory units more than a measurement in
>> > time units.
> 
> Yes, but... I am going to praise this video, even as I encourage all the 
> techies to be sure that they have the units correct.
> 
> I've been yammering about the evils of latency/excess queueing for 10 years 
> on my blog, in forums, etc. I have not achieved anywhere near the notoriety 
> of this video (almost a third of a million views).
> 
> I am delighted that there's an engaging, mass-market Youtube video that makes 
> the case that bufferbloat even exists. 
> 
> Rich
> 
> This electronic communication and the information and any files transmitted 
> with it, or attached to it, are confidential and are intended solely for the 
> use of the individual or entity to whom it is addressed and may contain 
> information that is confidential, legally privileged, protected by privacy 
> laws, or otherwise restricted from disclosure to anyone else. If you are not 
> the intended recipient or the person responsible for delivering the e-mail to 
> the intended recipient, you are hereby notified that any use, copying, 
> distributing, dissemination, forwarding, printing, or copying of this e-mail 
> is strictly prohibited. If you received this e-mail in error, please return 
> the e-mail to the sender, delete it from your computer, and destroy any 
> printed copy of it.

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] [Rpm] [Make-wifi-fast] [Cake] The most wonderful video ever about bufferbloat

[Rich Brown via Bloat]

> On Oct 10, 2022, at 8:05 PM, Bob McMahon via Rpm  
> wrote:
> 
> > I think conflating bufferbloat with latency misses the subtle point in that
> > bufferbloat is a measurement in memory units more than a measurement in
> > time units.

Yes, but... I am going to praise this video, even as I encourage all the 
techies to be sure that they have the units correct.

I've been yammering about the evils of latency/excess queueing for 10 years on 
my blog, in forums, etc. I have not achieved anywhere near the notoriety of 
this video (almost a third of a million views).

I am delighted that there's an engaging, mass-market Youtube video that makes 
the case that bufferbloat even exists. 

Rich___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] [Rpm] Ookla - Introducing a Better Measure of Latency

[Rich Brown]

This is very cool! I'm glad to see them embrace "download ping" and "upload 
ping". 

> On May 12, 2022, at 1:15 PM, Christoph Paasch via Rpm 
>  wrote:
> 
> Ookla's measure of "loaded latency":
> 
> https://www.ookla.com/articles/introducing-loaded-latency 
> 
> 
> This will hopefully be a shift in how operators approach the bufferbloat 
> problem.
> 
> 
> Christoph
> ___
> Rpm mailing list
> r...@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/rpm

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] fq_codel is *almost* 10 years old

[Rich Brown]

My calendar reminds me that fq_codel will be TEN YEARS OLD this coming Sunday, 
14 May 2022! Dave Täht announced a testable version on the CeroWrt list a 
decade ago at: 
https://lists.bufferbloat.net/pipermail/cerowrt-devel/2012-May/000233.html

Let's crowd-source an updated list of our accomplishments. Here is a link to 
the latest anniversary announcement (five year) I could find:
https://www.mail-archive.com/bloat@lists.bufferbloat.net/msg04146.html I'lll 
plan to send out the compilation on Sunday.

Congratulations, everyone!

Rich
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] broadband cost analysis

[Rich Brown]

> On Apr 15, 2022, at 12:00 PM, bloat-requ...@lists.bufferbloat.net wrote:
> 
>> 
>>   $2K-$4K per premise for the drop cable from the pole and the router in the 
>> premise
> 
> What is the CPE router? Does it have AQM or some other latency control 
> mechanism?

I should have included the details in my original note:

The cost per premise includes the ONT - they're using Calix Gigacenter - plus 
all the materials and labor to run the drop from the pole to the home, plus all 
the interior work, plus any time spent making sure the customer is on the air. 

NB: The $40K/mile includes the cost of system design, obtaining licenses to use 
the utility poles, other regulatory hoo-hah, make-ready - surveying the exact 
location of all the poles, inspecting them for suitability (not rotten, tall 
enough to carry another service, etc), replacing any sub-standard poles, then 
the cost of actually hanging the steel strand and the fiber on the poles.

And no, the CPE doesn't include any AQM that I'm aware of. Latency isn't 
dreadful, but many people still use their IQrouters (installed because we could 
only get 7/1 mbps DSL prior to the arrival of the fiber connectivity.)___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] [Rpm] broadband cost analysis

[Rich Brown]

One item to consider re: costs. My rural NH town just installed fiber to run 
past all premises. A quick estimate of total capital cost uses two numbers:

- $40K/mile to hang the fiber on existing utility poles on the road
- $2K-$4K per premise for the drop cable from the pole and the router in the 
premise


> On Apr 14, 2022, at 11:24 AM, Dave Taht via Rpm  
> wrote:
> 
> Looking at figure 7 (non-adoption rates by age group), nearly 30% of
> those under 30 do not have fixed broadband.
> From an informal survey of those I know in that age range, they are
> primarily dependent on their cell phones,
> cannot live at a fixed address for long enough to adopt fixed
> broadband solutions, and go to coffee shops and
> libraries (and the office) to get their connectivity. I am kind of
> curious as to the trendline here - a cellphone is a must
> for this generation, quality fixed internet merely a nice to have.
> 
> On Thu, Apr 14, 2022 at 8:18 AM Dave Taht  wrote:
>> 
>> pretty good:
>> 
>> https://static1.squarespace.com/static/5f5282b71117310d16e654d3/t/6256eb4efbb468024f396969/1649863506445/Toward+Effective+Administration+of+State+and+Local+Fixed+Broadband+Programs+-+04.12.22+Final+Report.pdf
>> 
>> My lowball cost estimate for "better, recycled routers" would be
>> somewhere in the 20 dollar range for the 25/3mbit segment, which
>> depending on how you do the math per above is somewhere between 10 and
>> 65 million people.
>> 
>> It would be cool to have good bufferbloat statistics for the 25/3mbit
>> portion of the population.
>> 
>> --
>> I tried to build a better future, a few times:
>> https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
>> 
>> Dave Täht CEO, TekLibre, LLC
> 
> 
> 
> -- 
> I tried to build a better future, a few times:
> https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
> 
> Dave Täht CEO, TekLibre, LLC
> ___
> Rpm mailing list
> r...@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/rpm

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] [Starlink] [Make-wifi-fast] Talk now up: How the internet really works

[Rich Brown]

I loved this video!

> On Mar 11, 2022, at 10:51 AM, Dave Taht  wrote:
> 
> On Thu, Mar 10, 2022 at 8:20 PM Vint Cerf  wrote:
>> 
>> just watching the first few minutes I could tell this was going to be a 
>> really fun talk - will watch the rest soon.
> 
> 
> I cavort on the shoulders of giants!

Excellent! You do it so admirably!

> I'm going to give myself an "A" for concept, but a "B" for
> performance.

> The latter half could have gone a bit faster, and one of
> the more difficult tricks,...

Nope. "A" for concept, and "A" for performance.

I teach juggling (semi-professionally - I'm TheJugglerMan.com) and this was a 
first-rate show. All the more impressive since the three jugglers probably only 
had a short time to internalize your script and rehearse the tricks.

I especially liked how the initial club exchanges made them look like 
inexperienced jugglers - using two hands to throw and catch. Followed by the 
real surprise when suddenly there were all those clubs flying, passed by the 
three people :-)

Thank you for your continuing effort to popularize explanations of how the 
internet *really* works.

Rich


___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] Zero-Drop network stacks...

[Rich Brown]

Google Alerts led me to Reddit (at 
https://www.reddit.com/r/VFIO/comments/s5joe6/comment/hsy7jmx/?utm_source=reddit_medium=web2x=3)
 which led me to this article about

Zero-Drop Network Stacks: https://saeed.github.io/files/zd_conext19.pdf

Is there any "there" there?
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] tp-link request for SQM

[Rich Brown]

> On Dec 3, 2021, at 7:00 AM, Dave Täht wrote:
> 
> tp-link, is, so far as I know, the last major home router vendor NOT
> shipping a SQM system. Perhaps this could be modded up with someones
> with accounts?
> 
> https://community.tp-link.com/us/home/forum/topic/511156

Wait... what? Other vendors are providing SQM now? Cool!

How should we change the text on 
https://www.bufferbloat.net/projects/bloat/wiki/What_can_I_do_about_Bufferbloat/
 that says:

> • Install an off-the-shelf router with SQM: Several commercial router vendors 
> have a clue. Here is a list of those we have found:
> 
>   • IQrouter provides a good setup wizard for configuring SQM, and 
> automatically tunes its settings. IQrouter v3 is good to about 350 mbps. 
> (Version 2 was good for 200-250 mbps.)
>   • Ubiquiti gear has fq_codel settings. People say its EdgeRouter will 
> handle over 400 mbps.
>   • The eero mesh routers list SQM as a feature. Their third generation 
> devices support SQM at speeds up to a gigabit/second as of November 2021.
>   • Many other mesh router vendors claim to solve bufferbloat. Check 
> their spec’s or ask them about latency.
>   • Untangle NG Firewall has fq_codel settings.
>   • ipfire.org has fq_codel settings.
>   • If you’re a Comcast/Xfinity customer, see if you can get the XB6 / 
> CGM4140COM cable modem that has PIE enabled. 
>   Read p13 of Improving Latency with Active Queue Management 
> (AQM) During COVID-19. for details.

Thanks.
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] Has anyone looked at FarPlay.io?

[Rich Brown]

https://farplay.io/about


___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] Slowly ... slowly ... the word is getting out...

[Rich Brown]

Yet another anecdote: Google Alerts for "bufferbloat" showed me this: 
https://www.reddit.com/r/Xiaomi/comments/pb7jpc/bufferbloat_latency_while_streaming_on_mi_aiot/

> > ... Currently, I am using the Mi R3 (very old) and while it is better than 
> > my ISP's one that doesn't even have QoS, it's not amazing like fq_codel or 
> > cake...

> ... Short answer: no, not even close to fq-codel, and qos is mostly garbage...

Five (or maybe even two) years ago, the comments would have been virulently 
pro-QoS, impugning your manhood for not getting it set up properly for whatever 
combination of games, apps, etc. you were running.

Now people on Reddit are actively talking about SQM as the right solution to 
the problem. Maybe the tide is turning.

Rich

PS That doesn't mean that we should stop blowing our own horn on this, trying 
to spread the word more broadly...
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Apple WWDC Talks on Latency/Bufferbloat

[Rich Brown]

> On Jun 12, 2021, at 12:00 PM, bloat-requ...@lists.bufferbloat.net wrote:
> 
> Some relevant talks / publicity at WWDC -- the first mentioning CoDel,
> queueing, etc. Featuring Stuart Cheshire. iOS 15 adds a developer test for
> loaded latency, reported in "RPM" or round-trips per minute.
> 
> I ran it on my machine:
> nowens@mac1015 ~ % /usr/bin/networkQuality
>  SUMMARY 
> Upload capacity: 90.867 Mbps
> Download capacity: 93.616 Mbps
> Upload flows: 16
> Download flows: 20
> Responsiveness: Medium (840 RPM)

Does anyone know how to get the command-line version for current (not upcoming) 
macOS? Thanks.

Rich
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] Fwd: Traffic shaping at 10~300mbps at a 10Gbps link

[Rich Brown]

Saw this on the lartc mailing list... For my own information, does anyone have 
thoughts, esp. for this quote:

"... when the speed comes to about 4.5Gbps download (upload is about 500mbps), 
chaos kicks in. CPU load goes sky high (all 24x2.4Ghz physical cores above 90% 
- 48x2.4Ghz if count that virtualization is on)..."

Thanks.

Rich


> Begin forwarded message:
> 
> From: "Ethy H. Brito" 
> Subject: Traffic shaping at 10~300mbps at a 10Gbps link
> Date: June 7, 2021 at 12:38:53 PM EDT
> To: lartc 
> 
> 
> Hi
> 
> I am having a hard time trying to shape 3000 users at ceil speeds from 10 to 
> 300mbps in a 7/7Gbps link using HTB+SFQ+TC(filter by IP hashkey mask) for a 
> few days now tweaking HTB and SFQ parameters with no luck so far.
> 
> Everything seems right, up 4Gbps overall download speed with shaping on.
> I have no significant packets delay, no dropped packets and no high CPU 
> average loads (not more than 20% - htop info)
> 
> But when the speed comes to about 4.5Gbps download (upload is about 500mbps), 
> chaos kicks in.
> CPU load goes sky high (all 24x2.4Ghz physical cores above 90% - 48x2.4Ghz if 
> count that virtualization is on) and as a consequence packets are dropped (as 
> reported by tc -s class sh ...), RTT goes above 200ms and a lots of ungry 
> users. This goes from about 7PM to 11 PM every day.
> 
> If I turn shaping off, everything return to normality immediately and peaks 
> of not more than 5Gbps (1 second average) are observed and a CPU load of 
> about 5%. So I infer the uplink is not crowded.
> 
> I use one root HTB qdisc and one root (1:) HTB class.
> Then about 20~30 same level (1:xx) inner classes to (sort of) separate the 
> users by region 
> And under these inner classes, goes the almost 3000 leaves (1:). 
> I have one class with about 900 users and this quantity decreases by the 
> other inner classes having some of them with just one user.
> 
> Is the way I'm using HTB+SFQ+TC suitable for this job?
> 
> Since the script that creates the shaping environment is too long I do not 
> post it here.
> 
> What can I inform you guys to help me solve this?
> Fragments of code, stats, some measurements? What?
> 
> Thanks.
> 
> Regards
> 
> Ethy

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Questions for Bufferbloat Wikipedia article - question #2

[Rich Brown]

Thanks, all, for the responses re: Bufferbloat definition. I can work with that 
information.

Next question...

> 2) All network equipment can be bloated. I have seen (but not really 
> followed) controversy regarding the amount of buffering needed in the Data 
> Center. Is it worth having the Wikipedia article distinguish between Data 
> Center equipment and CPE/home/last mile equipment? Similarly, is the "bloat 
> condition" and its mitigation qualitatively different between those 
> applications? Finally, do any of us know how frequently data centers/backbone 
> ISPs experience buffer-induced latencies? What's the magnitude of the impact?

Many thanks!

Rich

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] Questions for Bufferbloat Wikipedia article

[Rich Brown]

Dave Täht has put me up to revising the current Bufferbloat article on 
Wikipedia (https://en.wikipedia.org/wiki/Bufferbloat)

Before I get into it, I want to ask real experts for some guidance... Here goes:

1) What is *our* definition of Bufferbloat? (We invented the term, so I think 
we get to define it.) 

a) Are we content with the definition from the bufferbloat.net site, 
"Bufferbloat is the undesirable latency that comes from a router or other 
network equipment buffering too much data." (This suggests bufferbloat is 
latency, and could be measured in seconds/msec.)

b) Or should we use something like Jim Gettys' definition from the Dark Buffers 
article (https://ieeexplore.ieee.org/document/5755608), "Bufferbloat is the 
existence of excessively large (bloated) buffers in systems, particularly 
network communication systems." (This suggests bufferbloat is an unfortunate 
state of nature, measured in units of "unhappiness" :-) 

c) Or some other definition?

2) All network equipment can be bloated. I have seen (but not really followed) 
controversy regarding the amount of buffering needed in the Data Center. Is it 
worth having the Wikipedia article distinguish between Data Center equipment 
and CPE/home/last mile equipment? Similarly, is the "bloat condition" and its 
mitigation qualitatively different between those applications? Finally, do any 
of us know how frequently data centers/backbone ISPs experience buffer-induced 
latencies? What's the magnitude of the impact?

3) The Wikipedia article mentions guidance that network gear should accommodate 
buffering 250 msec of traffic(!) Is this a real "rule of thumb" or just an 
often-repeated but unscientific suggestion? Can someone give pointers to best 
practices?

4) Meta question: Can anyone offer any advice on making a wholesale change to a 
Wikipedia article? Before I offer a fork-lift replacement I would a) solicit 
advice on the new text from this list, and b) try to make contact with some of 
the reviewers and editors who've been maintaining the page to establish some 
bona fides and rapport...

Many thanks!

Rich
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Best Bufferbloat Analogy - Ever (was: FCC Input)

[Rich Brown]

> On Mar 27, 2021, at 4:54 AM, bloat-requ...@lists.bufferbloat.net wrote:
> 
> Jitter and bloat aren't intuitive and need a lot more thinking and patience 
> to understand. Perhaps we need a good animated cartoon to explain it.

I recently riffed off the Waveform bufferbloat test 
(https://www.waveform.com/tools/bufferbloat#question-0) that includes, "Can you 
explain bufferbloat like I'm five?"

My blog posting (complete with a SmartSink(tm)) is at: 
https://randomneuronsfiring.com/best-bufferbloat-analogy-ever/

Maybe that's a start toward an animation...

Rich___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] Juliusz & Dave talks - were they recorded?

[Rich Brown]

I missed these talks - were either of them recorded? Thanks.

-- Forwarded message -
From: Juliusz Chroboczek 
Date: Mon, Feb 22, 2021 at 5:32 AM
Subject: [Galene] Talk about Galène today (monday) at 17pm CET
To: 


I'll be giving a talk about Galène this afternoon:

 https://ffwd.flashgrants.org/calendar.html#event-15/

=== and === 

-- Forwarded message -
Subject: [Bloat] Fwd: [Galene] Dave on bufferbloat and jitter at 8pm
CET Tuesday 23
Message-ID:

Content-Type: text/plain; charset="UTF-8"

It's been a while since I did a talk, this is the shuttleworth flash
grants conference... feel free to hang out with me at:

https://crocus.irif.fr:8443/group/bufferbloat
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] FCC Filing - October 2015

[Rich Brown]

> On Feb 8, 2021, at 12:00 PM, cerowrt-devel-requ...@lists.bufferbloat.net 
> wrote:
> 
> Ironically enough I can't find the original link to the huge FCC
> filing we'd done5+ years bafck, just the ancillary documents. Anybody
> know where it is?

I found this from 9 Oct 2015: 

https://docs.google.com/document/d/1E1D1vWP9uA97Yj5UuBPZXuQEPHARp-AhRqUOeQB2WPk/edit

that links so something called "final, frozen draft" at:

https://docs.google.com/document/d/15QhugvMlIOjH7iCxFdqJFhhwT6_nmYT2j8xAscCImX0/edit

as well as these signatures:

https://docs.google.com/spreadsheets/d/1Xd-Zzv4QV3ptfDBaloL6H0iu0DAAW68TCh3ohnNFv-s/edit#gid=1584333954

Rich___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] Robert X. Cringley: Wi-Fi 6 & Bufferbloat

[Rich Brown]

Once again, Cringely waves the bufferbloat flag...

https://betanews.com/2021/02/05/wi-fi-6-is-a-bust-bufferbloat/


___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] OpenWrt Stability?

[Rich Brown]

> On Jan 5, 2021, at 12:00 PM, Stephen Hemminger wrote:
> 
> I having lots of issues with openwrt stability.

I know the OpenWrt dev's are going hammer and tongs to produce a 2x.0x release 
(which will be out soon-ish...)

What release and what hardware are you using? (There may be fixes in the newest 
snapshots, and if not, you may be able to provide valuable debugging info.)

Rich___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] BBR talk from Stony Brook University

[Rich Brown]

Google Alerts for "bufferbloat" pointed to this talk at Stony Brook University 
in New York State:

Dates:  Thursday, December 17, 2020 - 12:30pm to 2:00pm (US EST: UTC-5)
Location:  Zoom - contact eve...@cs.stonybrook.edu for Zoom info.

Abstract: BBR is a new congestion control algorithm and is seeing increased 
adoption especially for video traffic. BBR solves the bufferbloat problem in 
legacy loss-based congestion control algorithms where application performance 
drops considerably when router buffers are deep. BBR regulates traffic such 
that router queues don’t build up to avoid the bufferbloat problem while still 
maintaining high throughput. Though BBR is able to combat bufferbloat for 
sustained steady traffic such as large file downloads, our analysis shows that 
video applications experience significantly poor performance when using BBR 
under deep buffers. In fact, we find that video traffic sees inflated latencies 
because of long queues at the router, ultimately degrading video performance.

In this talk, I will present our work on the interaction between BBR and 
streaming video. We investigate the relationship between network metrics such 
as delay and delivery rate during the video run and the quality of subsequent 
video segments. However, we find only weak correlations, suggesting that 
another factor is at play. Our empirical investigation reveals that BBR under 
deep buffers and high network burstiness severely overestimates available 
bandwidth and is slower to converge to steady state, both of which result in 
BBR sending substantially more data into the network, causing a queue buildup. 
This elevated packet sending rate under BBR is ultimately caused by the 
router’s ability to absorb bursts in traffic, which destabilizes BBR’s 
bandwidth estimation and overrides BBR’s expected logic for exiting the startup 
phase.

Original Notice: 
https://www.cs.stonybrook.edu/Rebecca-Drucker-Research-Proficiency-Presentation-Investigating-BBR-Bufferbloat-Problem-DASH-Video


___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Good Wi-Fi test programs?

[Rich Brown]

Thanks for this response. 

> On Dec 6, 2020, at 8:00 PM, Jonathan Morton  wrote:
> 
>> On 7 Dec, 2020, at 1:00 am, Rich Brown  wrote:
>> 
>> I would first do the following "easy tests":
>> 
>> - Check for conflicting/overlapping Wi-Fi channels. I am fond of the free 
>> app, WiFi Analyzer from farproc (http://a.farproc.com/wifi-analyzer) for 
>> this test, but there are several similar Android apps. 
>> - Compare the signal strength for the DSL modem and the Calix modem, as 
>> shown by WiFi Analyzer 
>> - Be sure that all computer(s) are using the Calix modem.
>> - Use a variety of speed tests: DSLReports, Fast.com, other favorites?
>> - Compare speedtest results when the test computer is close to, or far from 
>> the router.
>> - (If possible) compare the performance for both Wi-Fi and Ethernet
>> - Shut off the DSL modem on my way out the door to be sure it's not causing 
>> interference or confusing the situation.
>> 
>> Anything else you'd recommend?
> 
> Make sure the customer's devices are using 5GHz rather than 2.4GHz band, 
> where possible.  The Calix devices apparently support both and try to perform 
> "band steering", but it's worth double checking.
> 
> https://www.calix.com/content/calix/en/site-prod/library-html/systems-products/prem/op/p-gw-op/eth-gw/800e-gc-spg/index.htm?toc.htm?76518.htm

Good point - although 2.4GHz may "go farther" in the absence of interference 
from neighbors. (See below)

> I also read while briefly scanning the accessible documentation that Calix 
> operates at maximum permitted wifi transmit power and with up to 80MHz RF 
> bandwidth.  While this does maximise the range and throughput of an 
> individual AP, many such APs in close proximity will see the RF channel as 
> "occupied" by each others' transmissions more often than if a lower transmit 
> power were used.  The result is that they all shout so much that they can't 
> hear themselves think, and clients can't get a word in edgewise to send acks 
> (with generally lower transmit power themselves).


> You should look for evidence of this while analysing channel occupancy, 
> especially in multi-occupancy buildings.  It's probably less of a concern in 
> detached or semi-detached housing.

Our community is extremely rural - most homes are at least 100m apart. (But 
that's why I plan to use the WiFi Analyzer app on my android to look for 
interfering channels. (One of the items high on my list of differential 
diagnosis is that the old DSL modem may still be turned on and using the same 
channel...)

> I didn't see any mention of Airtime Fairness technology, which is now a 
> highlighted feature on some other manufacturers' products (specifically 
> TP-Link).  Ask whether that is present or can be implemented.  You may be 
> able to test for it, if you have established a case where wifi is clearly the 
> bottleneck, by passing a saturating ECN Capable flow through it and looking 
> for CE marks (and/or ECE feedback), since Airtime Fairness comes with 
> built-in fq_codel.

I have fantasies of our little fiber company telling Calix to "get with the 
program" and adopt SQM/ATF. I'll keep dreaming.

Thanks.

Rich
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] Good Wi-Fi test programs?

[Rich Brown]

A local company is rolling out fiber-to-the-home. Subscribers are switching 
from crummy Comtrend or ZyXEL DSL modems (whose PPPoE interface runs at 3 to 
20mbps) to Calix GigaCenter equipment with symmetric ISP link speeds starting 
at 25mbps on up.

The complaints I've started to hear are that the new Wi-Fi signal "is weak" - 
not reaching as far as the DSL equipment did, and that speeds are abysmal (one 
report is 2.4mbps down, 0.25mbps upload). 

Before I insert myself into the "unhappy customer" loop, I would like to ask 
this august group for thoughts about things to check/test programs to 
run/phenomena to consider.

I would first do the following "easy tests":

- Check for conflicting/overlapping Wi-Fi channels. I am fond of the free app, 
WiFi Analyzer from farproc (http://a.farproc.com/wifi-analyzer) for this test, 
but there are several similar Android apps. 
- Compare the signal strength for the DSL modem and the Calix modem, as shown 
by WiFi Analyzer 
- Be sure that all computer(s) are using the Calix modem.
- Use a variety of speed tests: DSLReports, Fast.com, other favorites?
- Compare speedtest results when the test computer is close to, or far from the 
router.
- (If possible) compare the performance for both Wi-Fi and Ethernet
- Shut off the DSL modem on my way out the door to be sure it's not causing 
interference or confusing the situation.

Anything else you'd recommend?

Second, are there other more fine-grained/analytic tests that I could run? I'll 
have a Linux or macOS laptop that I could use to look at other measures: 
retransmits, CRC errors on the Wi-Fi link, etc. And, finally, what do I 
recommend based on the information gained from those tools?

Many thanks.

Rich

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] netperf server news

[Rich Brown]

> 
> 
> --
> 
> Message: 1
> Date: Tue, 06 Oct 2020 19:39:54 -0700
> From: Kenneth Porter 
> To: bloat 
> Subject: Re: [Bloat] netperf server news
> Message-ID: <38F0B196CFEA470FEEBE0520@[172.27.17.193]>
> Content-Type: text/plain; charset=us-ascii; format=flowed
> 
> --On Tuesday, October 06, 2020 7:52 AM -0400 Rich Brown 
>  wrote:
> 
>> 3) I would be pleased to get comments on the set of scripts. I'm a newbie
>> at iptables, so it wouldn't hurt to have someone else check the rules I
>> devised. See the README at https://github.com/richb-hanover/netperfclean
> 
> A couple of alternatives to custom scripts are fail2ban and the 
> rate-limiting modules available for iptables such as hashlimit and recent. 
> I haven't used fail2ban for rate-limiting so I'm not sure if it's the right 
> tool for that but it monitors log files to add iptables rules for 
> short-term banning. It's not hard to add your own log monitoring rule. I 
> haven't used the iptables modules but they look like a natural solution for 
> this.
> 
> <https://poorlydocumented.com/2017/08/understanding-iptables-hashlimit-module/>
> 
> <https://serverfault.com/questions/682045/source-ip-rate-limiting-in-iptables-hashlimit-vs-recent>
> 
> Instead of using a unique iptables rule for each blocklist member, I 
> suggest using an ipset. (I use firewalld as a front-end to iptables so I 
> let it manage my ipsets, but you can also install ipset's service for use 
> with raw iptables to save and restore the sets across boots.) Your block 
> rule could be as simple as this:
> 
> iptables -I INPUT 1 -p tcp --dport netperf -m set --match-set 
> NetPerfAbusers src -m conntrack --ctstate NEW -j DROP

Thanks for these thoughts. I looked briefly at rate-limiting schemes, but 
didn't see a good way for them to distinguish good users from bad:

- Good users (who are setting up their SQM, or testing various algorithms) run 
a test (that creates 10 connections in ~10-60 seconds), tweak a parameter, then 
re-run that test, repeating until they're happy.

- Bad users who test every five minutes 24x7 create 10 connections every 300 
seconds - a slower "rate" of establishing new connections than the good guys.

The primary characteristic that distinguishes the good guys from the bad is 
that good guys *stop.*

So, my reasoning goes, I need to look at a longer time window and set a limit 
on the number of connections over the course of a day or two (not minutes or 
hours). And that's the genesis of my question to the group:

What is *your* pattern of testing? How many successive tests are you 
likely to make over the course of a day? 

I'm also aware of ipset, which I take to be an optimized alternative to 
searching a long set of iptables rules (true?) I don't believe that my OpenVZ 
VPS has kernel support for this, so as long as the long-list-of-rules seems to 
work well, I'm going to leave it alone.

That's my thinking, but please let me know if I'm missing something. Thanks 
again.



___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] netperf server news

[Rich Brown]

Thanks for the feedback. Some responses:

1) I'm glad that people are seeing reasonable speeds from the VPS. (I don't 
know what I can do to make it go faster, so I'm relieved...)

2) I don't think I posed the right question for the number-of-tests threshold. 
(Most of the responses were like, "Sure, that sounds like enough..." Let me 
reframe the question: 

In your normal testing/troubleshooting process, what is the maximum 
number of tests YOU might need to run in any two-day period?

3) If you can't get through to netperf.bufferbloat.net, send me your IP address 
because it might have been blacklisted.

Thanks!

Rich


> On Oct 6, 2020, at 6:52 AM, Rich Brown  wrote:
> 
> To the Bloat list,
> 
> I had some time, so I looked into what it might take to keep the 
> netperf.bufferbloat.net server on-line in the face of an unwitting "DDoS" 
> attack - automated scripts that run tests every 5 minutes 24x7. The problem 
> was that these tests would blow through my 4TB/month bandwidth allocation in 
> a few days.
> 
> In the past, I had been irregularly running a set of scripts to count 
> incoming netperf connections and blacklist (in iptables) those whose counts 
> were too high. This wasn't good enough: it wasn't keeping up with the tidal 
> wave of connections.
> 
> Last week, I revised those scripts to work as a cron job. The current 
> parameters are: run the script every hour; process the last two days' of 
> kern.log files; look for > 500 connections; drop those addresses in iptables.
> 
> There are currently 479 addresses blacklisted in iptables (that explains why 
> the bandwidth was being consumed so quickly). There are only a few new 
> addresses being added per day, so it seems that we have flushed out most of 
> the abusers.
> 
> My questions for this august group:
> 
> 1) The server at netperf.bufferbloat.net is up and running. I get full rate 
> speed from my 7mbps DSL circuit, but that's not much of a test. I would be 
> interested to hear your results.
> 
> 2) The current threshold comes from this estimate: most speed tests use 10 
> connections: 5 connections up and 5 down. So 500 connections would permit 
> about 50 tests over the course of two days. Is that enough for "real 
> research"? (If you need more, I can add your address to my whitelist file...)
> 
> 3) I would be pleased to get comments on the set of scripts. I'm a newbie at 
> iptables, so it wouldn't hurt to have someone else check the rules I devised. 
> See the README at https://github.com/richb-hanover/netperfclean
> 
> Thanks.
> 
> Rich
> 

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] netperf server news

[Rich Brown]

To the Bloat list,

I had some time, so I looked into what it might take to keep the 
netperf.bufferbloat.net server on-line in the face of an unwitting "DDoS" 
attack - automated scripts that run tests every 5 minutes 24x7. The problem was 
that these tests would blow through my 4TB/month bandwidth allocation in a few 
days.

In the past, I had been irregularly running a set of scripts to count incoming 
netperf connections and blacklist (in iptables) those whose counts were too 
high. This wasn't good enough: it wasn't keeping up with the tidal wave of 
connections.

Last week, I revised those scripts to work as a cron job. The current 
parameters are: run the script every hour; process the last two days' of 
kern.log files; look for > 500 connections; drop those addresses in iptables.

There are currently 479 addresses blacklisted in iptables (that explains why 
the bandwidth was being consumed so quickly). There are only a few new 
addresses being added per day, so it seems that we have flushed out most of the 
abusers.

My questions for this august group:

1) The server at netperf.bufferbloat.net is up and running. I get full rate 
speed from my 7mbps DSL circuit, but that's not much of a test. I would be 
interested to hear your results.

2) The current threshold comes from this estimate: most speed tests use 10 
connections: 5 connections up and 5 down. So 500 connections would permit about 
50 tests over the course of two days. Is that enough for "real research"? (If 
you need more, I can add your address to my whitelist file...)

3) I would be pleased to get comments on the set of scripts. I'm a newbie at 
iptables, so it wouldn't hurt to have someone else check the rules I devised. 
See the README at https://github.com/richb-hanover/netperfclean

Thanks.

Rich

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Phoronix: Linux 5.9 to allow FQ_PIE as default

[Rich Brown]

I was asking whether there was anything "there" (that is, interesting) in that 
Phoronix posting / Linux announcement.

> On Jul 15, 2020, at 9:02 AM, Toke Høiland-Jørgensen  wrote:
> 
>> Is there any "there" here?
> 
> I'm sorry, what? :)

Sorry, this is a reference to Gertrude Stein's quote... See Wiktionary - 
https://en.wiktionary.org/wiki/there_is_no_there_there

Rich___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] Phoronix: Linux 5.9 to allow FQ_PIE as default

[Rich Brown]

Is there any "there" here?

https://www.phoronix.com/forums/forum/phoronix/latest-phoronix-articles/1193738-linux-5-9-to-allow-defaulting-to-fq-pie-queuing-discipline-for-fighting-bufferbloat

Thanks.
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] fq_codel is now EIGHT years old...

[Rich Brown]

Yesterday marks the eighth anniversary of fq_codel's inclusion in CeroWrt. See 
Dave's announcement at: 
https://lists.bufferbloat.net/pipermail/cerowrt-devel/2012-May/000233.html

It's kind of fun to read some of the messages to see what occupied our minds 
back in 2012...

Enjoy!

Rich
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] dslreports is no longer free

[Rich Brown]

A request to Sergey,

As a long-time lurker on this list, I lament the passing of DSLReports, and 
applaud Netflix for incorporating latency measurements into its Fast.com page.

However, I have always been vaguely unsettled that "Show more info" displays a 
single latency value for the two separate "processes" - downloading and 
uploading - and the fact that uploading seems to overwrite the download value. 

I imagine that you have already considered this but I have come to realize that 
simply displaying separate values for these latencies would address my unease - 
something along the lines of this photoshopped screen shot.

Thanks again for providing the Fast.com service - it's useful for me every day.

Rich Brown

PS Yes, my DSL download speed really is < 1mbps today. Dratted ISP...

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] Multiple WAN ports & SQM?

[Rich Brown]

Given the crummy internet service in my area (DSL, max of 15mbps/1mbps), I 
wonder if we could improve things by getting a second connection from our ISP 
and "bonding" the two links together in my OpenWrt router.

I see both Multiwan (which is self-described as old) and mwan3. 

But neither would seem to offer the kinds of latency control 
(SQM/fq_codel/cake) that the cool kids in networking have come to expect. 

Any recommendations from this group for such an effort? Thanks.

Rich
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] OT: Netflix vs 6in4 from HE.net

[Rich Brown]

 I love knowing smart people. 

Yes, it does appear to be Netflix geo-fencing their services. Given that I only 
watch Netflix on one computer, I am taking Sebastian's advice and turning off 
IPv6 DNS queries in Firefox. 

Thanks again for these responses.

Rich

> On Mar 21, 2020, at 6:14 PM, Sebastian Moeller  wrote:
> 
> Hi Rich,
> 
> since it seems to be IPv6 related, why not use firefox for netflix and 
> disable IPv6 in firefox (see 
> https://support.mozilla.org/en-US/kb/firefox-cant-load-websites-other-browsers-can#w_ipv6)
>  maybe that works well enough?
> 
> Best Regards
>   Sebastian
> 
> 
> 
> 
>> On Mar 21, 2020, at 21:20, Rich Brown  wrote:
>> 
>> to Bloat & CeroWrt folks: This is a little OT for either of these lists, but 
>> I figured there are plenty of experts here, and I would be delighted to get 
>> your thoughts.
>> 
>> I just tried to view a Netflix movie and got a F7111-5059 error message. 
>> This prevented the video from playing. (As recently as a month or two ago, 
>> it worked fine.)
>> 
>> Googling the error message gets to this page 
>> https://help.netflix.com/en/node/54085 that singles out use of an IPv6 Proxy 
>> Tunnel.
>> 
>> Sure enough, I'm have a 6in4 tunnel through Hurricane Electric on WAN6. 
>> Stopping that WAN6 interface caused Netflix to work.
>> 
>> What advice could you offer? (I could, of course, turn off WAN6 to watch 
>> movies. But that's a drag, and other family members couldn't do this.) Many 
>> thanks.
>> 
>> Rich
>> ___
>> Bloat mailing list
>> Bloat@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat
> 

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] OT: Netflix vs 6in4 from HE.net

[Rich Brown]

to Bloat & CeroWrt folks: This is a little OT for either of these lists, but I 
figured there are plenty of experts here, and I would be delighted to get your 
thoughts.

I just tried to view a Netflix movie and got a F7111-5059 error message. This 
prevented the video from playing. (As recently as a month or two ago, it worked 
fine.)

Googling the error message gets to this page 
https://help.netflix.com/en/node/54085 that singles out use of an IPv6 Proxy 
Tunnel.

Sure enough, I'm have a 6in4 tunnel through Hurricane Electric on WAN6. 
Stopping that WAN6 interface caused Netflix to work.

What advice could you offer? (I could, of course, turn off WAN6 to watch 
movies. But that's a drag, and other family members couldn't do this.) Many 
thanks.

Rich
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] macOS Catalina causes 'hugo serve' to segfault...

[Rich Brown]

> On Feb 22, 2020, at 7:26 AM, Toke Høiland-Jørgensen  wrote:

...

> In case anyone else wants to build the old version of Hugo, we managed
> to resolve this, and with a recent version of Go, the following command
> should now get you a working 'hugo' binary in your $GOPATH/bin:
> 
> go get github.com/tohojo/hugo
> 
> -Toke

And for posterity (and so I can find it by searching the archives of the 
list...), here's how I got it building on my macOS Catalina machine:

* Upgrade to latest Go - https://golang.org/dl/ then double-click the .pkg file

* Ensure that there are enough files: see 
https://medium.com/mindful-technology/too-many-open-files-limit-ulimit-on-mac-os-x-add0f1bfddde

* Use these commands:

export GOPATH=~/github/hugo-from-source
# save source in $GOPATH & build it
go get -v github.com/tohojo/hugo 
cd ~/github/bufferbloat.net
$GOPATH/bin/hugo serve

-Rich
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] macOS Catalina causes 'hugo serve' to segfault...

[Rich Brown]

> On Feb 21, 2020, at 6:20 AM, Toke Høiland-Jørgensen  wrote:
> 
> Looking at that thread, I don't really get the feeling that they're any
> more helpful now :/

I can second that thought. 

> As an alternative, maybe it'll be possible to build a new version of the
> 0.16 Hugo binary with a newer Go runtime? I very much doubt that the
> segfault is because of the Hugo code itself...

My thought exactly. In case anyone has a burning urge to get distracted on 
this, I'll take a whack at this over the weekend. (After all, it's my computer 
that's busted...)

Rich___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] macOS Catalina causes 'hugo serve' to segfault...

[Rich Brown]

I took a deep breath, and upgraded my macOS machine to Catalina. I knew it 
would cause problems, and it did. Most were small, or simply annoying.

One of the flat tires is the ancient version of hugo (0.16) which I had been 
keeping around to rebuild the bufferbloat.net site. It now segfaults reliably 
(instead of only a third of the time when I was running Sierra (10.12)) 

I just posted a message on the Hugo forum (a Discourse forum!) asking for 
guidance. 
https://discourse.gohugo.io/t/convert-a-0-16-site-to-modern-hugo-syntax/23579

I'll summarize if I hear anything back. (As an incentive, I offered that we 
could tell people how to get rid of their lag while gaming :-)

Rich
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] Debugging a crash

[Rich Brown]

Folks,

I was running OpenWrt 19.07-rc2 on my Archer C7v2, and experienced a 
potentially-repeatable crash under heavy network traffic load.

I was uploading several long videos to Youtube (> 5GBytes each) and watching 
Netflix on my 7mbps/768kbps DSL connection. Things were working fine (latency 
remained acceptable - YAY SQM!) My upstream pipe (slow as it is) was totally 
filled by the three competing uploads, and Netflix was doing its best to show 
me the West Wing.

A couple times during the evening, the Netflix stream just crapped out and I 
lost the connection. (Infinite buffering message on-screen, couldn't get to 
Google, etc. The LuCI GUI showed uptime of a minute or so, and then things were 
fine again.)

I now have OpenWrt 19.07.1 installed, and have a little bandwidth for repeating 
the experiment.

What debugging information should I turn on/look for in case this happens 
again? Thanks.

Rich

PS The new Youtube uploader facility is excellent. If you keep the window open, 
it automagically resumes the transfer after interruption. It's great for people 
like me who have large files to upload on a slow link, where the likelihood of 
a link failure (or needing to move my laptop to a different location) is high.
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Can't Run Tests Against netperf.bufferbloat.net

[Rich Brown]

Update: (I thought I had sent the previous message yesterday. My mistake.)

I now have atl3.richb-hanover.com running a netperf server. it's a stock Ubuntu 
18.04.4 LTS - uname -a shows: Linux atl3 4.15.0-76-generic #86-Ubuntu SMP Fri 
Jan 17 17:24:28 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux. I have installed 
netperf 2.6.0, and little else.

Next steps:

1) Please hammer on the server to see if it's a suitable replacement for the 
canonical "netperf.bufferbloat.net". Please feel free to check both its ability 
to handle traffic as well as any security surprises you discover...

2) I welcome suggestions for configuring the server's TCP stack to be most 
useful for researchers. fq_codel, bbr, - I'm open to your thoughts. 

3) It's not too soon for advice on an iptables strategy for limiting the 
access/bandwidth/traffic to people who're abusing the service...

Once we have all this in place, we can change the netperf.bufferbloat.net name 
to point to this server. Thanks.

Rich

> On Feb 8, 2020, at 5:35 PM, Rich Brown  wrote:
> 
> Toke and Jesper,
> 
> Thanks both for these responses. 
> 
> netperf.bufferbloat.net is running an OpenVZ VPS with a 3.10 kernel. Tech 
> support at Ramnode tells me that I need to get to a KVM instance in order to 
> use ipset and other fancy kernel stuff.
> 
> Here's my plan:
> 
> 1) Unless anyone can recommend a better hosting service ...
> 
> 2) Over the weekend, I'll stand up a new KVM server at Ramnode. They offer a 
> 2GB RAM, 2 core, 65 GB SSD, with 3TB per month of data. It'll cost $10/month: 
> adding 2x1TB at $4/month brings it to a total of $18/month, about what the 
> current server costs. I can get Ubuntu 18.04 LTS as a standard install.
> 
> 3) While that's in-flight I would request that an iptables expert on the list 
> recommend a better strategy. (I was just makin' stuff up in the current setup 
> - as you could tell :-)
> 
> 4) I'd also accept any thoughts about tc commands for setting up the 
> networking on the host to work best as a netperf server. (Maybe enable 
> fq_codel or better...) 
> 
> Thanks
> 
> Rich
> 
>> On Feb 7, 2020, at 7:02 AM, Jesper Dangaard Brouer  wrote:
>> 
>> On Thu, 6 Feb 2020 18:47:06 -0500
>> Rich Brown  wrote:
>> 
>>>> On Feb 6, 2020, at 12:00 PM, Matt Taggart wrote:
>>>> 
>>>> This smells like a munin or smokeping plugin (or some other sort of 
>>>> monitoring) gathering data for graphing.  
>>> 
>>> Yup. That is a real possibility. The question is what we do about it.
>>> 
>>> If I understood, we left it at:
>>> 
>>> 1) Toke was going to look into some way to spread the
>>> 'netperf.bufferbloat.net' load across several of our netperf servers.
>>> 
>>> 2) Can someone give me advice about iptables/tc/? to identify IP
>>> addresses that make "too many" connections and either shut them off
>>> or dial their bandwidth back to a 3 or 5 kbps? 
>> 
>> Look at man iptables-extensions and find "connlimit" and "recent".
>> 
>> 
>>> (If you're terminally curious, Line 5 of
>>> https://github.com/richb-hanover/netperfclean/blob/master/addtoblacklist.sh
>>> shows the current iptables command to drop connections from "heavy
>>> users" identified in the findunfilteredips.sh script. You can read
>>> the current iptables rules at:
>>> https://github.com/richb-hanover/netperfclean/blob/master/iptables.txt)
>> 
>> Sorry but this is a wrong approach.  Creating an iptables rule per
>> source IP-address, will (as you also demonstrate) give you a VERY long
>> list of rules (which is evaluated sequentially by the kernel).
>> 
>> This should instead be solved by using an ipset (howto a match from
>> iptables see man iptables-extensions(8) and "set").  And use the
>> cmdline tool ipset to add and remove entries.
>> 
>> -- 
>> Best regards,
>> Jesper Dangaard Brouer
>> MSc.CS, Principal Kernel Engineer at Red Hat
>> LinkedIn: http://www.linkedin.com/in/brouer
>> 
> 

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Can't Run Tests Against netperf.bufferbloat.net

[Rich Brown]

Toke and Jesper,

Thanks both for these responses. 

netperf.bufferbloat.net is running an OpenVZ VPS with a 3.10 kernel. Tech 
support at Ramnode tells me that I need to get to a KVM instance in order to 
use ipset and other fancy kernel stuff.

Here's my plan:

1) Unless anyone can recommend a better hosting service ...

2) Over the weekend, I'll stand up a new KVM server at Ramnode. They offer a 
2GB RAM, 2 core, 65 GB SSD, with 3TB per month of data. It'll cost $10/month: 
adding 2x1TB at $4/month brings it to a total of $18/month, about what the 
current server costs. I can get Ubuntu 18.04 LTS as a standard install.

3) While that's in-flight I would request that an iptables expert on the list 
recommend a better strategy. (I was just makin' stuff up in the current setup - 
as you could tell :-)

4) I'd also accept any thoughts about tc commands for setting up the networking 
on the host to work best as a netperf server. (Maybe enable fq_codel or 
better...) 

Thanks

Rich

> On Feb 7, 2020, at 7:02 AM, Jesper Dangaard Brouer  wrote:
> 
> On Thu, 6 Feb 2020 18:47:06 -0500
> Rich Brown  wrote:
> 
>>> On Feb 6, 2020, at 12:00 PM, Matt Taggart wrote:
>>> 
>>> This smells like a munin or smokeping plugin (or some other sort of 
>>> monitoring) gathering data for graphing.  
>> 
>> Yup. That is a real possibility. The question is what we do about it.
>> 
>> If I understood, we left it at:
>> 
>> 1) Toke was going to look into some way to spread the
>> 'netperf.bufferbloat.net' load across several of our netperf servers.
>> 
>> 2) Can someone give me advice about iptables/tc/? to identify IP
>> addresses that make "too many" connections and either shut them off
>> or dial their bandwidth back to a 3 or 5 kbps? 
> 
> Look at man iptables-extensions and find "connlimit" and "recent".
> 
> 
>> (If you're terminally curious, Line 5 of
>> https://github.com/richb-hanover/netperfclean/blob/master/addtoblacklist.sh
>> shows the current iptables command to drop connections from "heavy
>> users" identified in the findunfilteredips.sh script. You can read
>> the current iptables rules at:
>> https://github.com/richb-hanover/netperfclean/blob/master/iptables.txt)
> 
> Sorry but this is a wrong approach.  Creating an iptables rule per
> source IP-address, will (as you also demonstrate) give you a VERY long
> list of rules (which is evaluated sequentially by the kernel).
> 
> This should instead be solved by using an ipset (howto a match from
> iptables see man iptables-extensions(8) and "set").  And use the
> cmdline tool ipset to add and remove entries.
> 
> -- 
> Best regards,
>  Jesper Dangaard Brouer
>  MSc.CS, Principal Kernel Engineer at Red Hat
>  LinkedIn: http://www.linkedin.com/in/brouer
> 

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Can't Run Tests Against netperf.bufferbloat.net

[Rich Brown]

> On Feb 6, 2020, at 12:00 PM, Matt Taggart wrote:
> 
> This smells like a munin or smokeping plugin (or some other sort of 
> monitoring) gathering data for graphing.

Yup. That is a real possibility. The question is what we do about it.

If I understood, we left it at:

1) Toke was going to look into some way to spread the 'netperf.bufferbloat.net' 
load across several of our netperf servers.

2) Can someone give me advice about iptables/tc/? to identify IP addresses that 
make "too many" connections and either shut them off or dial their bandwidth 
back to a 3 or 5 kbps? 

(If you're terminally curious, Line 5 of 
https://github.com/richb-hanover/netperfclean/blob/master/addtoblacklist.sh 
shows the current iptables command to drop connections from "heavy users" 
identified in the findunfilteredips.sh script. You can read the current 
iptables rules at: 
https://github.com/richb-hanover/netperfclean/blob/master/iptables.txt)

Thanks.

Rich___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Can't Run Tests Against netperf.bufferbloat.net

[Rich Brown]

Hi all,

Thanks for the note. Yes, the netperf server at netperf.bufferbloat.net 
 is turned off. The VPS that runs it is 
consuming its bandwidth limit (4TB per month) at an ever-increasing rate. When 
that happens, my hosting service (Ramnode.com  - good 
guys, stable hosting, great tech support service) automatically turns off the 
VPS 'til the start of the next month.

In the distant past, the 4 TB sufficed for the entire month. More recently, I 
would occasionally get a 90% warning by the 25th or 26th of the month. Last 
month, I hit 90% on Jan 6th(!) so I shut off the netperf server so I could 
continue to work on it.

I briefly turned on netserver on the VPS today. At 08:15 today, the VPS control 
panel showed: 181.7 MB of 3.9 TB Used. At 09:16 today, it showed 46.3 GB. 46 
GBytes in 1 hour => ~30 TB/month (!) 

I'm going to appeal to the group's collective wisdom to find a better solution.

Current mitigations:

- I use iptables to log all netperf connections. I see a pattern of certain IP 
addresses that seem to be firing off a test every five minutes, 24 x 7 for days 
at a time.

- A few times a month, I run a script (see findunfilteredips.sh in 
https://github.com/richb-hanover/netperfclean 
) that scans the log files to 
count netperf connections and to block devices (using iptables) that have made 
more than 5,000 connections in the last seven days. This helps, but only delays 
the inevitable.

Potential (additional) mitigations:

- We could change DNS to spread the load of netperf.bufferbloat.net 
 across our fleet of servers. (Researchers who 
need consistent results could still choose a specific server: netperf-east, 
netperf-west, etc.)

- I could automate the current script to look for heavy users every day or two.

- Maybe I'm doing iptables imperfectly - comments appreciated.

- I have toyed with the notion of tweaking the iptables rules to throttle heavy 
users (over a certain number of tests/connections per time-period). That way, 
the 24x7 people would receive, say 3kbps instead of the actual link speed. 
There are a couple difficulties:
a) I don't want to inconvenience actual researchers/bufferbloat 
testers. When I test a connection, I typically make 3-10 tests in rapid 
succession before I go away. This looks an awful lot like the 24x7 folks, 
except that real testers stop after 15 minutes. Could iptables be tweaked to 
tell one from the other?
b) When I looked into this, I realized I might need to move the VPS 
from OpenVZ (which has limited iptables capabilities - no 'ipset' for example) 
to KVM (which is full virtualization).

- I could just buy more bandwidth. Currently, I pay $194/year for this server 
with the 4TB limit. Additional bandwidth on this provider is $48/year per 
additional TB. But 30 TB/month would be pricey.

- I could move to a different hosting service where bandwidth is cheaper. (Any 
recommendations?)

- Other thoughts? 

Thanks.

Rich


> On Feb 5, 2020, at 3:15 AM, Toke Høiland-Jørgensen  wrote:
> 
> Taran Lynn  writes:
> 
>> All of my attempts to run netperf (and flent) against
>> netperf.bufferbloat.net have failed for the past couple of weeks.
>> Netperf seems to work fine between my desktop and laptop, so I don't
>> think the issue is on my end. Can anyone verify if the server is up?
> 
> Rich, I think this one is yours? Did netserver die or something?
> 
> -Toke

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] bufferbloat.net updated

[Rich Brown]

Folks,

I updated a bunch of pages on bufferbloat.net that I often refer to:

- Main Bloat page - Added a TL;DR with links for measuring and fixing 
bufferbloat above the Detailed Info
https://www.bufferbloat.net/projects/bloat/wiki/

- What Can I Do About Bufferbloat? I simplified the page to actually describe 
what people can do...

https://www.bufferbloat.net/projects/bloat/wiki/What_can_I_do_about_Bufferbloat/

- Getting SQM Right - I improved the description of RRUL charts, and indicated 
that the tuning session is still relevant

https://www.bufferbloat.net/projects/bloat/wiki/Getting_SQM_Running_Right/

- Throughout the site: 
- Note that CeroWrt became OpenWrt
- Note that LEDE became OpenWrt
- Fixed menu structure on the right to list OpenWrt (not LEDE) and 
CeroWrt (link to its archive of articles)

- Assorted minor changes and link fixes.

You're free to make continued edits - I'm content with the site for now. Thanks.

Rich
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] Dave Täht's Bufferbloat talk from linux.conf.au is available

[Rich Brown]

Did you know how to demonstrate TCP Slow Start & Congestion Avoidance, and Fair 
Queueing algorithms with humans? 

Dave does, and he did so at linux.conf.au on 15 Jan 2020.

Watch his talk on the Youtubes at: https://www.youtube.com/watch?v=ZeCIbCzGY6k 
There's a decent (text) summary at: 
http://www.publicnow.com/view/B1DB3833A3B12B3431BF42BE26DE0D8227912926

Nice work, Dave!
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Bufferbloat on 4G Connexion

[Rich Brown]

> On Oct 23, 2019, at 5:54 AM, > wrote:
> 
> If you could influence the 4G vendors to de-bloat their equipment, would you 
> recommend BQL, L4S or codel/cake?

I've been enjoying this discussion and wonder whether the work going on in the 
make-wifi-fast (https://lists.bufferbloat.net/pipermail/make-wifi-fast/) is 
relevant.

I only have a 30,000 foot understanding of this work, but it seems the use of 
AQL (Airtime Queue Limit) maps better onto the vagaries of 4G/5G radio 
transmissions than BQL. Specifically, having a measurement of the actual time 
it takes to transmit a packet might give additional information about the 
current link speed, with the potential for adjusting the codel target, etc.

Separately, I also wonder whether the Air Time Fairness algorithm might provide 
a benefit if the cellphone tower station manufacturers chose to get into the 
game.

Thanks.

Rich___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] coffee-shop-bloat-test.sh

[Rich Brown]

Dave put together a script with a succession of Flent tests to give a coffee 
shop network a real workout.

I have tweaked it a bit to make it easier/more informative to use. 
https://github.com/richb-hanover/coffee-shop-bloat-test

Enjoy!

Rich
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Rigorous Coffee Shop Bloat Testing

[Rich Brown]

> On Sep 3, 2019, at 11:22 AM, bloat-requ...@lists.bufferbloat.net wrote:
> 
> The coffee shop tests were fun, but I(we) needed more rigor when doing
> them. What I'd typically do is go in,
> get on the wifi, start 6 minutes worth of tests, get in line, get
> coffee...

OK. I'll bite. What "six minutes of tests" do you queue up? What do you record?

And how do you broach the subject with the owner? Something like...

"Uh, I think I know why all those heads are popping up..." OR 
"This is a nice network you have here. It'd be a shame if something happened to 
it..." OR
"I know I look like [don't look like] a pointy-headed geek, but there's this 
thing called bufferbloat..." OR 
"Do you ever get complaints that your wifi is really slow?" 

Thanks.

Rich
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Getting bloat tests into open source speedtest

[Rich Brown]

Another bit of good news - guidosarducci over at OpenWrt has packaged up an 
improved version of the "betterspeedtest.sh" script to work with OpenWrt 18.06. 

https://forum.openwrt.org/t/speedtest-new-package-to-measure-network-performance/24647/71
 shows the announcement of the package. Run it with:

speedtest-netperf.sh --help

Enjoy!

Rich___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] will starlink have bufferbloat?

[Rich Brown]

Hi Dave,

Google Alerts today found your Reddit post 
(https://www.reddit.com/r/Starlink/comments/brn6gg/will_starlink_have_bufferbloat/
 
)

It also provided a link to a Slashdot post that seems to have been marked as 
spam... https://slashdot.org/submission/9718636/will-starlink-have-bufferbloat

Was this your original submission? In any case, I upvoted it.

Rich___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] fq_codel is SEVEN years old today...

[Rich Brown]

Folks,

If it feels like we've been at this for a long time, you're right. My calendar 
tells me that we've been using fq_codel for the last seven years.

https://lists.bufferbloat.net/pipermail/bloat/2014-May/001934.html

Let's all pat ourselves on the back for this good work!

Rich
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Flent-farm costs

[Rich Brown]

Dave wrote:

> Costs on the "flent-farm" continue to drop. Our earliest linode
> servers cost $20/month and our two latest ones (nanoservers) cost
> $5/month. For "science!" I've been generally unwilling to
> update/change these much ...

I've been running netperf.bufferbloat.net (the netperf server that we most 
publicize) for several years. It's a modest OpenVZ VPS from RamNode in Atlanta. 
It has two failings:

- It costs ~$16/month (I don't mind this expense, but $16/month >> $5/month for 
the nanoservers)
- About ever third month, its traffic goes over the 4TB/month limit, and 
RamNode shuts the server off. I regularly run a script to find heavy users and 
block their IP using iptables. (Many people are running a test every five 
minutes for days at a time.) But that's a hassle. And buying an additional 
terabyte per month from RamNode is $10/month, which gets expensive.

To address this, I stood up up a new (KVM-based) VPS with RamNode (also in 
Atlanta, presumably in their same data center) that will permit more in-depth 
iptables rules. My goal would be to look at connection frequency, and if 
someone is trying to do every-five-minute testing, limit their bandwidth to 
10kbps. 

This raises a host of questions:

- For Science - the current netperf.bufferbloat.net is atl.richb-hanover.com; 
the new server is atl2.richb-hanover.com. Do you get similar performance from 
both servers?

- Is this plan to bandwidth-limit abusers realistic? Will it be possible to 
design rules that exclude abusers while allowing legitimate research use? (I'm 
concerned that running five tests in a row in 10 minutes might look like an 
every-five-minute abuser...)

- Should I use one of the Linode nanoservers?

- Should we move the netperf.bufferbloat.net name to use the existing flent 
server farm machines?

- Are there other approaches to supporting netperf.bufferbloat.net?

Many thanks!

Rich


___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] Cake in Riverbed?

[Rich Brown]

I can't remember whether I had seen this on the list or not: It appears that 
Riverbed is offering CAKE on its branch gateways...

https://support.riverbed.com/bin/support/static/hgc5k5odj0e955sd2uk2qr4ir5/html/7h0cpt4lqflt1k1pfdpth18at9/sc_ug_html/index.html#page/sc_ug%2Fqos.html%23

Rich
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] SQM Settings for Bonded DSL?

[Rich Brown]

My DSL modem is running the native/factory OS.

A separate OpenWrt modem (WRT3200ACM with davidc502's firmware) that's running 
SQM handles traffic from the laptops/phones/desktops.

Rich


> On Jun 21, 2018, at 2:10 PM, David Lang  wrote:
> 
> Is that router running openwrt? or are you having to operate downstream of it?
> 
> I've got a bonded DSL line (10/2) and the current router has a 2.6 kernel, 
> and has horrible buffer bloat, but I have not been able to track down 
> something I can use to replace it.
> 
> David Lang
> 
> On Thu, 21 Jun 2018, Rich Brown wrote:
> 
>> Hi folks,
>> 
>> Our local DSL ISP (Consolidated Communications, Inc, formerly Fairpoint) 
>> recently installed a Smart/RG SR555ac (https://www.smartrg.com/sr555ac) 
>> bonded ADSL2 modem.
>> 
>> I seem to remember earlier messages stating that the dual queues in the DSL 
>> modem screwed up (or, de-optimized) the SQM in the router. 
>> The web GUI does provide info on SNR, sync rates, etc. Any advice for SQM 
>> beyond the standard, "measure the no-SQM speed, then start at 5% below..."? 
>> Thanks.
>> 
>> Rich
>> ___
>> Bloat mailing list
>> Bloat@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] SQM Settings for Bonded DSL?

[Rich Brown]

Hi folks,

It's great to get all this advice from people who know more than I do :-) (I 
should note that the bonded DSL seemed to be working as expected using the 
no-sqm speed minus 5% procedure.)

Once all the solstice celebrations this weekend settle down (Happy Summer!), 
I'll pull together some data from the DSL modem and the ATM Overhead Detector 
and share it with the list.

Best,

Rich

> On Jun 21, 2018, at 9:35 AM, Sebastian Moeller  wrote:
> 
> Hi Jonathan,
> 
>> On Jun 21, 2018, at 15:16, Jonathan Foulkes  wrote:
>> 
>> Hi Rich, Sebastian,
>> 
>> Most bonded modems do a good job of making the line look just like a single, 
>> higher-capacity line. The only major issues I’ve observed is when the bonded 
>> lines have some asymmetry to them (e.g. one link has weaker SNR), then the 
>> bond drops, re-synchs and continues. But occasionally, it will run for 
>> extended periods at 50% capacity (i.e. on only one of the lines).
>> It uses PMT protocols, so not at all like mwan3, as any one connection can 
>> achieve full throughput of the bonded set.
>> 
>> I have data on hundreds of bonded lines, and other than the scenarios I 
>> mentioned above about bad bonds, as far as SQM goes, it behaves just like 
>> any other DSL line of equal capacity. So all the same guidelines would apply.
>> 
> 
> Thank you very much, all very useful to know.
> 
> 
> @Rich, could I ask you to try to run the 
> https://github.com/moeller0/ATM_overhead_detector on your shiny bonded link? 
> I am really curious how this will cope with such a link (I expect no issues, 
> but the proof and the pudding thing still applies).
> 
> Also I see that bonding might have additional overhead that might not be per 
> packet so will not be picked up by ATM_overhead_detector, so you might need 
> to aim a bit lower with the shaper settings than usually.
> 
> Best Regards
>   Sebastian
> 
> 
> 
> 
>> I hope that helps,
>> 
>> Jonathan Foulkes
>> 
>>> On Jun 21, 2018, at 7:14 AM, Sebastian Moeller  wrote:
>>> 
>>> Hi Rich,
>>> 
>>> 
>>> 
>>>> On Jun 21, 2018, at 13:08, Rich Brown  wrote:
>>>> 
>>>> Hi folks,
>>>> 
>>>> Our local DSL ISP (Consolidated Communications, Inc, formerly Fairpoint) 
>>>> recently installed a Smart/RG SR555ac (https://www.smartrg.com/sr555ac) 
>>>> bonded ADSL2 modem.
>>>> 
>>>> I seem to remember earlier messages stating that the dual queues in the 
>>>> DSL modem screwed up (or, de-optimized) the SQM in the router. 
>>> 
>>> As far as I can see this should not really cause an issue (besides that 
>>> latency for single packets will be limited by the fact that you have two 
>>> half-total-bandwidth links) except maybe that there might be additional 
>>> overhead for the bonding on the link, but I have never looked at channel 
>>> bonding so this is pure spekulation. I assume here that your Modem handles 
>>> the bonding transparently. I could envision that running an non-transparent 
>>> load-balancer (like with mwan3 under openwrt) might introduce issues for 
>>> sqm, but if all you see is one ethernet link to the modem I do not expect 
>>> any major quirks (except latency not being in line with the expectancy from 
>>> total bandwidth).
>>> 
>>>> 
>>>> The web GUI does provide info on SNR, sync rates, etc. Any advice for SQM 
>>>> beyond the standard, "measure the no-SQM speed, then start at 5% 
>>>> below..."? Thanks.
>>> 
>>> I am curius myself and would like to ask you to keep me/the list posted 
>>> on whatever you find out about the applicability of sqm to bonding.
>>> 
>>> Best Regards
>>> Sebastian
>>> 
>>> 
>>>> 
>>>> Rich
>>>> ___
>>>> Bloat mailing list
>>>> Bloat@lists.bufferbloat.net
>>>> https://lists.bufferbloat.net/listinfo/bloat
>>> 
>>> ___
>>> Bloat mailing list
>>> Bloat@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/bloat
>> 
> 

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] SQM Settings for Bonded DSL?

[Rich Brown]

Hi folks,

Our local DSL ISP (Consolidated Communications, Inc, formerly Fairpoint) 
recently installed a Smart/RG SR555ac (https://www.smartrg.com/sr555ac) bonded 
ADSL2 modem.

I seem to remember earlier messages stating that the dual queues in the DSL 
modem screwed up (or, de-optimized) the SQM in the router. 

The web GUI does provide info on SNR, sync rates, etc. Any advice for SQM 
beyond the standard, "measure the no-SQM speed, then start at 5% below..."? 
Thanks.

Rich
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] bufferbloat.net server having troubles?

[Rich Brown]

I just went to bufferbloat.net and 
https://www.bufferbloat.net/projects/bloat/wiki/RRUL_Chart_Explanation/ and am 
receiving intermittent 502 & 522 errors. Is anyone else seeing this? Let me 
know if you need more details. Thanks.

Rich
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Update to status of Make Wi-Fi Fast page?

[Rich Brown]

> On Feb 14, 2018, at 12:00 PM, bloat-requ...@lists.bufferbloat.net wrote:
> 
>> Does someone who's close to the work want to update this with any
>> news/more conclusive statements? Thanks.
> 
> Sure, updated :)

Thanks! (https://www.bufferbloat.net/projects/make-wifi-fast/wiki/ 
)

Rich___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] Update to status of Make Wi-Fi Fast page?

[Rich Brown]

Hi folks,

I have been seeing Jim Gettys' post echo around the intertubes. This caused me 
to look at what we're saying about the Make Wi-Fi Fast project.

The "Current Status" at 
https://www.bufferbloat.net/projects/make-wifi-fast/wiki/ says:

---
As of early 2017, we have achieved many of these improvements, specifically a 
decrease of latency by at least an order of magnitude, with fair sharing of air 
time across fast and slow devices.

• Working software is available in LEDE firmware that runs on 
off-the-shelf routers, x86 boxes, and embedded systems.
• An academic paper describing the current state of the working 
software has been submitted for publication; a preprint is available at Ending 
the Anomaly: Achieving Low Latency and Airtime Fairness in Wifi.
• An earlier recorded presentation and status of the Make Wifi Fast 
Project
---

Does someone who's close to the work want to update this with any news/more 
conclusive statements? Thanks.

Rich

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Win10 Updates vs cake

[Rich Brown]

> On Dec 22, 2017, at 4:12 AM, Mario Hock  > wrote:
> 
> Can you also track packet loss rates / packet loss probability? And also 
> throughput/progress of the Windows update connections?

Oh, yes... I meant to add in the initial report that during times of Win10 
downloads, packet loss for all services went to pieces.

I run Intermapper at home to monitor uptime/packet loss/performance of lots of 
devices across the net. During those downloads, packet loss frequently jumped 
to 5-6% for most devices.

Thanks again,

Rich___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Win10 Updates vs cake

[Rich Brown]

Kevin,

> Once the cake repo is sorted out I can redo & resubmit the patches for both 
> master & 1701 - and with a bit of luck we’ll all be in an even place again.

Thanks for this update. I'm subscribed to the cake github stream, so I hear 
about changes there.

If I understand correctly, after those patches are resubmitted, 17.01 will have 
the newest cake with all bells/whistles. 

Would you post a note here to a) let us know when they're available, and b) how 
"normal people" (that is, non-developers) can get this newest code? (I'm happy 
to post instructions to the forum if no one beats me to it.) Thanks.

Rich___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] Win10 Updates vs cake

[Rich Brown]

I'm using LEDE 17.01.4 on my Archer C7v2. I have a 7mbps/768kbps ADSL2+ 
connection through Fairpoint. The modem stats page shows its "attainable rates" 
(kbps): 13330/1272 and Rates: 8271/1181. My SQM settings are:

Download: 7000 (kbps)
Upload: 925 
Queue Disc: Cake/piece_of_cake.qos
Link Layer: ATM/44 bytes overhead
Advanced Options: default

I have noticed that Win10 updates cause the network connection to become 
unusable for other services/people, as if I had bufferbloat. But ping times 
remain stable - they jump from ~20-22 msec unloaded to 40-50 msec.

Experiments I have tried:

- Setting download speed to 5000 makes the connection usable for other people, 
although the ping times remain about the same (40-50 msec)

- Setting the download speed to 8600 still keeps ping times down, but that 
really harms other people's performance.

- The link rates (download and upload) seem to track the SQM setting, measured 
with both YAMon and the built-in real-time graphs.  I get ~6,000 kbps with a 
7000 download setting, I got ~3,000kbps at the 5000 setting. I get ~8500 kbps 
after setting download to 8600.

- This doesn't seem to happen when I'm downloading other kinds of files (I 
haven't tried torrenting files...) Downloading non-Win10 update files seems to 
leave the connection in a fairly responsive state.

Any thoughts? What other experiments should I make? Thanks!

Rich
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] Benefits of Asymmetric links

[Rich Brown]

Hi folks,

Just to inject a touch of reality into the discussion...

> On Dec 3, 2017, at 10:44 PM, bloat-requ...@lists.bufferbloat.net wrote:
> 
>> I can buy 300/10 megabit/s access from my cable provider.
> 
> Don't!

It would be wonderful to get a fast, symmetric link from my ISP, but here's the 
situation for my town in the northeast US:

- I currently subscribe to 7mbps/768kbps DSL from Fairpoint 

- Fairpoint can provide up to 25/1 mbps DSL.

- There is one Wireless ISP, who can give 4 mbps service (but it is symmetric!) 
Their service is spotty - it's pretty hilly here.

- Cell reception is also spotty - there's one bar on the town common (in the 
center of town) - you can't reliably make phone calls.

- Comcast refuses to serve the town common with cable TV/internet, let alone my 
house which is 1.5 miles away.

- There's a committee working to see if we could provide fiber connections to 
people in town, but no details have been finalized, and would not happen 'til 
2019, if it did.

- This is symptomatic of the internet conditions for a huge number of folks in 
the US who aren't in heavily populated areas. I won't rant about the causes 
(you can guess my thoughts about monopoly providers) but this is the reality on 
the ground here. 

My conclusion: an asymmetric link is far better than no link, so I'm glad that 
that people are thinking about ack filtering.

Best,

Rich
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Update Cake page on bufferbloat.net?

[Rich Brown]

> On Nov 3, 2017, at 2:59 PM, Dave Taht  wrote:

>> I saw a blog posting that was enthusing about codel/fq_codel, and I was 
>> moved to
>> respond that the state of the art was now cake.
> 
> where?

This article (https://www.pcmech.com/article/bufferbloat-fix-slow-network/ 
) is a pretty 
sub-standard explanation of bufferbloat. But I didn't want to come across as 
the "smartest (smart-ass) guy in the room"

My plan was to gently correct the worst errors/misperceptions ("it's the 
bottleneck, stupid") and say that the state of the art had moved ahead, even of 
fq_codel, then point to the Cake page on the site.

>> But I looked at the Cake page on Bufferbloat.net  
>> and wonder if everything there
>> is true, or whether it would be good to update
>> it. https://www.bufferbloat.net/projects/codel/wiki/Cake/ 
>> 
> 
> It's pretty much true.

Good. I will try this weekend to organize that info into a page that would 
serve well readers like those from the pcmech.com  site 
who're new to the subject, and curious about Cake/Bufferbloat.

Rich___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] Update Cake page on bufferbloat.net?

[Rich Brown]

Hi folks,

I saw a blog posting that was enthusing about codel/fq_codel, and I was moved 
to respond that the state of the art was now cake.

But I looked at the Cake page on Bufferbloat.net and wonder if everything there 
is true, or whether it would be good to update it. 
https://www.bufferbloat.net/projects/codel/wiki/Cake/

(Once the technical information on that page is current, I would also be 
willing to tackle a modest reorganization to add some background info to the 
beginning so that we can cite that page as the primary reference for Cake.)

Thanks!

Rich
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] Bufferbloat's Greatest Hits :-)

[Rich Brown]

Your smile for today... My Google Alert for Bufferbloat gave me a link to this 
page. 

http://ytmp3.stream/mp3/bufferbloat-3.html

It appears to contain MP3s of our videos, but I have declined/been afraid to 
start playing any of them...

Rich
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] What is your favourite Flent feature?

[Rich Brown]

> On Apr 25, 2017, at 12:00 PM, bloat-requ...@lists.bufferbloat.net wrote:
> 
> What is your favourite Flent feature?

You already know this, but my favorite feature of Flent is that it gives 
repeatable tests. So much of what I see on various forums that passes for "a 
test" is likely to be a near-random result. If I knew people were using Flent, 
I'd be able to ascribe more value to their results.

Best,

Rich___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] Paper re: latency in cellular networks

[Rich Brown]

A Google Alert points out this paper about latency in cellular networks:

"On Bufferbloat and Delay Analysis of MultiPath TCP in Wireless Networks" by 
Yung-Chih Chen & Don Towsley of University of Massachusetts, Amherst.

https://www.scribd.com/document/341936507/14-Networking-Mptcp-Bufferbloat

 



___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] Big plug for Toke's paper in phys.org

[Rich Brown]

I just saw a nice article about reducing latency in the internet, with a 
citation of Toke's paper at:

https://phys.org/news/2017-02-smarter-routers-latency-internet.html
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] I wrote a blog entry about devices I know of out there...

[Rich Brown]

> On Feb 3, 2017, at 2:50 PM, David Lang <da...@lang.hm> wrote:
> 
> On Fri, 3 Feb 2017, Rich Brown wrote:
> 
>> Friends in town say that Fairpoint recommends a Zyxel VMG4380-B10A modem for 
>> bonded pairs. They seem to have fairly good luck with it.
> 
> I've got a VMG4325-B10A and it's horrific, still running a 2.6 kernel. I 
> routinely have several seconds of buffering on my upload. according to the 
> Zyxel site, the 4380 has a coax connection.

Yes, that's why it makes sense to stick LEDE in series with the Zyxel, to let 
LEDE handle the queueing. 

I've actually never seen a Zyxel, so can't speak to the connectors. Hmmm... The 
Zyxel specs at 
https://www.zyxel.com/me/en/promotions/promotion_20130416_076615_me_en.shtml 
show the 4380 as having both a GbE connector (as if... :-) and a coax HPNA 
connector.

Rich

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] Connection limits at netperf.bufferbloat.net

[Rich Brown]

Hi folks,

My bandwidth bill for netperf.bufferbloat.net was creeping up (exceeding the 4 
TByte/month default for my VPS). It's easy to buy more bandwidth, but...

Analysis of the logs show there are many IP addresses (remarkably, a large 
number in Portugal) that were establishing >1000 netperf connections per hour 
(most hosts were creating exactly 1080 connections/hour, every hour for days at 
a time, also remarkable).

I had created a script that would analyze the log files and block the heavy 
users in iptables. This worked for a while (~6 months) but the tide keeps 
coming in, and I needed a new algorithm.

I have just (within the last hour) implemented an iptables filter that blocks 
new connections after it has received 20 connections within 120 seconds. It 
seems to work in my simple testing [1]

I write to you because:

1) I've changed the test server for many people. I'm hopeful that it isn't a 
big change, but I want to alert you to the possibility of different results.

2) If this affects your test regime(s), let's talk about whether there's a way 
to tweak the filter

Many thanks!

Rich

[1] Test procedure

- sh betterspeedtest.sh -t 10# 10 seconds, normal settings, worked 
as expected
- sh betterspeedtest.sh -t 10 -n 100 # 100 simultaneous connections, upload 
test failed (speed=0Mbps)
- Wait three minutes
- sh betterspeedtest.sh -t 10# 10 seconds, worked again
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] Reasons to prefer netperf vs iperf?

[Rich Brown]

As I browse the web, I see several sets of performance measurement using either 
netperf or iperf, and never know if either offers an advantage.

I know Flent uses netperf by default: what are the reason(s) for selecting it? 
Thanks.

Rich
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Fixing bufferbloat in 2017

[Rich Brown]

Dave Täht attempts to refocus the group, and asks: 

> Can I encourage folk to think big and out of the technical box?
> 
> On Tue, Nov 22, 2016 at 7:32 AM, Dave Taht  wrote:
>> What's left to do?
>> 
>> What else can we do?
>> 
>> What should we stop doing?
>> 
>> What can we do better?

Lots of good thoughts on this thread. 

My impression is that we have reached a strong technical point. We have solved 
some really hard, really significant problems. We are in a position to Declare 
Victory on a large part of the problem, even though there are loads of details 
to clean up.

Most of the suggestions in this thread deal with Getting the Word Out. That's 
good - that's the declaring victory part. The bad news is that this is not our 
collective skill set. 

Some thoughts about what we *can* do:

1) Toke et al published (are publishing?) a scholarly paper on the 
make-wifi-fast efforts that "looks like real academic research" (by *actually 
being* academic research :-) This makes it credible to other academicians, and 
throws down the gauntlet with a low latency value that others need to improve 
upon. (No more academic papers that say, "We really worked hard, and got 
latency down to 100 ms. Aren't you proud of us?")

Are there other papers bottled up inside team members?

2) I wonder if we would gain credibility by updating the bufferbloat web site. 
I see two things that could be done.
a) Change the www.bufferbloat.net home page to use a one-page design 
(see, just as an example, https://bootstrapmade.com/demo/Baker/) with sections 
that address our primary constituencies: Home users, Gamers, Manufacturers, 
Software Developers, and Network Researchers. It adds a bit of polish, while 
keeping our message simple. People can drill down into the (existing) pages for 
more information. 
b) We should make a pass through the site, organizing according those 
constituencies, and removing content that is no longer relevant.
c) I also grabbed the DNS name "makewififast.com" in case we want to 
use it.

3) I think it's great to contact reviewers - ArsTechnica and AnandTech were 
mentioned. (I did reach out to Wirecutter and ask that they incorporate 
bufferbloat tests in their router recommendations. I was disappointed by the 
total radio silence.)

4) Do we know people at any of the cell phone companies, or router vendors on 
whom we could try one last push?

As part of organizing my thoughts for this note, I also collected the following 
ideas from this thread. I add my $0.02 below.

Rich

1) I don't see that Ookla has much incentive to include bufferbloat 
measurements in their test, since they private-label it for lots of ISPs who 
(presumably) wouldn't want their CPE to be proven crappy. ("It is difficult to 
get a man to understand something, when his salary depends upon his not 
understanding it!" -Upton Sinclair)

2) The gamer community seems like such a perfect target for these improvements. 
But I fear that the thought leaders are so wrapped up in the fame generated by 
their own clever QoS tricks that they can't believe that fq_codel plus the 
make-wifi-fast fixes could possibly address such a complicated subject. (Upton 
Sinclair, again.)

3) On the other hand, Comcast (whose DOCSIS modems *might* someday support PIE 
or other SQM) is in a position to benefit from an increased awareness of the 
phenomenon, leaving a little ray of hope.

[Note - I wrote 4 & 5 below before I learned of IQrouter... I'm still skeptical 
of the mainline router vendors adopting this technology anytime soon into their 
stock firmware.] 

4) I do wish that there were a way to we could stop saying, "Just update your 
router firmware (trust us...)"  as a solution. It would be so much better to 
say, "Just buy this low-cost (or medium-cost) router that'll make you supremely 
happy."

5) But I'm not hopeful that any of the COTS router vendors are going to adopt 
these techniques, simply because they've been impervious to our earlier 
entreaties. That doesn't mean we shouldn't try again - it'd be a helluva 
competitive advantage to incorporate the 25-50 man years of intense software 
development that has gone into this work.

6) It *is* a good idea to think about attracting the attention of vendors who 
are hurt by bufferbloat - VoIP, video streaming folks, gaming companies, etc. 
But it feels like the wrong end of the lever - a gaming company can't fix 
crappy CPE, and they're stuck saying 

7) Cell phones are another place that obviously would benefit, although, again, 
it's hard to break through the notion that "It's always been like that..."

What else?

Rich

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] fixing bufferbloat in 2017

[Rich Brown]

I feel particularly acutely the fact that we don't have a good "simple to 
deliver" solution today for the curious (not deeply commited) person. I was 
trying to help a friend with a TP-Link Archer C7, but was stymied because I 
can't simply install OpenWrt CC because of their "FCC fix".

Besides, OpenWrt CC is now eight months old: although it's stable, it probably 
doesn't have the latest bufferbloat/make-wifi-fast fixes. The DD development 
and release process is uncertain. DD-Wrt is of unknown quality (does it include 
fq_codel? make-wifi-fast? will it ever?), with a really ugly and not 
terribly-welcoming forum. I have great hopes for LEDE, but it is still 
sprinting toward its first stable release, so we can't count on it today.

So we are stuck in the short term with a "lack of a product". I have faith that 
it will happen, but we need to build our plan around its arrival date.

Rich
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] "Best Practices for Benchmarking" article still up to date?

[Rich Brown]

The "Best Practices for Benchmarking" article 
(https://www.bufferbloat.net/projects/codel/wiki/Best_practices_for_benchmarking_Codel_and_FQ_Codel/)
 came up in a forum 
(https://forum.lede-project.org/t/build-for-netgear-r7800/316/9)

Is this still good advice? Thanks.

Rich
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] Differences between dslr cli program vs. web test

[Rich Brown]

I find a marked difference between the measurements that come from the dslr cli 
program and the web tester.

The CLI seems to show slower speeds than the web tester. Here's data from one 
web run, and two CLI runs.

I'm running on a mid-2015 MacBook Pro, 2.5 GHz i7, 16 GBytes of RAM, OSX 
10.10.5, with 7mbps/768kbps DSL. What other information would you want to see? 
Thanks!

Rich

---
Web GUI: http://www.dslreports.com/speedtest/6241603 (shows 6.91/0.57 Mbps)

bash-3.2$ ./dslrcli-darwin-amd64 --version
Dslrcli version 0.1 - 15-Nov-2016

bash-3.2$ ./dslrcli-darwin-amd64 --uploadtime 30 --downloadtime 30
Selecting nearest servers
Download Testing.
Upload Testing.
Uploading results...
Download : 6.62 Megabit/sec Upload : 901.93 Kilobit/sec
http://www.dslreports.com/speedtest/6241561

bash-3.2$ ./dslrcli-darwin-amd64
Selecting nearest servers
Download Testing.
Upload Testing.
Uploading results...
Download : 6.58 Megabit/sec Upload : 778.23 Kilobit/sec
http://www.dslreports.com/speedtest/6241492
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] QCut - "Understanding on-device bufferbloat for cellular upload", Guo et al

[Rich Brown]

Google Alerts sent a link to this paper: "Understanding on-device bufferbloat 
for cellular upload" available at: http://dl.acm.org/citation.cfm?id=2987490 
(ACM Paywall...)

In my two-minute skim of the paper, I see they describe how latency gets bad on 
cellular devices (no kidding) and then propose QCut, a queue inserted between 
the qdisc and the hardware. They say:

---
... QCut operates in the kernel space and takes as input only information of 
buffer occupancy and transmission statistics, which is exposed by most cellular 
radio firmware from Qualcomm and likely other vendors.

Since directly limiting the firmware buffer occupancy is difficult, QCut 
controls the firmware queuing delay indirectly in the kernel by controlling how 
fast packets from Qdisc flow into the firmware buffer. QCUT estimates the radio 
firmware buffer occupancy and queuing delay to decide the transmission of 
packets to the firmware dynamically...
---

Their analysis and charts show that QCut helps a lot.

I found it interesting that they make measurements with both weak and strong 
signal strength, to indicate the hits that are caused by differing signal 
strength.

Enjoy!

Rich
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

[Bloat] Comcast's NANOG slides re Bufferbloat posted (Oct 2016)

[Rich Brown]

https://www.nanog.org/sites/default/files/20160922_Klatsky_First_Steps_In_v1.pdf
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Open Source Speed Test

[Rich Brown]

Thoughts:

- The page is attractive, and definitely easy to use.
- Grammar: "For best results, please limit the *number* of applications or 
devices..."  (not "amount")
- Inaccurate display of measured speeds (see below)
- When will the source code be opened up?

I submitted the following on the feedback page. Thanks! -Rich

--
I realize that I'm not a prime candidate (7mbps/768kbps DSL) but the test 
results were grossly inaccurate:

- Ping results were 36 msec (about right for HTTP ping tests)
- Download speed toggled between 6 & 7, and settled on exactly "7" (not 7.0 or 
6.9...)
- Upload speed displayed as "0.0" - this is plain wrong.

Comments:

- Provide at least two (or three, if warranted) significant digits: 6.9 mbps 
and 0.71 mbps would be more believable.

- And add latency measurements *during* transfers, too
--
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat