Re: [Bloat] Bufferbloat on 4G Connexion

[Toke Høiland-Jørgensen]

Luca Muscariello  writes:

> On Thu, Oct 24, 2019 at 11:34 AM Toke Høiland-Jørgensen 
> wrote:
>
>> Luca Muscariello  writes:
>>
>> > On Wed, Oct 23, 2019 at 2:27 PM Toke Høiland-Jørgensen 
>> > wrote:
>> >
>> >> Rich Brown  writes:
>> >>
>> >> >> On Oct 23, 2019, at 5:54 AM,> >> erik.tarald...@telenor.com>> wrote:
>> >> >>
>> >> >> If you could influence the 4G vendors to de-bloat their equipment,
>> >> >> would you recommend BQL, L4S or codel/cake?
>> >> >
>> >> > I've been enjoying this discussion and wonder whether the work going
>> >> > on in the make-wifi-fast
>> >> > (https://lists.bufferbloat.net/pipermail/make-wifi-fast/) is
>> relevant.
>> >> >
>> >> > I only have a 30,000 foot understanding of this work, but it seems the
>> >> > use of AQL (Airtime Queue Limit) maps better onto the vagaries of
>> >> > 4G/5G radio transmissions than BQL. Specifically, having a measurement
>> >> > of the actual time it takes to transmit a packet might give additional
>> >> > information about the current link speed, with the potential for
>> >> > adjusting the codel target, etc.
>> >>
>> >> Indeed, I suspect something like AQL would work for LTE as well. At the
>> >> right level; think this might need to be in the firmware (which in turn
>> >> could push back on the host).
>> >>
>> >> > Separately, I also wonder whether the Air Time Fairness algorithm
>> >> > might provide a benefit if the cellphone tower station manufacturers
>> >> > chose to get into the game.
>> >>
>> >> LTE base stations already does TDMA scheduling (which they can do easily
>> >> because they are centralised and own the license band); airtime fairness
>> >> is about getting the same benefits into WiFi that LTE has been enjoying
>> >> from the get-go :)
>> >>
>> >
>> > There is one main difference between ATF and the kind of TDMA
>> > realized by an LTE scheduler (but also HSDPA/HSUPA).
>> > Toke correct me if I'm wrong.
>> >
>> > The current ATF scheduler for WiFi does airtime-DRR based on the
>> > current PHY rates, is that right? Side question, how do you measure
>> > current?
>>
>> s/current/last/. The ATF scheduler does everything after-the-fact, by
>> accounting the actual TX time of a transmission after it has completed.
>> So no fancy scheduling or prediction tricks are needed; with the
>> tradeoff being coarser granularity of the fairness achieved (i.e., there
>> can be unfairness on short timescales).
>>
>> In the airtime queue limit work that's ongoing, we do ahead-of-time
>> airtime estimation to limit queueing in firmware. But this still just
>> uses the last TX rate recorded for the given station to calculate the
>> estimate.
>>
>> > In LTE TDMA makes use of what is called multi-user diversity gain
>> > by scheduling users when they are at their relative best radio condition.
>> > Typically the user with the best current radio condition NORMALIZED
>> > over the average radio conditions. The average can be based on a
>> > moving average or a sliding window. This is the case of the widely used
>> > David Tse's proportional fair scheduler.
>> >
>> > This means that TDMA is still in place to share air-time fairly but the
>> > scheduler will tend to avoid bad radio conditions.
>> >
>> > From a theoretical point of view if you do that the total capacity
>> > of the AP can increase with the number of stations (I think
>> logarithmically)
>> > as the scheduler surfs across radio quality peaks and not the average
>> radio
>> > quality. Very smart.
>> >
>> > In LTE this is doable as the scheduling time slot is 1ms and the
>> > feedback channel is as fast. Not all TDMAs are equal.
>>
>> Yeah, the LTE MAC is pretty cool. Just a shame that the equipment is so
>> expensive :(
>>
>
> It looks like there is a positive correlation between the size
> of the specifications and the cost to build the associated product :)

Hehe, yeah, funny how that works.

IMO the LTE people went a little bit overboard on the complexity,
though... ;)

-Toke

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Bufferbloat on 4G Connexion

[Luca Muscariello]

On Thu, Oct 24, 2019 at 11:34 AM Toke Høiland-Jørgensen 
wrote:

> Luca Muscariello  writes:
>
> > On Wed, Oct 23, 2019 at 2:27 PM Toke Høiland-Jørgensen 
> > wrote:
> >
> >> Rich Brown  writes:
> >>
> >> >> On Oct 23, 2019, at 5:54 AM, >> erik.tarald...@telenor.com>> wrote:
> >> >>
> >> >> If you could influence the 4G vendors to de-bloat their equipment,
> >> >> would you recommend BQL, L4S or codel/cake?
> >> >
> >> > I've been enjoying this discussion and wonder whether the work going
> >> > on in the make-wifi-fast
> >> > (https://lists.bufferbloat.net/pipermail/make-wifi-fast/) is
> relevant.
> >> >
> >> > I only have a 30,000 foot understanding of this work, but it seems the
> >> > use of AQL (Airtime Queue Limit) maps better onto the vagaries of
> >> > 4G/5G radio transmissions than BQL. Specifically, having a measurement
> >> > of the actual time it takes to transmit a packet might give additional
> >> > information about the current link speed, with the potential for
> >> > adjusting the codel target, etc.
> >>
> >> Indeed, I suspect something like AQL would work for LTE as well. At the
> >> right level; think this might need to be in the firmware (which in turn
> >> could push back on the host).
> >>
> >> > Separately, I also wonder whether the Air Time Fairness algorithm
> >> > might provide a benefit if the cellphone tower station manufacturers
> >> > chose to get into the game.
> >>
> >> LTE base stations already does TDMA scheduling (which they can do easily
> >> because they are centralised and own the license band); airtime fairness
> >> is about getting the same benefits into WiFi that LTE has been enjoying
> >> from the get-go :)
> >>
> >
> > There is one main difference between ATF and the kind of TDMA
> > realized by an LTE scheduler (but also HSDPA/HSUPA).
> > Toke correct me if I'm wrong.
> >
> > The current ATF scheduler for WiFi does airtime-DRR based on the
> > current PHY rates, is that right? Side question, how do you measure
> > current?
>
> s/current/last/. The ATF scheduler does everything after-the-fact, by
> accounting the actual TX time of a transmission after it has completed.
> So no fancy scheduling or prediction tricks are needed; with the
> tradeoff being coarser granularity of the fairness achieved (i.e., there
> can be unfairness on short timescales).
>
> In the airtime queue limit work that's ongoing, we do ahead-of-time
> airtime estimation to limit queueing in firmware. But this still just
> uses the last TX rate recorded for the given station to calculate the
> estimate.
>
> > In LTE TDMA makes use of what is called multi-user diversity gain
> > by scheduling users when they are at their relative best radio condition.
> > Typically the user with the best current radio condition NORMALIZED
> > over the average radio conditions. The average can be based on a
> > moving average or a sliding window. This is the case of the widely used
> > David Tse's proportional fair scheduler.
> >
> > This means that TDMA is still in place to share air-time fairly but the
> > scheduler will tend to avoid bad radio conditions.
> >
> > From a theoretical point of view if you do that the total capacity
> > of the AP can increase with the number of stations (I think
> logarithmically)
> > as the scheduler surfs across radio quality peaks and not the average
> radio
> > quality. Very smart.
> >
> > In LTE this is doable as the scheduling time slot is 1ms and the
> > feedback channel is as fast. Not all TDMAs are equal.
>
> Yeah, the LTE MAC is pretty cool. Just a shame that the equipment is so
> expensive :(
>

It looks like there is a positive correlation between the size
of the specifications and the cost to build the associated product :)




>
> > Maybe the current scheduler in WiFi can be improved to do that. Maybe.
>
> I think 802.11ax is going in that direction. Nothing nearly as advanced,
> but at least there's the possibility of a dedicated back channel...
>

That's right. ax does much better.



>
> -Toke
>
>
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Bufferbloat on 4G Connexion

[Jonathan Morton]

> On 24 Oct, 2019, at 2:51 pm,  
>  wrote:
> 
>> The key thing in both BQL and AQL is that instead of treating all packets the
>> same, you account for how long it will take to transmit them.
>> 
>> In Wifi, your transmission rate can vary by a factor of 1000x (1Mb to over 
>> 1Gb
>> per second), so just counting bytes as BQL does is no longer good enough.
>> 
>> how much does the actual over-the-air transmission rate vary in 4G? I'd bet 
>> that
>> it varies less than wifi does because wifi must retain compatibility with the
>> earliest equipment while 4G no longer supports 2G protocols.
> 
> So in a scenario with a fixed (wall mounted outdoor unit) LTE device BQL 
> could work 
> quite well as the radio will not fluctuate too much.  
> 
> But for a mobil "pocket router" being carried around an AQL approach would be 
> needed?  

Even in a fixed application, there's a lot of variation in where the unit is 
installed - from practically underneath the cell site tower to several miles 
away behind some trees - and service outages at the primary cell site could 
cause large, temporary changes of link bandwidth as the unit switches to a more 
distant cell.  Also, over the course of a day I personally observe tenfold 
variations in available bandwidth due to congestion at peak times, though I'm 
not quite certain whether that's due to radio contention on this particular 
cell site, or on my ISP's backhaul.

AQL would correct for these variations automatically.  All it needs in addition 
to BQL is some indication of how many bytes are appropriate to queue in the 
hardware at any given moment.

 - Jonathan Morton

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Bufferbloat on 4G Connexion

[Toke Høiland-Jørgensen]

 writes:

>> The key thing in both BQL and AQL is that instead of treating all packets the
>> same, you account for how long it will take to transmit them.
>>
>> In Wifi, your transmission rate can vary by a factor of 1000x (1Mb to over 
>> 1Gb
>> per second), so just counting bytes as BQL does is no longer good enough.
>>
>> how much does the actual over-the-air transmission rate vary in 4G? I'd bet 
>> that
>> it varies less than wifi does because wifi must retain compatibility with the
>> earliest equipment while 4G no longer supports 2G protocols.
>
> So in a scenario with a fixed (wall mounted outdoor unit) LTE device
> BQL could work quite well as the radio will not fluctuate too much.
>
> But for a mobil "pocket router" being carried around an AQL approach
> would be needed?

Having setup such an outdoor-mounted unit, I'd say that it's probably a
bit too optimistic to think that it doesn't fluctuate much. Cell
scheduling parameters (including activity from other stations), as well
as weather, can lead to quite significant fluctuations even in a
physically static setup.

So ideally, you'd want to be able to react to changes in either case.
The firmware should certainly have enough information to do that, but if
you can't get that changed, it may be possible to do something BQL-like
(either AQL, or simply a more dynamic variant of BQL) on the host.

-Toke

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Bufferbloat on 4G Connexion

[erik.taraldsen]

> The key thing in both BQL and AQL is that instead of treating all packets the
> same, you account for how long it will take to transmit them.
>
> In Wifi, your transmission rate can vary by a factor of 1000x (1Mb to over 1Gb
> per second), so just counting bytes as BQL does is no longer good enough.
>
> how much does the actual over-the-air transmission rate vary in 4G? I'd bet 
> that
> it varies less than wifi does because wifi must retain compatibility with the
> earliest equipment while 4G no longer supports 2G protocols.

So in a scenario with a fixed (wall mounted outdoor unit) LTE device BQL could 
work 
quite well as the radio will not fluctuate too much.  

But for a mobil "pocket router" being carried around an AQL approach would be 
needed?  


-Erik
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Bufferbloat on 4G Connexion

[Toke Høiland-Jørgensen]

Luca Muscariello  writes:

> On Wed, Oct 23, 2019 at 2:27 PM Toke Høiland-Jørgensen 
> wrote:
>
>> Rich Brown  writes:
>>
>> >> On Oct 23, 2019, at 5:54 AM,> erik.tarald...@telenor.com>> wrote:
>> >>
>> >> If you could influence the 4G vendors to de-bloat their equipment,
>> >> would you recommend BQL, L4S or codel/cake?
>> >
>> > I've been enjoying this discussion and wonder whether the work going
>> > on in the make-wifi-fast
>> > (https://lists.bufferbloat.net/pipermail/make-wifi-fast/) is relevant.
>> >
>> > I only have a 30,000 foot understanding of this work, but it seems the
>> > use of AQL (Airtime Queue Limit) maps better onto the vagaries of
>> > 4G/5G radio transmissions than BQL. Specifically, having a measurement
>> > of the actual time it takes to transmit a packet might give additional
>> > information about the current link speed, with the potential for
>> > adjusting the codel target, etc.
>>
>> Indeed, I suspect something like AQL would work for LTE as well. At the
>> right level; think this might need to be in the firmware (which in turn
>> could push back on the host).
>>
>> > Separately, I also wonder whether the Air Time Fairness algorithm
>> > might provide a benefit if the cellphone tower station manufacturers
>> > chose to get into the game.
>>
>> LTE base stations already does TDMA scheduling (which they can do easily
>> because they are centralised and own the license band); airtime fairness
>> is about getting the same benefits into WiFi that LTE has been enjoying
>> from the get-go :)
>>
>
> There is one main difference between ATF and the kind of TDMA
> realized by an LTE scheduler (but also HSDPA/HSUPA).
> Toke correct me if I'm wrong.
>
> The current ATF scheduler for WiFi does airtime-DRR based on the
> current PHY rates, is that right? Side question, how do you measure
> current?

s/current/last/. The ATF scheduler does everything after-the-fact, by
accounting the actual TX time of a transmission after it has completed.
So no fancy scheduling or prediction tricks are needed; with the
tradeoff being coarser granularity of the fairness achieved (i.e., there
can be unfairness on short timescales).

In the airtime queue limit work that's ongoing, we do ahead-of-time
airtime estimation to limit queueing in firmware. But this still just
uses the last TX rate recorded for the given station to calculate the
estimate.

> In LTE TDMA makes use of what is called multi-user diversity gain
> by scheduling users when they are at their relative best radio condition.
> Typically the user with the best current radio condition NORMALIZED
> over the average radio conditions. The average can be based on a
> moving average or a sliding window. This is the case of the widely used
> David Tse's proportional fair scheduler.
>
> This means that TDMA is still in place to share air-time fairly but the
> scheduler will tend to avoid bad radio conditions.
>
> From a theoretical point of view if you do that the total capacity
> of the AP can increase with the number of stations (I think logarithmically)
> as the scheduler surfs across radio quality peaks and not the average radio
> quality. Very smart.
>
> In LTE this is doable as the scheduling time slot is 1ms and the
> feedback channel is as fast. Not all TDMAs are equal.

Yeah, the LTE MAC is pretty cool. Just a shame that the equipment is so
expensive :(

> Maybe the current scheduler in WiFi can be improved to do that. Maybe.

I think 802.11ax is going in that direction. Nothing nearly as advanced,
but at least there's the possibility of a dedicated back channel...

-Toke

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Bufferbloat on 4G Connexion

[Luca Muscariello]

On Wed, Oct 23, 2019 at 2:27 PM Toke Høiland-Jørgensen 
wrote:

> Rich Brown  writes:
>
> >> On Oct 23, 2019, at 5:54 AM, erik.tarald...@telenor.com>> wrote:
> >>
> >> If you could influence the 4G vendors to de-bloat their equipment,
> >> would you recommend BQL, L4S or codel/cake?
> >
> > I've been enjoying this discussion and wonder whether the work going
> > on in the make-wifi-fast
> > (https://lists.bufferbloat.net/pipermail/make-wifi-fast/) is relevant.
> >
> > I only have a 30,000 foot understanding of this work, but it seems the
> > use of AQL (Airtime Queue Limit) maps better onto the vagaries of
> > 4G/5G radio transmissions than BQL. Specifically, having a measurement
> > of the actual time it takes to transmit a packet might give additional
> > information about the current link speed, with the potential for
> > adjusting the codel target, etc.
>
> Indeed, I suspect something like AQL would work for LTE as well. At the
> right level; think this might need to be in the firmware (which in turn
> could push back on the host).
>
> > Separately, I also wonder whether the Air Time Fairness algorithm
> > might provide a benefit if the cellphone tower station manufacturers
> > chose to get into the game.
>
> LTE base stations already does TDMA scheduling (which they can do easily
> because they are centralised and own the license band); airtime fairness
> is about getting the same benefits into WiFi that LTE has been enjoying
> from the get-go :)
>

There is one main difference between ATF and the kind of TDMA
realized by an LTE scheduler (but also HSDPA/HSUPA).
Toke correct me if I'm wrong.

The current ATF scheduler for WiFi does airtime-DRR based on the current
PHY rates,
is that right? Side question, how do you measure current?

In LTE TDMA makes use of what is called multi-user diversity gain
by scheduling users when they are at their relative best radio condition.
Typically the user with the best current radio condition NORMALIZED
over the average radio conditions. The average can be based on a
moving average or a sliding window. This is the case of the widely used
David Tse's proportional fair scheduler.

This means that TDMA is still in place to share air-time fairly but the
scheduler will tend to avoid bad radio conditions.

>From a theoretical point of view if you do that the total capacity
of the AP can increase with the number of stations (I think logarithmically)
as the scheduler surfs across radio quality peaks and not the average radio
quality. Very smart.

In LTE this is doable as the scheduling time slot is 1ms and the feedback
channel
is as fast. Not all TDMAs are equal.
Maybe the current scheduler in WiFi can be improved to do that. Maybe.

Luca




>
> -Toke
>
> ___
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Bufferbloat on 4G Connexion

[David Lang]

On Wed, 23 Oct 2019, Rich Brown wrote:


I only have a 30,000 foot understanding of this work, but it seems the use of 
AQL (Airtime Queue Limit) maps better onto the vagaries of 4G/5G radio 
transmissions than BQL. Specifically, having a measurement of the actual time 
it takes to transmit a packet might give additional information about the 
current link speed, with the potential for adjusting the codel target, etc.


The key thing in both BQL and AQL is that instead of treating all packets the 
same, you account for how long it will take to transmit them.


In Wifi, your transmission rate can vary by a factor of 1000x (1Mb to over 1Gb 
per second), so just counting bytes as BQL does is no longer good enough.


how much does the actual over-the-air transmission rate vary in 4G? I'd bet that 
it varies less than wifi does because wifi must retain compatibility with the 
earliest equipment while 4G no longer supports 2G protocols.


is it just a matter of each phone getting a smaller slice of the airtime when 
things get busy? or does the actual data rate for that slice of airtime 
decrease as well?


David Lang

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Bufferbloat on 4G Connexion

[Toke Høiland-Jørgensen]

Rich Brown  writes:

>> On Oct 23, 2019, at 5:54 AM,> > wrote:
>> 
>> If you could influence the 4G vendors to de-bloat their equipment,
>> would you recommend BQL, L4S or codel/cake?
>
> I've been enjoying this discussion and wonder whether the work going
> on in the make-wifi-fast
> (https://lists.bufferbloat.net/pipermail/make-wifi-fast/) is relevant.
>
> I only have a 30,000 foot understanding of this work, but it seems the
> use of AQL (Airtime Queue Limit) maps better onto the vagaries of
> 4G/5G radio transmissions than BQL. Specifically, having a measurement
> of the actual time it takes to transmit a packet might give additional
> information about the current link speed, with the potential for
> adjusting the codel target, etc.

Indeed, I suspect something like AQL would work for LTE as well. At the
right level; think this might need to be in the firmware (which in turn
could push back on the host).

> Separately, I also wonder whether the Air Time Fairness algorithm
> might provide a benefit if the cellphone tower station manufacturers
> chose to get into the game.

LTE base stations already does TDMA scheduling (which they can do easily
because they are centralised and own the license band); airtime fairness
is about getting the same benefits into WiFi that LTE has been enjoying
from the get-go :)

-Toke

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Bufferbloat on 4G Connexion

[Rich Brown]

> On Oct 23, 2019, at 5:54 AM, > wrote:
> 
> If you could influence the 4G vendors to de-bloat their equipment, would you 
> recommend BQL, L4S or codel/cake?

I've been enjoying this discussion and wonder whether the work going on in the 
make-wifi-fast (https://lists.bufferbloat.net/pipermail/make-wifi-fast/) is 
relevant.

I only have a 30,000 foot understanding of this work, but it seems the use of 
AQL (Airtime Queue Limit) maps better onto the vagaries of 4G/5G radio 
transmissions than BQL. Specifically, having a measurement of the actual time 
it takes to transmit a packet might give additional information about the 
current link speed, with the potential for adjusting the codel target, etc.

Separately, I also wonder whether the Air Time Fairness algorithm might provide 
a benefit if the cellphone tower station manufacturers chose to get into the 
game.

Thanks.

Rich___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Bufferbloat on 4G Connexion

[Jonathan Morton]

> On 23 Oct, 2019, at 10:28 am,  
>  wrote:
> 
> If you could influence the 4G vendors to de-bloat their equipment, would you 
> recommend BQL, L4S or codel/cake?

BQL is a good start, but needs to go with something else to actually fix the 
problem.  All it does is move (most of) the queue out of hardware and into 
software.  The software side then needs to capitalise on having control of the 
bottleneck queue, by implementing FQ, AQM, or both.

L4S is a complete waste of time.  It requires basically replacing the entire 
Internet, including endpoints, to work properly.  If the endpoints are not 
replaced, then the middleboxes behave like a conventional AQM (see below).

Codel would be a HUGE help.  It's a well-tested example of an AQM that's 
specifically designed to work with TCP, and should work equally well with other 
protocols designed to be TCP friendly.  There are other AQMs, such as PIE, 
BLUE, and even the old standby WRED, that would also be major improvements over 
the status quo, though they don't perform as well on common TCP traffic as 
Codel.  The basic requirement here is to bring the maximum *standing* queue 
down from hundreds or thousands of milliseconds to tens or singles; Codel 
achieves 5ms with its default Internet-tuned settings.

I use a slight variant of Codel in Cake, which I call COBALT.  The main 
algorithm is just a refactoring of Codel, but I've also addressed two blind 
spots in Codel's design.  One is the behaviour upon reactivating very shortly 
after the standing queue was initially brought under control; COBALT keeps 
better track of previous state in that case.  The other is addressing heavy 
load from non-TCP-friendly traffic, which requires a probabilistic drop 
function rather than a scheduled drop or mark.  To take care of the latter, 
I've overlaid a version of the BLUE algorithm.

I believe Codel and COBALT should be reasonably straightforward to implement in 
hardware.  I could help to explain the algorithmic details and rationale to 
hardware engineers if need be, and help them determine the right design 
tradeoff.

FQ's main benefit is allowing "sparse" flows of traffic, which tend to be more 
sensitive to latency, to bypass queues of "bulk" traffic which tends to be more 
throughput sensitive.  This amounts to reducing "inter-flow induced latency", 
where AQM focuses on reducing "intra-flow induced latency".  The main 
difficulty is allegedly in identifying flows and maintaining the per-flow state 
and individual queues; these should not be insurmountable obstacles, but I will 
admit the complexity of implementing this in hardware is higher than for a 
plain AQM.

I would note, however, that 4G already incurs some of this complexity through 
the need to individually queue, aggregate, schedule, and transmit a stream of 
traffic to each mobile station.  It should be relatively easy to add an AQM 
instance per station, thereby ensuring that congestion signals incurred by one 
subscriber saturating their own capacity don't spill over into other 
subscribers' traffic.

Combining AQM with FQ is the gold standard, minimising both inter-flow and 
intra-flow induced latency.  That's what fq_codel and Cake do.  If that was 
deployed in the right places in a 4G network, you could consider the problem 
solved.  For an estimate of implementation complexity, take an FQ 
implementation and add an AQM for each flow's queue.

As a middle ground I could suggest one of my latest projects, CNQ (Cheap Nasty 
Queuing).  This aims to provide some of the benefits of FQ+AQM, but with only 
slightly higher implementation complexity than a plain AQM, at the cost of 
inferior flow-isolating performance than true FQ.  It should therefore be more 
suitable for high-volume nodes than an FQ-based algorithm.  A software 
implementation for demo purposes is presently in the works, and we should have 
some initial performance data fairly soon.  I would seriously consider 
deploying CNQ on a per-station basis in 4G.

 - Jonathan Morton

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Bufferbloat on 4G Connexion

[Sebastian Moeller]

> On Oct 23, 2019, at 09:28,  
>  wrote:
> 
> If you could influence the 4G vendors to de-bloat their equipment, would you 
> recommend BQL, L4S or codel/cake?

IMHO, something like BQL would be great, as would codel/cake in the devices. 
L4S however, has not been thoroughly tested and hence should be considered 
research-grade at best. To my knowledge none of the L4S papers/thesis so far 
have been performed bi-directional saturation of the network, even the PhD 
thesis titled Destruction Testing: Ultra-Low Delay using Dual Queue Coupled 
Active Queue Management).
The IMHO best set of L4S tests at https://github.com/heistp/sce-l4s-bakeoff 
indicates a number of "rough" edges in L4S that should be addressed before 
considering a roll-out
Personally I doubt that all of these can be fixed within the restraint solution 
space the L4S project forced upon itself (avoid flow-queueing at any cost).
So as it stands both codel and cake (and especially the fq_codel variant) have 
proven their usefulness in the real world, one of the big issues with both is 
that unless the transport interface implements some thing similar to BQL both 
require a computationally costly traffic shaper (cake has its own shaper built 
in codel/fq_codel need an additional shaper). It is not necessarily sheer 
number of CPU cycles, but that they seem to require low latency access to the 
CPU when ever deadlines approach, to try to put things to simplistically.



Best Regards
Sebastian


> 
> 
> -Erik
> 
> 
> 
> Fra: Bloat  på vegne av Jonathan Morton 
> 
> Sendt: 22. oktober 2019 23:02
> Til: Guillaume ROBIER
> Kopi: bloat@lists.bufferbloat.net
> Emne: Re: [Bloat] Bufferbloat on 4G Connexion
> 
>> On 11 Oct, 2019, at 5:56 pm, Guillaume ROBIER  
>> wrote:
>> 
>> I am new to this mailing list and I discovered the bufferbloat in December 
>> 2018. I work on 4G routers and the bufferbloat is very present on this type 
>> of link (4G). I contact you today to find out if people have experimented 
>> with solutions on this type of link or have configuration suggestions, 
>> because the classic fq_codel or piece_of_cake and pie do not allow to fix 
>> the bufferbloat.
> 
> This is actually my own situation at home.  My solution is to insert Cake 
> shapers on both upstream *and* downstream directions, and adjust their 
> bandwidth settings according to variations in available 4G speed.  To do this 
> I use an IQrouter, which is basically a TP-Link Archer C7 with custom 
> firmware.
> 
> One of the difficulties with 4G in particular is that the link capacity 
> varies a great deal according to both radio propagation conditions (weather, 
> obstructions) and local usage by other subscribers.  That means the right 
> bandwidth setting for the small hours of the night, when nobody is awake, 
> will leave you with a lot of bloat in the evening, when everyone is both 
> awake and home from school/work.  You will need to measure these trends and 
> set up a bandwidth schedule accordingly.
> 
> - Jonathan Morton
> ___
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
> ___
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Bufferbloat on 4G Connexion

[David Lang]

As I understand it, BQL is pretty much a prereq for anything better, if you 
treat a 64 byte packet as if it was a 1500 byte packet, everything else is going 
to fail.


fq_codel is the next stage, it keeps large queues from forming, which makes 
everything work much better.


Cake layers on additional 'fairness' optimizations, but for optimal results, you 
want to have it know the available bandwidth (which is dynamic and changing).


there isn't a single 'do this' right answer, there are a series of 
optimizations. Each additional layer gives better results when tuned, but can
require more tuning, and in many cases, can give worse results than a simpler 
layer if the more complex one is badly tuned.


The most complex part of things is when you try to have one side control the 
queues on the other. If you can say that you will always have active queue 
management on both sides of the link, things get much simpler.


David Lang


On Wed, 23 Oct 2019, erik.tarald...@telenor.com wrote:


Date: Wed, 23 Oct 2019 07:28:05 +
From: erik.tarald...@telenor.com
To: chromati...@gmail.com, grob...@icow-systems.com
Cc: bloat@lists.bufferbloat.net
Subject: Re: [Bloat] Bufferbloat on 4G Connexion

If you could influence the 4G vendors to de-bloat their equipment, would you 
recommend BQL, L4S or codel/cake?


-Erik



Fra: Bloat  på vegne av Jonathan Morton 

Sendt: 22. oktober 2019 23:02
Til: Guillaume ROBIER
Kopi: bloat@lists.bufferbloat.net
Emne: Re: [Bloat] Bufferbloat on 4G Connexion


On 11 Oct, 2019, at 5:56 pm, Guillaume ROBIER  wrote:

I am new to this mailing list and I discovered the bufferbloat in December 
2018. I work on 4G routers and the bufferbloat is very present on this type 
of link (4G). I contact you today to find out if people have experimented 
with solutions on this type of link or have configuration suggestions, 
because the classic fq_codel or piece_of_cake and pie do not allow to fix the 
bufferbloat.


This is actually my own situation at home.  My solution is to insert Cake 
shapers on both upstream *and* downstream directions, and adjust their 
bandwidth settings according to variations in available 4G speed.  To do this 
I use an IQrouter, which is basically a TP-Link Archer C7 with custom 
firmware.


One of the difficulties with 4G in particular is that the link capacity varies 
a great deal according to both radio propagation conditions (weather, 
obstructions) and local usage by other subscribers.  That means the right 
bandwidth setting for the small hours of the night, when nobody is awake, will 
leave you with a lot of bloat in the evening, when everyone is both awake and 
home from school/work.  You will need to measure these trends and set up a 
bandwidth schedule accordingly.


- Jonathan Morton ___ Bloat 
mailing list Bloat@lists.bufferbloat.net 
https://lists.bufferbloat.net/listinfo/bloat 
___ Bloat mailing list 
Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Bufferbloat on 4G Connexion

[erik.taraldsen]

If you could influence the 4G vendors to de-bloat their equipment, would you 
recommend BQL, L4S or codel/cake?


-Erik



Fra: Bloat  på vegne av Jonathan Morton 

Sendt: 22. oktober 2019 23:02
Til: Guillaume ROBIER
Kopi: bloat@lists.bufferbloat.net
Emne: Re: [Bloat] Bufferbloat on 4G Connexion

> On 11 Oct, 2019, at 5:56 pm, Guillaume ROBIER  
> wrote:
>
> I am new to this mailing list and I discovered the bufferbloat in December 
> 2018. I work on 4G routers and the bufferbloat is very present on this type 
> of link (4G). I contact you today to find out if people have experimented 
> with solutions on this type of link or have configuration suggestions, 
> because the classic fq_codel or piece_of_cake and pie do not allow to fix the 
> bufferbloat.

This is actually my own situation at home.  My solution is to insert Cake 
shapers on both upstream *and* downstream directions, and adjust their 
bandwidth settings according to variations in available 4G speed.  To do this I 
use an IQrouter, which is basically a TP-Link Archer C7 with custom firmware.

One of the difficulties with 4G in particular is that the link capacity varies 
a great deal according to both radio propagation conditions (weather, 
obstructions) and local usage by other subscribers.  That means the right 
bandwidth setting for the small hours of the night, when nobody is awake, will 
leave you with a lot of bloat in the evening, when everyone is both awake and 
home from school/work.  You will need to measure these trends and set up a 
bandwidth schedule accordingly.

 - Jonathan Morton
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Bufferbloat on 4G Connexion

[Jonathan Morton]

> On 11 Oct, 2019, at 5:56 pm, Guillaume ROBIER  
> wrote:
> 
> I am new to this mailing list and I discovered the bufferbloat in December 
> 2018. I work on 4G routers and the bufferbloat is very present on this type 
> of link (4G). I contact you today to find out if people have experimented 
> with solutions on this type of link or have configuration suggestions, 
> because the classic fq_codel or piece_of_cake and pie do not allow to fix the 
> bufferbloat.

This is actually my own situation at home.  My solution is to insert Cake 
shapers on both upstream *and* downstream directions, and adjust their 
bandwidth settings according to variations in available 4G speed.  To do this I 
use an IQrouter, which is basically a TP-Link Archer C7 with custom firmware.

One of the difficulties with 4G in particular is that the link capacity varies 
a great deal according to both radio propagation conditions (weather, 
obstructions) and local usage by other subscribers.  That means the right 
bandwidth setting for the small hours of the night, when nobody is awake, will 
leave you with a lot of bloat in the evening, when everyone is both awake and 
home from school/work.  You will need to measure these trends and set up a 
bandwidth schedule accordingly.

 - Jonathan Morton
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat