Re: [Bloat] CAKE in openwrt high CPU

2020-09-04 Thread Sebastian Moeller
Hi Mikael, Thanks! That looks like a fully saturated core, no? I do not know how to parse the symbols here, so not sure what "class" of load is denoted by the star, but I would guess something including sirqs? Anyway the average is ~49% load, while clearly CPU is pegged already. I assume the ht

Re: [Bloat] CAKE in openwrt high CPU

2020-09-04 Thread Mikael Abrahamsson via Bloat
On Thu, 3 Sep 2020, Sebastian Moeller wrote: Mmmh, how did you measure the sirq percentage? Some top versions show overall percentage with 100% meaning all CPUs, so 35% in a quadcore could mean 1 fully maxed out CPU (25%) plus an additional 10% spread over the other three, or something more b

Re: [Bloat] CAKE in openwrt high CPU

2020-09-03 Thread Jonathan Morton
> On 3 Sep, 2020, at 5:32 pm, Toke Høiland-Jørgensen via Bloat > wrote: > > Yeah, offloading of some sort is another option, but I consider that > outside of the "CAKE stays relevant" territory, since that will most > likely involve an entirely programmable packet scheduler. Offload of *just* s

Re: [Bloat] CAKE in openwrt high CPU

2020-09-03 Thread Toke Høiland-Jørgensen via Bloat
On 3 September 2020 17:31:07 CEST, Luca Muscariello wrote: >On Thu, Sep 3, 2020 at 4:32 PM Toke Høiland-Jørgensen >wrote: >> >> Luca Muscariello writes: >> >> > On Thu, Sep 3, 2020 at 3:19 PM Mikael Abrahamsson via Bloat >> > wrote: >> >> >> >> On Tue, 1 Sep 2020, Toke Høiland-Jørgensen wrot

Re: [Bloat] CAKE in openwrt high CPU

2020-09-03 Thread Luca Muscariello
On Thu, Sep 3, 2020 at 4:32 PM Toke Høiland-Jørgensen wrote: > > Luca Muscariello writes: > > > On Thu, Sep 3, 2020 at 3:19 PM Mikael Abrahamsson via Bloat > > wrote: > >> > >> On Tue, 1 Sep 2020, Toke Høiland-Jørgensen wrote: > >> > >> > Yup, the number of cores is only going to go up, so for C

Re: [Bloat] CAKE in openwrt high CPU

2020-09-03 Thread Toke Høiland-Jørgensen via Bloat
Luca Muscariello writes: > On Thu, Sep 3, 2020 at 3:19 PM Mikael Abrahamsson via Bloat > wrote: >> >> On Tue, 1 Sep 2020, Toke Høiland-Jørgensen wrote: >> >> > Yup, the number of cores is only going to go up, so for CAKE to stay >> > relevant it'll need to be able to take advantage of this event

Re: [Bloat] CAKE in openwrt high CPU

2020-09-03 Thread Sebastian Moeller
Ho Toke, > On Sep 3, 2020, at 15:29, Toke Høiland-Jørgensen via Bloat > wrote: > > Mikael Abrahamsson writes: > >> On Mon, 31 Aug 2020, Toke Høiland-Jørgensen wrote: >> >>> And what about when you're running CAKE in 'unlimited' mode? >> >> I tried this: >> >> # tc qdisc add dev eth0 root c

Re: [Bloat] CAKE in openwrt high CPU

2020-09-03 Thread Sebastian Moeller
Hi Mikael, > On Sep 3, 2020, at 15:10, Mikael Abrahamsson via Bloat > wrote: > > On Mon, 31 Aug 2020, Toke Høiland-Jørgensen wrote: > >> And what about when you're running CAKE in 'unlimited' mode? > > I tried this: > > # tc qdisc add dev eth0 root cake bandwidth 900mbit That stil

Re: [Bloat] CAKE in openwrt high CPU

2020-09-03 Thread Toke Høiland-Jørgensen via Bloat
Mikael Abrahamsson writes: > On Mon, 31 Aug 2020, Toke Høiland-Jørgensen wrote: > >> And what about when you're running CAKE in 'unlimited' mode? > > I tried this: > > # tc qdisc add dev eth0 root cake bandwidth 900mbit So the difference from before is just the lack of inbound shaping, or? -Tok

Re: [Bloat] CAKE in openwrt high CPU

2020-09-03 Thread Toke Høiland-Jørgensen via Bloat
Mikael Abrahamsson writes: > On Tue, 1 Sep 2020, Toke Høiland-Jørgensen wrote: > >> Yup, the number of cores is only going to go up, so for CAKE to stay >> relevant it'll need to be able to take advantage of this eventually :) > > https://www.hardkernel.com/shop/odroid-h2plus/ is an interesting

Re: [Bloat] CAKE in openwrt high CPU

2020-09-03 Thread Luca Muscariello
On Thu, Sep 3, 2020 at 3:19 PM Mikael Abrahamsson via Bloat wrote: > > On Tue, 1 Sep 2020, Toke Høiland-Jørgensen wrote: > > > Yup, the number of cores is only going to go up, so for CAKE to stay > > relevant it'll need to be able to take advantage of this eventually :) > > https://www.hardkernel.

Re: [Bloat] CAKE in openwrt high CPU

2020-09-03 Thread Mikael Abrahamsson via Bloat
On Tue, 1 Sep 2020, Toke Høiland-Jørgensen wrote: Yup, the number of cores is only going to go up, so for CAKE to stay relevant it'll need to be able to take advantage of this eventually :) https://www.hardkernel.com/shop/odroid-h2plus/ is an interesting platform, it has a quad core machine w

Re: [Bloat] CAKE in openwrt high CPU

2020-09-03 Thread Mikael Abrahamsson via Bloat
On Mon, 31 Aug 2020, Toke Høiland-Jørgensen wrote: And what about when you're running CAKE in 'unlimited' mode? I tried this: # tc qdisc add dev eth0 root cake bandwidth 900mbit This seems fine from a performance point of view (not that high sirq%, around 35%) and does seem to limit my upst

Re: [Bloat] CAKE in openwrt high CPU

2020-09-02 Thread Toke Høiland-Jørgensen via Bloat
Jonathan Foulkes writes: >> Right, so some benefit might be possible here. Does the NIC have >> multiple hardware queues (`ls /sys/class/net/$IFACE/queues` should tell >> you)? > > Here is the output of: > /sys/devices/virtual/net/eth0.2/queues# ls > rx-0 tx-0 > /sys/devices/virtual/net/eth0.2/q

Re: [Bloat] CAKE in openwrt high CPU

2020-09-02 Thread Jonathan Foulkes
> Right, so some benefit might be possible here. Does the NIC have > multiple hardware queues (`ls /sys/class/net/$IFACE/queues` should tell > you)? Here is the output of: /sys/devices/virtual/net/eth0.2/queues# ls rx-0 tx-0 /sys/devices/virtual/net/eth0.2/queues/rx-0# cat rps_cpus 0 /sys/devic

Re: [Bloat] CAKE in openwrt high CPU

2020-09-01 Thread Toke Høiland-Jørgensen via Bloat
Jonathan Foulkes writes: > Thanks Toke, we currently are on an MT7621a @880, so a dual-core. Right, so some benefit might be possible here. Does the NIC have multiple hardware queues (`ls /sys/class/net/$IFACE/queues` should tell you)? > And we are looking for a good quad-core platform that wil

Re: [Bloat] CAKE in openwrt high CPU

2020-09-01 Thread Toke Høiland-Jørgensen via Bloat
Jonathan Morton writes: >> On 1 Sep, 2020, at 9:45 pm, Toke Høiland-Jørgensen via Bloat >> wrote: >> >> CAKE takes the global qdisc lock. > > Presumably this is a default mechanism because CAKE doesn't handle any > locking itself. > > Obviously it would need to be replaced with at least a lock

Re: [Bloat] CAKE in openwrt high CPU

2020-09-01 Thread Jonathan Morton
> On 1 Sep, 2020, at 11:04 pm, Sebastian Moeller wrote: > >> The challenge are the end users, who only understand the silly ’speed’ >> metric, and feel anything that lowers that number is a ‘bad’ thing. It takes >> effort to get even technical users to get it. > > I repeatedly fall into

Re: [Bloat] CAKE in openwrt high CPU

2020-09-01 Thread Sebastian Moeller
Hi Jonathan, > On Sep 1, 2020, at 21:31, Jonathan Foulkes wrote: > > Hi Sebastian, Cake functions wonderfully, it’s a marvel in terms of goodput. > > My comment was more oriented at the metrics process users use to evaluate > results. Only those who spend time analyzing just how busy an ‘idl

Re: [Bloat] CAKE in openwrt high CPU

2020-09-01 Thread Jonathan Foulkes
Hi Sebastian, Cake functions wonderfully, it’s a marvel in terms of goodput. My comment was more oriented at the metrics process users use to evaluate results. Only those who spend time analyzing just how busy an ‘idle’ network can be know that there are a lot of processes in constant communicat

Re: [Bloat] CAKE in openwrt high CPU

2020-09-01 Thread Jonathan Morton
> On 1 Sep, 2020, at 9:45 pm, Toke Høiland-Jørgensen via Bloat > wrote: > > CAKE takes the global qdisc lock. Presumably this is a default mechanism because CAKE doesn't handle any locking itself. Obviously it would need to be replaced with at least a lock over CAKE's complete data structure

Re: [Bloat] CAKE in openwrt high CPU

2020-09-01 Thread Jonathan Foulkes
Thanks Toke, we currently are on an MT7621a @880, so a dual-core. And we are looking for a good quad-core platform that will support 600Mbps or more with Cake enabled, hopefully with AX radios as well. Jonathan > On Sep 1, 2020, at 12:11 PM, Toke Høiland-Jørgensen wrote: > > Jonathan Foulkes

Re: [Bloat] CAKE in openwrt high CPU

2020-09-01 Thread Toke Høiland-Jørgensen via Bloat
Sebastian Moeller writes: > Hi Toke, > > >> On Sep 1, 2020, at 18:11, Toke Høiland-Jørgensen via Bloat >> wrote: >> >> Jonathan Foulkes writes: >> >>> Toke, that link returns a 404 for me. >> >> Ah, seems an extra character snuck in at the end - try this: >> >> https://github.com/dtaht/sch

Re: [Bloat] CAKE in openwrt high CPU

2020-09-01 Thread Sebastian Moeller
Hi Toke, > On Sep 1, 2020, at 18:11, Toke Høiland-Jørgensen via Bloat > wrote: > > Jonathan Foulkes writes: > >> Toke, that link returns a 404 for me. > > Ah, seems an extra character snuck in at the end - try this: > > https://github.com/dtaht/sch_cake/commit/3152477235c934022049fcddc063c

Re: [Bloat] CAKE in openwrt high CPU

2020-09-01 Thread Sebastian Moeller
HI Jonathan, > On Sep 1, 2020, at 17:41, Jonathan Foulkes wrote: > > Toke, that link returns a 404 for me. > > For others, I’ve found that testing cake throughput with isolation options > enabled is tricky if there are many competing connections. Are you talking about the fact that w

Re: [Bloat] CAKE in openwrt high CPU

2020-09-01 Thread Toke Høiland-Jørgensen via Bloat
Jonathan Foulkes writes: > Toke, that link returns a 404 for me. Ah, seems an extra character snuck in at the end - try this: https://github.com/dtaht/sch_cake/commit/3152477235c934022049fcddc063c45d37ec10e6 > For others, I’ve found that testing cake throughput with isolation options > enable

Re: [Bloat] CAKE in openwrt high CPU

2020-09-01 Thread Jonathan Foulkes
Toke, that link returns a 404 for me. For others, I’ve found that testing cake throughput with isolation options enabled is tricky if there are many competing connections. Like I keep having to tell my customers, fairness algorithms mean no one device will ever gain 100% of the bandwidth so lon

Re: [Bloat] CAKE in openwrt high CPU

2020-08-31 Thread Toke Høiland-Jørgensen via Bloat
Mikael Abrahamsson writes: > On Mon, 31 Aug 2020, Toke Høiland-Jørgensen wrote: > >> Hmm, you say CAKE and FQ-Codel - so you're not enabling the shaper (that >> would be FQ-CoDel+HTB)? An exact config might be useful (or just the >> output of tc -s qdisc). > > Yeah, I guess I'm also using HTB to

Re: [Bloat] CAKE in openwrt high CPU

2020-08-31 Thread Mikael Abrahamsson via Bloat
On Mon, 31 Aug 2020, Toke Høiland-Jørgensen wrote: Hmm, you say CAKE and FQ-Codel - so you're not enabling the shaper (that would be FQ-CoDel+HTB)? An exact config might be useful (or just the output of tc -s qdisc). Yeah, I guess I'm also using HTB to get the 900 megabit/s SQM is looking for

Re: [Bloat] CAKE in openwrt high CPU

2020-08-31 Thread Toke Høiland-Jørgensen via Bloat
Mikael Abrahamsson via Bloat writes: > Hi, > > I migrated to an APU2 (https://www.pcengines.ch/apu2.htm) as residential > router, from my previous WRT1200AC (marvell armada 385). > > I was running OpenWrt 18.06 on that one, now I am running latest 19.07.3 > on the APU2. > > Before I had 500/100

Re: [Bloat] CAKE in openwrt high CPU

2020-08-30 Thread Dave Taht
cake reschedules too much compared to the tweaks we have to keep htb fed, at these rates. It was kind of my hope to gain a hw assist in future versions of the apu series. a programmable completion interrupt is available in some versions of that chipset, On Sun, Aug 30, 2020 at 10:27 AM Mikael Abr

[Bloat] CAKE in openwrt high CPU

2020-08-30 Thread Mikael Abrahamsson via Bloat
Hi, I migrated to an APU2 (https://www.pcengines.ch/apu2.htm) as residential router, from my previous WRT1200AC (marvell armada 385). I was running OpenWrt 18.06 on that one, now I am running latest 19.07.3 on the APU2. Before I had 500/100 and I had to use FQ_CODEL because CAKE took too m