Jose Blanquicet <[email protected]> writes: > Hi Toke, > > Thanks for your reply. > > On Mon, Jun 22, 2020 at 5:47 PM Toke Høiland-Jørgensen <[email protected]> > wrote: >> > We have an embedded system with limited CPU resources that acts as a >> > gateway to provide Internet access from LTE to a private USB-NCM >> > network (And also to a Wi-Fi private network but we will work on it >> > later). Our problem is that the bandwidth on LTE and USB link is >> > higher than what the system is able to handle thus it reaches 100% of >> > CPU load when we perform a simple speed test from a device on the >> > private network. >> >> What speeds were you getting without shaping? > > Between 35 and 40Mbps. > >> > Therefore, we want to limit the bandwidth to avoid system getting >> > saturated in such use-case. To do so, we thought to use the CAKE on >> > the USB interface. For instance, we tried: >> > >> > tc qdisc replace root dev eth0 cake bandwidth 20mbit ethernet >> > internet flowblind nonat besteffort nowash >> > >> > It worked correctly and the maximum rate was limited but there are two >> > things that are worrying us: >> > >> > 1) The maximum rate reached after applying CAKE was in between 12Mbps >> > and 15Mbps which is quite lower than the 20Mbps we are configuring, we >> > were expecting around 18-19. Why? Is there something in the parameters >> > we are doing wrong? Please take into account that our goal is to limit >> > the rate but adding as little CPU load as possible. >> >> Hmm, are you actually running out of CPU? I.e., is the CPU pegged at >> 100% when you hit this limit? What kind of platform are you running on? >> And what kernel and CAKE versions are you using? > > I checked the CPU with top and there is still free CPU to be used. We > also tried with lower values like 10 and it is again far away from the > configured limit. > > We have just a percentage of an ARM Cortex A7 (1.2GHz) because the > rest is reserved for modem. We are now trying to optimize all the > applications in the system but LTE<->WIFI/USB data transfer is indeed > the > use-case that puts our system in crisis. > > The kernel version is 3.18 and for we are using the latest commit on > master branch (9d79e2b) for CAKE. In case, we could change CAKE but > not the kernel version, at most some specific patches.
Right, well if you're not running out of CPU I guess it could be a timing issue. The CAKE shaper relies on accurate timestamps and the qdisc watchdog timer to schedule the transmission of packets. A loaded system can simply miss deadlines, which would be consistent with the behaviour you're seeing. In fact, when looking into this a bit more, I came across this commit that seemed to observe the same behaviour in sch_fq: https://git.kernel.org/torvalds/c/fefa569a9d4b So I guess we could try to do something similar in CAKE. Could you please post the output of 'tc -s qdisc' after a test run? That should give some indication on how much the shaper is throttling... >> > 2) The CPU load added by CAKE was not negligible for our system. In >> > fact, we compared the CPU load when limitation was done by CAKE and by >> > the device on the private network, e.g. curl tool with parameter >> > "--limit-rate". As a result, we found that the CPU load when using >> > CAKE was 30%. Is there any way to make it lighter with a different >> > configuration? >> >> No, you've already turned off most of the features that might incur >> overhead, so I don't think there's anything more you can do >> configuration-wise to improve CPU load. Shaping does tend to use up a >> lot of CPU, so it's not too surprising you run into issues here. > > Could you please help us to identify which one is still active? We > thought we had already turned off all the features not needed to apply > a limitation with a single queue (Besteffor mode). Well the only thing more you can turn off by configuration is the shaper itself :) >> We did recently get a pull request whose author states that he was >> seeing a 1/3 improvement in performance from it. See: >> https://github.com/dtaht/sch_cake/pull/136 >> >> You could try this; if your ingress network device driver has the same >> issue with skbs being allocated in smaller bits, you may see a similar >> increase with this patch. For a quick test you could also just try >> commenting out the call to cake_handle_diffserv() entirely since you're >> running in besteffort mode anyway :) > > Interesting. We will try this, we commented out the call to > cake_handle_diffserv() as you said and just to be sure, we also > applied the 2nd commit of the PR. I will be back soon with news. OK, great! -Toke _______________________________________________ Cake mailing list [email protected] https://lists.bufferbloat.net/listinfo/cake
