On Sun, Dec 16 2018, Florian Fainelli wrote: > On December 16, 2018 3:19:22 PM PST, NeilBrown <n...@brown.name> wrote: >>On Sun, Dec 16 2018, David Miller wrote: >> >>> From: NeilBrown <n...@brown.name> >>> Date: Mon, 17 Dec 2018 09:08:54 +1100 >>> >>>> In my 4.4 kernel, the build_skb() call in (the equivalent of) >>>> mtk_poll_rx() takes about 1.2usec and the call to napi_gro_receive() >>>> takes about 3usec. >>>> >>>> In my 4.20 kernel, these calls take about 30 and 24 usec >>respectively. >>>> This easily explains the slowdown. >>> >>> That's a huge difference. >>> >>> Nothing jumps out as a possible cause except perhaps retpoline or >>> something like that. >> >>I'll keep that in mind - thanks. >> >>My guess was CPU-cache invalidation. >>I just checked and the other CPU core (there are two - each >>hyper-threaded - "other" meaning not the one that handles ethernet >>interrupts) gets several thousand "IPI resched" interrupts while >>running a 10 second (226MByte) iperf3 receive test. >>About 17KB transferred per IPI. >>I cannot see where build_skb() would do cache invalidation though. > > It doesn't the driver is responsible for that. How is coherency maintained > between cores?
I suspect so - yes. Coherency only needs explicit management with DMA is used. This wasn't the problem. Further investigation showed that the problem was that I had CONFIG_SLUB_DEBUG set. That was probably useful in some earlier debugging exercise, but it clearly isn't useful when performance-testing the network. I removed that and I have much nicer numbers - not quite the consistent 900+ that I saw with 4.4, but a lot closer. Thanks for the encouragement, and sorry of the noise. NeilBrown > > The IPI could be due to receive packet steering, is the MAC multi queue aware > on the RX path? > -- > Florian
signature.asc
Description: PGP signature
