On Wed, 2017-09-13 at 09:42 -0700, Eric Dumazet wrote: > On Wed, 2017-09-13 at 19:27 +0300, Denys Fedoryshchenko wrote: > > On 2017-09-13 19:16, Eric Dumazet wrote: > > > On Wed, 2017-09-13 at 18:34 +0300, Denys Fedoryshchenko wrote: > > >> Well, probably i am answering my own question, removing estimator from > > >> classes seems drastically improve situation. > > >> It seems estimator has some issues that cause shaper to behave > > >> incorrectly (throttling traffic while it should not). > > >> But i guess thats a bug? > > >> As i was not able to predict such bottleneck by CPU load measurements. > > > > > > Well, there was a reason we disabled HTB class estimators by default ;) > > > > > > > > > https://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next.git/commit/?id=64153ce0a7b61b2a5cacb01805cbf670142339e9 > > > > As soon as disabling it solve my problem - i'm fine, hehe, but i guess > > other people who might hit this problem, should be aware how to find > > reason. > > They should not be disappointed in Linux :) > > Well, if they enable rate estimators while kernel does not set them by > default, they get what they want, at a cost. > > > Because i can't measure this bottleneck before it happens, i'm seeing on > > mpstat all cpu's are idle, and same time traffic is throttled. > > Normally things were supposed to get much better in linux-4.10 > > ( > https://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next.git/commit/?id=1c0d32fde5bdf1184bc274f864c09799278a1114 > ) > > But I apparently added a scaling bug. > > I will try : > > diff --git a/net/core/gen_estimator.c b/net/core/gen_estimator.c > index > 0385dece1f6fe5e26df1ce5f40956a79a2eebbf4..7c1ffd6f950172c1915d8e5fa2b5e3f77e4f4c78 > 100644 > --- a/net/core/gen_estimator.c > +++ b/net/core/gen_estimator.c > @@ -83,10 +83,10 @@ static void est_timer(unsigned long arg) > u64 rate, brate; > > est_fetch_counters(est, &b); > - brate = (b.bytes - est->last_bytes) << (8 - est->ewma_log); > + brate = (b.bytes - est->last_bytes) << (10 - est->ewma_log - > est->intvl_log); > brate -= (est->avbps >> est->ewma_log); > > - rate = (u64)(b.packets - est->last_packets) << (8 - est->ewma_log); > + rate = (u64)(b.packets - est->last_packets) << (10 - est->ewma_log - > est->intvl_log); > rate -= (est->avpps >> est->ewma_log); > > write_seqcount_begin(&est->seq);
Much better indeed # tc -s -d class sh dev eth0 classid 7002:11 ; sleep 10 ;tc -s -d class sh dev eth0 classid 7002:11 class htb 7002:11 parent 7002:1 prio 5 quantum 200000 rate 5Gbit ceil 5Gbit linklayer ethernet burst 80000b/1 mpu 0b cburst 80000b/1 mpu 0b level 0 rate_handle 1 Sent 389085117074 bytes 256991500 pkt (dropped 0, overlimits 5926926 requeues 0) rate 4999Mbit 412762pps backlog 136260b 2p requeues 0 TCP pkts/rtx 256991584/0 bytes 389085252840/0 lended: 5961250 borrowed: 0 giants: 0 tokens: -1664 ctokens: -1664 class htb 7002:11 parent 7002:1 prio 5 quantum 200000 rate 5Gbit ceil 5Gbit linklayer ethernet burst 80000b/1 mpu 0b cburst 80000b/1 mpu 0b level 0 rate_handle 1 Sent 395336315580 bytes 261120429 pkt (dropped 0, overlimits 6021776 requeues 0) rate 4999Mbit 412788pps backlog 68Kb 2p requeues 0 TCP pkts/rtx 261120469/0 bytes 395336384730/0 lended: 6056793 borrowed: 0 giants: 0 tokens: -1478 ctokens: -1478 echo "(395336315580-389085117074)/10*8" | bc 5000958800