Re: Performance comparision / qdisc / no qdisc / queueing linux kernel 4.12

2017-08-12 Thread Tom Herbert 
On Fri, Aug 11, 2017 at 4:16 PM, Stephen Hemminger
 wrote:
> Any of the current qdisc that do rate control are single threaded.
> Since bandwidth is a shared resource, the qdisc has to acquire a lock
> and update cache shared data structures.
>
> Don't expect HTB or HFSC to be free.
>
This is a well known problem and there have been several attempts to
address it over the years (a long time ago at Google I implemented a
horrible hack called "hold queue" that put a parallelized queue in
front of HTB. This eliminated the lock contention, but not without
side effects).

Now there's a new scheduler proposed by some Googlers called Carousel
that promises scalable host traffic shaping
(https://www.cc.gatech.edu/~amsmti3/files/carousel-sigcomm17.pdf).
Looks like they didn't implement a kernel Qdisc, but might be
interesting for someone to look into...

Tom

>>
>> 4.13-rc3 have kernel panic when deleting qdisc - will try to get panic trace
>
> Please do.
>

Re: Performance comparision / qdisc / no qdisc / queueing linux kernel 4.12

2017-08-11 Thread Stephen Hemminger 
Any of the current qdisc that do rate control are single threaded.
Since bandwidth is a shared resource, the qdisc has to acquire a lock
and update cache shared data structures.

Don't expect HTB or HFSC to be free.

> 
> 4.13-rc3 have kernel panic when deleting qdisc - will try to get panic trace

Please do.

Re: Performance comparision / qdisc / no qdisc / queueing linux kernel 4.12

2017-08-05 Thread Paweł Staszewski 

With

4.13-rc3 have kernel panic when deleting qdisc - will try to get panic trace


W dniu 2017-08-05 o 18:38, Paweł Staszewski pisze:

Hi


Want to send some comparisions about poor performance of kernel when 
simple queueing is enabled



So first - kernel performance for forwarded udp packets generated from 
pktgen:


UDP stream ~10Mpps - random destination from network 172.16.0.0/12 + 
random udp port



Forwarding performance with 8x RSS queues

  enp216s0f0:  9772859.00 P/s 0.00 P/s 9772859.00 P/s
   enp216s0f1:0.00 P/s   9772883.00 P/s 9772883.00 
P/s
-- 

total:  9772863.00 P/s   9772886.00 P/s 
19545750.00 P/s


mpstat -P ALL 1 10

Average: CPU%usr   %nice%sys %iowait%irq   %soft 
%steal  %guest  %gnice   %idle
Average: all0.000.000.000.000.00 11.89 0.00
0.000.00   88.11
Average:   00.000.000.000.000.00 0.10 0.00
0.000.00   99.90
Average:   10.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:   20.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:   30.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:   40.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:   50.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:   60.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:   70.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:   80.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:   90.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  100.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  110.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  120.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  130.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  140.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  150.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  160.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  170.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  180.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  190.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  200.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  210.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  220.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  230.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  240.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  250.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  260.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  270.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  280.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  290.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  300.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  310.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  320.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  330.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  340.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  350.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  360.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  370.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  380.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  390.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  400.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  410.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  420.000.000.000.000.00 0.00 0.00
0.000.00  100.00
Average:  430.000.000.000.000.00 0.00 0.00

Performance comparision / qdisc / no qdisc / queueing linux kernel 4.12

2017-08-05 Thread Paweł Staszewski 

Hi


Want to send some comparisions about poor performance of kernel when 
simple queueing is enabled



So first - kernel performance for forwarded udp packets generated from 
pktgen:


UDP stream ~10Mpps - random destination from network 172.16.0.0/12 + 
random udp port



Forwarding performance with 8x RSS queues

  enp216s0f0:  9772859.00 P/s 0.00 P/s 9772859.00 P/s
   enp216s0f1:0.00 P/s   9772883.00 P/s 9772883.00 P/s
--
total:  9772863.00 P/s   9772886.00 P/s 19545750.00 P/s

mpstat -P ALL 1 10

Average: CPU%usr   %nice%sys %iowait%irq   %soft %steal  
%guest  %gnice   %idle
Average: all0.000.000.000.000.00 11.89
0.000.000.00   88.11
Average:   00.000.000.000.000.00 0.100.00
0.000.00   99.90
Average:   10.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:   20.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:   30.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:   40.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:   50.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:   60.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:   70.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:   80.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:   90.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  100.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  110.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  120.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  130.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  140.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  150.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  160.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  170.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  180.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  190.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  200.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  210.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  220.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  230.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  240.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  250.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  260.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  270.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  280.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  290.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  300.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  310.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  320.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  330.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  340.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  350.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  360.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  370.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  380.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  390.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  400.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  410.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  420.000.000.000.000.00 0.000.00
0.000.00  100.00
Average:  430.000.000.000.000.00 0.000.00
0.