On 2020-12-07 21:34, Alex Kiselev wrote:
On 2020-12-07 20:29, Singh, Jasvinder wrote:On 7 Dec 2020, at 19:09, Alex Kiselev <[email protected]> wrote:On 2020-12-07 20:07, Alex Kiselev wrote:Ok, these new two tests show even more clearly that TC_OV feature is broken.On 2020-12-07 19:18, Alex Kiselev wrote: On 2020-12-07 18:59, Singh, Jasvinder wrote:On 7 Dec 2020, at 17:45, Alex Kiselev <[email protected]> wrote: On 2020-12-07 18:31, Singh, Jasvinder wrote:-----Original Message----- From: Alex Kiselev <[email protected]> Sent: Monday, December 7, 2020 4:50 PM To: Singh, Jasvinder <[email protected]>Cc: [email protected]; Dumitrescu, Cristian <[email protected]>;Dharmappa, Savinay <[email protected]> Subject: Re: [dpdk-users] scheduler issueOn 2020-12-07 12:32, Singh, Jasvinder wrote:>> -----Original Message----- >> From: Alex Kiselev <[email protected]> >> Sent: Monday, December 7, 2020 10:46 AM >> To: Singh, Jasvinder <[email protected]> >> Cc: [email protected]; Dumitrescu, Cristian >> <[email protected]>; Dharmappa, Savinay >> <[email protected]> >> Subject: Re: [dpdk-users] scheduler issue >> >> On 2020-12-07 11:00, Singh, Jasvinder wrote: >> >> -----Original Message----- >> >> From: users <[email protected]> On Behalf Of Alex Kiselev >> >> Sent: Friday, November 27, 2020 12:12 PM >> >> To: [email protected] >> >> Cc: Dumitrescu, Cristian <[email protected]> >> >> Subject: Re: [dpdk-users] scheduler issue >> >> >> >> On 2020-11-25 16:04, Alex Kiselev wrote: >> >> > On 2020-11-24 16:34, Alex Kiselev wrote: >> >> >> Hello, >> >> >> >> >> >> I am facing a problem with the scheduler library DPDK 18.11.10 >> >> >> with default scheduler settings (RED is off). >> >> >> It seems like some of the pipes (last time it was 4 out of 600 >> >> >> pipes) start incorrectly dropping most of the traffic after a >> >> >> couple of days of successful work. >> >> >> >> >> >> So far I've checked that there are no mbuf leaks or any other >> >> >> errors in my code and I am sure that traffic enters problematic pipes. >> >> >> Also switching a traffic in the runtime to pipes of another >> >> >> port restores the traffic flow. >> >> >> >> >> >> Ho do I approach debugging this issue? >> >> >> >> >> >> I've added using rte_sched_queue_read_stats(), but it doesn't >> >> >> give me counters that accumulate values (packet drops for >> >> >> example), it gives me some kind of current values and after a >> >> >> couple of seconds those values are reset to zero, so I can say nothing based on that API. >> >> >> >> >> >> I would appreciate any ideas and help. >> >> >> Thanks. >> >> > >> >> > Problematic pipes had very low bandwidth limit (1 Mbit/s) and >> >> > also there is an oversubscription configuration event at subport >> >> > 0 of port >> >> > 13 to which those pipes belongs and >> >> CONFIG_RTE_SCHED_SUBPORT_TC_OV is >> >> > disabled. >> >> > >> >> > Could a congestion at that subport be the reason of the problem? >> >> > >> >> > How much overhead and performance degradation will add enabling >> >> > CONFIG_RTE_SCHED_SUBPORT_TC_OV feature? >> >> > >> >> > Configuration: >> >> > >> >> > # >> >> > # QoS Scheduler Profiles >> >> > # >> >> > hqos add profile 1 rate 8 K size 1000000 tc period 40 >> >> > hqos add profile 2 rate 400 K size 1000000 tc period 40 >> >> > hqos add profile 3 rate 600 K size 1000000 tc period 40 >> >> > hqos add profile 4 rate 800 K size 1000000 tc period 40 >> >> > hqos add profile 5 rate 1 M size 1000000 tc period 40 >> >> > hqos add profile 6 rate 1500 K size 1000000 tc period 40 >> >> > hqos add profile 7 rate 2 M size 1000000 tc period 40 >> >> > hqos add profile 8 rate 3 M size 1000000 tc period 40 >> >> > hqos add profile 9 rate 4 M size 1000000 tc period 40 >> >> > hqos add profile 10 rate 5 M size 1000000 tc period 40 >> >> > hqos add profile 11 rate 6 M size 1000000 tc period 40 >> >> > hqos add profile 12 rate 8 M size 1000000 tc period 40 >> >> > hqos add profile 13 rate 10 M size 1000000 tc period 40 >> >> > hqos add profile 14 rate 12 M size 1000000 tc period 40 >> >> > hqos add profile 15 rate 15 M size 1000000 tc period 40 >> >> > hqos add profile 16 rate 16 M size 1000000 tc period 40 >> >> > hqos add profile 17 rate 20 M size 1000000 tc period 40 >> >> > hqos add profile 18 rate 30 M size 1000000 tc period 40 >> >> > hqos add profile 19 rate 32 M size 1000000 tc period 40 >> >> > hqos add profile 20 rate 40 M size 1000000 tc period 40 >> >> > hqos add profile 21 rate 50 M size 1000000 tc period 40 >> >> > hqos add profile 22 rate 60 M size 1000000 tc period 40 >> >> > hqos add profile 23 rate 100 M size 1000000 tc period 40 >> >> > hqos add profile 24 rate 25 M size 1000000 tc period 40 >> >> > hqos add profile 25 rate 50 M size 1000000 tc period 40 >> >> > >> >> > # >> >> > # Port 13 >> >> > # >> >> > hqos add port 13 rate 40 G mtu 1522 frame overhead 24 queue >> >> > sizes >> >> > 64 >> >> > 64 64 64 >> >> > hqos add port 13 subport 0 rate 1500 M size 1000000 tc period 10 >> >> > hqos add port 13 subport 0 pipes 3000 profile 2 >> >> > hqos add port 13 subport 0 pipes 3000 profile 5 >> >> > hqos add port 13 subport 0 pipes 3000 profile 6 >> >> > hqos add port 13 subport 0 pipes 3000 profile 7 >> >> > hqos add port 13 subport 0 pipes 3000 profile 9 >> >> > hqos add port 13 subport 0 pipes 3000 profile 11 >> >> > hqos set port 13 lcore 5 >> >> >> >> I've enabled TC_OV feature and redirected most of the traffic to TC3. >> >> But the issue still exists. >> >> >> >> Below is queue statistics of one of problematic pipes. >> >> Almost all of the traffic entering the pipe is dropped. >> >> >> >> And the pipe is also configured with the 1Mbit/s profile. >> >> So, the issue is only with very low bandwidth pipe profiles. >> >> >> >> And this time there was no congestion on the subport. >> >> >> >> >> >> Egress qdisc >> >> dir 0 >> >> rate 1M >> >> port 6, subport 0, pipe_id 138, profile_id 5 >> >> tc 0, queue 0: bytes 752, bytes dropped 0, pkts 8, pkts dropped 0 >> >> tc 0, queue 1: bytes 0, bytes dropped 0, pkts 0, pkts dropped 0 >> >> tc 0, queue 2: bytes 0, bytes dropped 0, pkts 0, pkts dropped 0 >> >> tc 0, queue 3: bytes 0, bytes dropped 0, pkts 0, pkts dropped 0 >> >> tc 1, queue 0: bytes 0, bytes dropped 0, pkts 0, pkts dropped 0 >> >> tc 1, queue 1: bytes 0, bytes dropped 0, pkts 0, pkts dropped 0 >> >> tc 1, queue 2: bytes 0, bytes dropped 0, pkts 0, pkts dropped 0 >> >> tc 1, queue 3: bytes 0, bytes dropped 0, pkts 0, pkts dropped 0 >> >> tc 2, queue 0: bytes 0, bytes dropped 0, pkts 0, pkts dropped 0 >> >> tc 2, queue 1: bytes 0, bytes dropped 0, pkts 0, pkts dropped 0 >> >> tc 2, queue 2: bytes 0, bytes dropped 0, pkts 0, pkts dropped 0 >> >> tc 2, queue 3: bytes 0, bytes dropped 0, pkts 0, pkts dropped 0 >> >> tc 3, queue 0: bytes 56669, bytes dropped 360242, pkts 150, >> >> pkts dropped >> >> 3749 >> >> tc 3, queue 1: bytes 63005, bytes dropped 648782, pkts 150, >> >> pkts dropped >> >> 3164 >> >> tc 3, queue 2: bytes 9984, bytes dropped 49704, pkts 128, pkts >> >> dropped >> >> 636 >> >> tc 3, queue 3: bytes 15436, bytes dropped 107198, pkts 130, >> >> pkts dropped >> >> 354 >> > >> > >> > Hi Alex, >> > >> > Can you try newer version of the library, say dpdk 20.11? >> >> Right now no, since switching to another DPDK will take a lot of time >> because I am using a lot of custom patches. >> >> I've tried to simply copy the entire rte_sched lib from DPDK 19 to >> DPDK 18. >> And I was able to successful back port and resolve all dependency >> issues, but it also will take some time to test this approach. >> >> >> > Are you >> > using dpdk qos sample app or your own app? >> >> My own app. >> >> >> What are the packets size? >> >> Application is used as BRAS/BNG server, so it's used to provide >> internet access to residential customers. Therefore packet sizes are >> typical to the internet and vary from 64 to 1500 bytes. Most of the >> packets are around >> 1000 bytes. >> >> > >> > Couple of other things for clarification- 1. At what rate you are >> > injecting the traffic to low bandwidth pipes? >> >> Well, the rate vary also, there could be congestion on some pipes at >> some date time. >> >> But the problem is that once the problem occurs at a pipe or at some >> queues inside the pipe, the pipe stops transmitting even when >> incoming traffic rate is much lower than the pipe's rate. >> >> > 2. How is traffic distributed among pipes and their traffic class? >> >> I am using IPv4 TOS field to choose the TC and there is a tos2tc map. >> Most of my traffic has 0 tos value which is mapped to TC3 inside my >> app. >> >> Recently I've switched to a tos2map which maps all traffic to TC3 to >> see if it solves the problem. >> >> Packet distribution to queues is done using the formula (ipv4.src + >> ipv4.dst) & 3 >> >> > 3. Can you try putting your own counters on those pipes queues >> > which periodically show the #packets in the queues to understand >> > the dynamics? >> >> I will try. >> >> P.S. >> >> Recently I've got another problem with scheduler. >> >> After enabling the TC_OV feature one of the ports stops transmitting. >> All port's pipes were affected. >> Port had only one support, and there were only pipes with 1 Mbit/s >> profile. >> The problem was solved by adding a 10Mit/s profile to that port. Only >> after that port's pipes started to transmit. >> I guess it has something to do with calculating tc_ov_wm as it >> depends on the maximum pipe rate. >> >> I am gonna make a test lab and a test build to reproduce this.I've made some tests and was able to reproduce the port configuration issueusing a test build of my app.Tests showed that TC_OV feature works not correctly in DPDK 18.11, butthere are workarounds.I still can't reproduce my main problem which is random pipes stoptransmitting. Here are details: All tests use the same test traffic generator that produce10 traffic flows entering 10 different pipes of port 1 subport 0.Only queue 0 of each pipe is used. TX rate is 800 kbit/s. packet size is 800 byte. Pipes rate are 1 Mbit/s. Subport 0 rate is 500 Mbit/s. ### ### test 1 ### Traffic generator is configured to use TC3. Configuration: hqos add profile 27 rate 1 M size 1000000 tc period 40 hqos add profile 23 rate 100 M size 1000000 tc period 40 # qos test porthqos add port 1 rate 10 G mtu 1522 frame overhead 24 queue sizes 64 6464 64 hqos add port 1 subport 0 rate 500 M size 1000000 tc period 10 hqos add port 1 subport 0 pipes 2000 profile 27 hqos set port 1 lcore 3 Results: h5 ~ # rcli sh qos rcvrcv 0: rx rate 641280, nb pkts 501, ind 1 rcv 1: rx rate 641280, nb pkts 501, ind 1 rcv 2: rx rate 641280, nb pkts 501, ind 1 rcv 3: rx rate 641280, nb pkts 501, ind 1 rcv 4: rx rate 641280, nb pkts 501, ind 1 rcv 5: rx rate 641280, nb pkts 501, ind 1 rcv 6: rx rate 641280, nb pkts 501, ind 1 rcv 7: rx rate 641280, nb pkts 501, ind 1 rcv 8: rx rate 641280, nb pkts 501, ind 1 rcv 9: rx rate 641280,nb pkts 501, ind 1 ! BUG! RX rate is lower then expected 800000 bit/s despite that there is nocongestion neither at subport nor at pipes levels.[JS] - Can you elaborate on your scheduler hierarchy?sure, take a look below at the output "number of pipes per subport" TR application always round the total number of pipes per port to a power2 value. I mean- howmany pipes per subport? It has to be the number that can be expressed as power of 2, for e.g 4K, 2K, 1K etc. In run time, scheduler will scan all the pipes and will process only those which have got packetsin their queue.Configuration of port 1 with enabled profile 23 h5 ~ # rcli sh hqos ports hqos scheduler port: 1 lcore_id: 3 socket: 0 rate: 0 mtu: 1522 frame overhead: 24 number of pipes per subport: 4096 pipe profiles: 2 pipe profile id: 27 pipe rate: 1000000 number of pipes: 2000 pipe pool size: 2000 number of pipes in use: 0 pipe profile id: 23 pipe rate: 100000000 number of pipes: 200 pipe pool size: 200 number of pipes in use: 0 Configuration with only one profile at port 1 hqos scheduler port: 1 lcore_id: 3 socket: 0 rate: 0 mtu: 1522 frame overhead: 24 number of pipes per subport: 2048 pipe profiles: 1 pipe profile id: 27 pipe rate: 1000000 number of pipes: 2000 pipe pool size: 2000 number of pipes in use: 0[JS] what is the meaning of number of pipes , Pipe pool size, and number of pipes in use which is zero above? Does your application map packet field values to these number of pipes in run time ? Can you give me example of mapping of packet field values to pipe id, tc, queue?please, ignore all information from the outputs above except the "number of pipes per subport". since the tests were made with a simple test application which is based on TR but doesn't use it's production QoS logic.The tests 1 - 5 were made with a simple test application with a verystraitforward qos mappings which I described at the beginning. Here they are: All tests use the same test traffic generator that produce10 traffic flows entering 10 different pipes (0 - 9) of port 1 subport 0.Only queue 0 of each pipe is used. TX rate is 800 kbit/s. packet size is 800 byte. Pipes rate are 1 Mbit/s. Subport 0 rate is 500 Mbit/s.### ### test 2 ### Traffic generator is configured to use TC3. !!! profile 23 has been added to the test port. Configuration: hqos add profile 27 rate 1 M size 1000000 tc period 40 hqos add profile 23 rate 100 M size 1000000 tc period 40 # qos test porthqos add port 1 rate 10 G mtu 1522 frame overhead 24 queue sizes 64 6464 64 hqos add port 1 subport 0 rate 500 M size 1000000 tc period 10 hqos add port 1 subport 0 pipes 2000 profile 27 hqos add port 1 subport 0 pipes 200 profile 23 hqos set port 1 lcore 3 Results: h5 ~ # rcli sh qos rcvrcv 0: rx rate 798720, nb pkts 624, ind 1 rcv 1: rx rate 798720, nb pkts 624, ind 1 rcv 2: rx rate 798720, nb pkts 624, ind 1 rcv 3: rx rate 798720, nb pkts 624, ind 1 rcv 4: rx rate 798720, nb pkts 624, ind 1 rcv 5: rx rate 798720, nb pkts 624, ind 1 rcv 6: rx rate 798720, nb pkts 624, ind 1 rcv 7: rx rate 798720, nb pkts 624, ind 1 rcv 8: rx rate 798720, nb pkts 624, ind 1 rcv 9: rx rate 798720,nb pkts 624, ind 1 OK. Receiving traffic is rate is equal to expected values.So, just adding a pipes which are not being used solves the problem.### ### test 3 ###!!! traffic generator uses TC 0, so tc_ov is not being used in this test.profile 23 is not used. Configuration without profile 23. hqos add profile 27 rate 1 M size 1000000 tc period 40 hqos add profile 23 rate 100 M size 1000000 tc period 40 # qos test porthqos add port 1 rate 10 G mtu 1522 frame overhead 24 queue sizes 64 6464 64 hqos add port 1 subport 0 rate 500 M size 1000000 tc period 10 hqos add port 1 subport 0 pipes 2000 profile 27 hqos set port 1 lcore 3 Restuls: h5 ~ # rcli sh qos rcvrcv 0: rx rate 798720, nb pkts 624, ind 0 rcv 1: rx rate 798720, nb pkts 624, ind 0 rcv 2: rx rate 798720, nb pkts 624, ind 0 rcv 3: rx rate 798720, nb pkts 624, ind 0 rcv 4: rx rate 798720, nb pkts 624, ind 0 rcv 5: rx rate 798720, nb pkts 624, ind 0 rcv 6: rx rate 798720, nb pkts 624, ind 0 rcv 7: rx rate 798720, nb pkts 624, ind 0 rcv 8: rx rate 798720, nb pkts 624, ind 0 rcv 9: rx rate 798720,nb pkts 624, ind 0 OK. Receiving traffic is rate is equal to expected values. ### ### test 4 ### Traffic generator is configured to use TC3. no profile 23. !! subport tc period has been changed from 10 to 5. Configuration: hqos add profile 27 rate 1 M size 1000000 tc period 40 hqos add profile 23 rate 100 M size 1000000 tc period 40 # qos test porthqos add port 1 rate 10 G mtu 1522 frame overhead 24 queue sizes 64 6464 64 hqos add port 1 subport 0 rate 500 M size 1000000 tc period 5 hqos add port 1 subport 0 pipes 2000 profile 27 hqos set port 1 lcore 3 Restuls: rcv 0: rx rate 0, nb pkts 0, ind 1 rcv 1: rx rate 0, nb pkts 0, ind 1 rcv 2: rx rate 0, nb pkts 0, ind 1 rcv 3: rx rate 0, nb pkts 0, ind 1 rcv 4: rx rate 0, nb pkts 0, ind 1 rcv 5: rx rate 0, nb pkts 0, ind 1 rcv 6: rx rate 0, nb pkts 0, ind 1 rcv 7: rx rate 0, nb pkts 0, ind 1 rcv 8: rx rate 0, nb pkts 0, ind 1 rcv 9: rx rate 0, nb pkts 0, ind 1 ! zero traffic ### ### test 5 ### Traffic generator is configured to use TC3. profile 23 is enabled. subport tc period has been changed from 10 to 5. Configuration: hqos add profile 27 rate 1 M size 1000000 tc period 40 hqos add profile 23 rate 100 M size 1000000 tc period 40 # qos test porthqos add port 1 rate 10 G mtu 1522 frame overhead 24 queue sizes 64 6464 64 hqos add port 1 subport 0 rate 500 M size 1000000 tc period 5 hqos add port 1 subport 0 pipes 2000 profile 27 hqos add port 1 subport 0 pipes 200 profile 23 hqos set port 1 lcore 3 Restuls: h5 ~ # rcli sh qos rcvrcv 0: rx rate 800000, nb pkts 625, ind 1 rcv 1: rx rate 800000, nb pkts 625, ind 1 rcv 2: rx rate 800000, nb pkts 625, ind 1 rcv 3: rx rate 800000, nb pkts 625, ind 1 rcv 4: rx rate 800000, nb pkts 625, ind 1 rcv 5: rx rate 800000, nb pkts 625, ind 1 rcv 6: rx rate 800000, nb pkts 625, ind 1 rcv 7: rx rate 800000, nb pkts 625, ind 1 rcv 8: rx rate 800000, nb pkts 625, ind 1 rcv 9: rx rate 800000,nb pkts 625, ind 1 OK > > > Does this problem exist when you disable oversubscription mode? Worth > looking at grinder_tc_ov_credits_update() and grinder_credits_update() > functions where tc_ov_wm is altered. > > >> > >> > Thanks, >> > JasvinderTest 1 doesn't use TC_OV and all available support bandwidth isdistributed to 300 pipes in a very fair way. There are 300 generatorswith tx rate 1M. They produce 300 traffic flows which enter each own pipe (queue 0) of port 1 subport 0. port 1 subport rate 300 M Then application measures the rx rate of each flows after flow's traffic leaves the scheduler. For example, the following line rcv 284 rx rate 995840 nb pkts 778 shows that rx rate of the flow with number 284 is 995840 bit/s. All 300 rx rates are about 995840 bit/s (1Mbit/s) as expected.[JS] May be try repeat same test but change traffic from tc 0 to tc3. See if this works.The second test with incorrect restuls was already done with tc3.The second test uses the same configuration but uses TC3, so the TC_OV function is being used. And the distribution of traffic in the test is very unfair. Some of the pipes get 875520 bit/s, some of the pipes get only 604160 bit/s despite that there is[JS] try repeat test with increase pipe bandwidth let’s say 50 mbps or even greater.I increased pipe rate to 10Mbit/s and both tests (tc0 and tc3) showed correct and identical results. But, then I changed the tests and increased the number of pipes to 600 to see how it would work with subport congestion. I added 600 pipes generatinig 10Mbit/s and 3G subport limit, therefore each pipe should get equal share which is about 5mbits. And results of both tests (tc0 or tc3) are not very good. First pipes are getting much more bandwidth than the last ones. The difference is 3 times. So, TC_OV is still not working!!
decreasing subport tc_period from 10 to 5 has solved that problemand scheduler started to distribute subport bandwidth between 10 mbit/s pipes almost ideally.
rcv 0 rx rate 7324160 nb pkts 5722 rcv 1 rx rate 7281920 nb pkts 5689 rcv 2 rx rate 7226880 nb pkts 5646 rcv 3 rx rate 7124480 nb pkts 5566 rcv 4 rx rate 7324160 nb pkts 5722 rcv 5 rx rate 7271680 nb pkts 5681 rcv 6 rx rate 7188480 nb pkts 5616 rcv 7 rx rate 7150080 nb pkts 5586 rcv 8 rx rate 7328000 nb pkts 5725 rcv 9 rx rate 7249920 nb pkts 5664 rcv 10 rx rate 7188480 nb pkts 5616 rcv 11 rx rate 7179520 nb pkts 5609 rcv 12 rx rate 7324160 nb pkts 5722 rcv 13 rx rate 7208960 nb pkts 5632 rcv 14 rx rate 7152640 nb pkts 5588 rcv 15 rx rate 7127040 nb pkts 5568 rcv 16 rx rate 7303680 nb pkts 5706 .... rcv 587 rx rate 2406400 nb pkts 1880 rcv 588 rx rate 2406400 nb pkts 1880 rcv 589 rx rate 2406400 nb pkts 1880 rcv 590 rx rate 2406400 nb pkts 1880 rcv 591 rx rate 2406400 nb pkts 1880 rcv 592 rx rate 2398720 nb pkts 1874 rcv 593 rx rate 2400000 nb pkts 1875 rcv 594 rx rate 2400000 nb pkts 1875 rcv 595 rx rate 2400000 nb pkts 1875 rcv 596 rx rate 2401280 nb pkts 1876 rcv 597 rx rate 2401280 nb pkts 1876 rcv 598 rx rate 2401280 nb pkts 1876 rcv 599 rx rate 2402560 nb pkts 1877 rx rate sum 3156416000
... despite that there is _NO_ congestion... congestion at the subport or pipe.And the subport !! doesn't use about 42 mbit/s of available bandwidth. The only difference is those test configurations is TC of generated traffic. Test 1 uses TC 1 while test 2 uses TC 3 (which is use TC_OV function).So, enabling TC_OV changes the results dramatically. ## ## test1 ## hqos add profile 7 rate 2 M size 1000000 tc period 40 # qos test porthqos add port 1 rate 10 G mtu 1522 frame overhead 24 queue sizes 64 64 64 64hqos add port 1 subport 0 rate 300 M size 1000000 tc period 10 hqos add port 1 subport 0 pipes 2000 profile 7 hqos add port 1 subport 0 pipes 200 profile 23 hqos set port 1 lcore 3 port 1 subport rate 300 M number of tx flows 300 generator tx rate 1M TC 1 ... rcv 284 rx rate 995840 nb pkts 778 rcv 285 rx rate 995840 nb pkts 778 rcv 286 rx rate 995840 nb pkts 778 rcv 287 rx rate 995840 nb pkts 778 rcv 288 rx rate 995840 nb pkts 778 rcv 289 rx rate 995840 nb pkts 778 rcv 290 rx rate 995840 nb pkts 778 rcv 291 rx rate 995840 nb pkts 778 rcv 292 rx rate 995840 nb pkts 778 rcv 293 rx rate 995840 nb pkts 778 rcv 294 rx rate 995840 nb pkts 778 ... sum pipe's rx rate is 298 494 720 OK. The subport rate is equally distributed to 300 pipes. ## ## test 2 ## hqos add profile 7 rate 2 M size 1000000 tc period 40 # qos test porthqos add port 1 rate 10 G mtu 1522 frame overhead 24 queue sizes 64 64 64 64hqos add port 1 subport 0 rate 300 M size 1000000 tc period 10 hqos add port 1 subport 0 pipes 2000 profile 7 hqos add port 1 subport 0 pipes 200 profile 23 hqos set port 1 lcore 3 port 1 subport rate 300 M number of tx flows 300 generator tx rate 1M TC 3 h5 ~ # rcli sh qos rcv rcv 0 rx rate 875520 nb pkts 684 rcv 1 rx rate 856320 nb pkts 669 rcv 2 rx rate 849920 nb pkts 664 rcv 3 rx rate 853760 nb pkts 667 rcv 4 rx rate 867840 nb pkts 678 rcv 5 rx rate 844800 nb pkts 660 rcv 6 rx rate 852480 nb pkts 666 rcv 7 rx rate 855040 nb pkts 668 rcv 8 rx rate 865280 nb pkts 676 rcv 9 rx rate 846080 nb pkts 661 rcv 10 rx rate 858880 nb pkts 671 rcv 11 rx rate 870400 nb pkts 680 rcv 12 rx rate 864000 nb pkts 675 rcv 13 rx rate 852480 nb pkts 666 rcv 14 rx rate 855040 nb pkts 668 rcv 15 rx rate 857600 nb pkts 670 rcv 16 rx rate 864000 nb pkts 675 rcv 17 rx rate 866560 nb pkts 677 rcv 18 rx rate 865280 nb pkts 676 rcv 19 rx rate 858880 nb pkts 671 rcv 20 rx rate 856320 nb pkts 669 rcv 21 rx rate 864000 nb pkts 675 rcv 22 rx rate 869120 nb pkts 679 rcv 23 rx rate 856320 nb pkts 669 rcv 24 rx rate 862720 nb pkts 674 rcv 25 rx rate 865280 nb pkts 676 rcv 26 rx rate 867840 nb pkts 678 rcv 27 rx rate 870400 nb pkts 680 rcv 28 rx rate 860160 nb pkts 672 rcv 29 rx rate 870400 nb pkts 680 rcv 30 rx rate 869120 nb pkts 679 rcv 31 rx rate 870400 nb pkts 680 rcv 32 rx rate 858880 nb pkts 671 rcv 33 rx rate 858880 nb pkts 671 rcv 34 rx rate 852480 nb pkts 666 rcv 35 rx rate 874240 nb pkts 683 rcv 36 rx rate 855040 nb pkts 668 rcv 37 rx rate 853760 nb pkts 667 rcv 38 rx rate 869120 nb pkts 679 rcv 39 rx rate 885760 nb pkts 692 rcv 40 rx rate 861440 nb pkts 673 rcv 41 rx rate 852480 nb pkts 666 rcv 42 rx rate 871680 nb pkts 681 ... ... rcv 288 rx rate 766720 nb pkts 599 rcv 289 rx rate 766720 nb pkts 599 rcv 290 rx rate 766720 nb pkts 599 rcv 291 rx rate 766720 nb pkts 599 rcv 292 rx rate 762880 nb pkts 596 rcv 293 rx rate 762880 nb pkts 596 rcv 294 rx rate 762880 nb pkts 596 rcv 295 rx rate 760320 nb pkts 594 rcv 296 rx rate 604160 nb pkts 472 rcv 297 rx rate 604160 nb pkts 472 rcv 298 rx rate 604160 nb pkts 472 rcv 299 rx rate 604160 nb pkts 472 rx rate sum 258839040 FAILED. The subport rate is distributed NOT equally between 300 pipes. Some subport bandwith (about 42) is not being used!
