+1, "vpp# show hardware" and "$ cat /proc/cpuinfo" to make sure that the selected cores are on the right NUMA socket...
HTH... Dave -----Original Message----- From: vpp-dev@lists.fd.io <vpp-dev@lists.fd.io> On Behalf Of Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io Sent: Wednesday, November 14, 2018 11:51 AM To: Ray Kinsella <m...@ashroe.eu>; vpp-dev@lists.fd.io; Neale Ranns (nranns) <nra...@cisco.com>; Damjan Marion (damarion) <damar...@cisco.com> Cc: vpp-dev@lists.fd.io Subject: Re: [vpp-dev] ip4-load-balance > Any clues as to what might causing the delta? >> using FD.io VPP 18.07 If you are still using that, the performance might depend on the NUMA node used. There was a placement bug, and the fix [1] has been merged just before 18.10. Vratko. [1] https://gerrit.fd.io/r/15483 -----Original Message----- From: vpp-dev@lists.fd.io <vpp-dev@lists.fd.io> On Behalf Of Ray Kinsella Sent: Wednesday, 2018-November-14 15:03 To: vpp-dev@lists.fd.io; Neale Ranns (nranns) <nra...@cisco.com>; Damjan Marion (damarion) <damar...@cisco.com> Subject: Re: [vpp-dev] ip4-load-balance CSIT is measuring roughly 12mpps on HSW. When I measure with an equivalent system I get 10.5mpps on HSW. What I am finding is that DPDK is more efficient on CSIT HSW. CSIT Graph Node Clocks/Vector dpdk-input 6.07e1 TenGigabitEtherneta/0/0-tx 4.09e1 My System dpdk-input 8.30e1 TenGigabitEthernet83/0/1-tx 5.51e1 So between TX and Input, I am burning an extra 30 Clocks per vector. That ends up being the essential difference between 10.5mpps and 12mpps. I tried this with both binaries I built myself and the binaries in package cloud, and I get the same result. Any clues as to what might causing the delta? (I verified that my system setup was equivalent to CSIT) startup.conf @ https://pastebin.com/tuYZ6xt8 Ray K On 13/11/2018 18:09, Ray Kinsella wrote: > Mystery solved, > > I was missing the interface on the IP Route. > > ip route add count 1 20.20.20.0/24 via 1.1.1.2 > TenGigabitEthernet83/0/1 > > Ray K > > On 13/11/2018 15:39, Ray Kinsella wrote: >> Folks, >> >> I have configuring my system to get something comparable to CSIT >> performance and I am a few mpps off at the moment, using FD.io VPP 18.07. >> >> I duplicated the IPv4 Base and Scale Test Cases (and environment) >> locally and I end up with extra graph node 'ip4-load-balance' in my >> pipeline? >> >> CSIT records the following pipeline in Test Operation Data. >> https://docs.fd.io/csit/rls1807/report/test_operational_data >> >> Thread 1 vpp_wk_0 (lcore 2) >> Time 5.7, average vectors/node 245.79, last 128 main loops 13.03 >> per node 151.64 >> vector rates in 1.2082e7, out 1.2082e7, drop 0.0000e0, punt 0.0000e0 >> Name State Calls >> Vectors Suspends Clocks Vectors/Call >> TenGigabitEtherneta/0/0-output active 140125 >> 34429184 0 8.41e0 245.70 >> TenGigabitEtherneta/0/0-tx active 140125 >> 34429184 0 4.09e1 245.70 >> TenGigabitEtherneta/0/1-output active 140071 >> 34428928 0 8.58e0 245.79 >> TenGigabitEtherneta/0/1-tx active 140071 >> 34428928 0 3.93e1 245.79 >> dpdk-input polling 140580 >> 68858112 0 6.07e1 489.81 >> ip4-input-no-checksum active 280127 >> 68858112 0 2.05e1 245.81 >> ip4-lookup active 280127 >> 68858112 0 3.03e1 245.81 >> ip4-rewrite active 280127 >> 68858112 0 2.92e1 245.81 >> >> >> I get the following pipeline, with the additional graph node - >> ip4-load-balance. >> >> Thread 2 vpp_wk_1 (lcore 20) >> Time 188.9, average vectors/node 256.00, last 128 main loops 14.00 >> per node 256.00 >> vector rates in 9.3287e6, out 9.3287e6, drop 0.0000e0, punt 0.0000e0 >> Name State Calls >> Vectors Suspends Clocks Vectors/Call >> TenGigabitEthernet83/0/1-outpu active 6881842 >> 1761751552 0 8.46e0 256.00 >> TenGigabitEthernet83/0/1-tx active 6881842 >> 1761751552 0 5.53e1 256.00 dpdk-input >> polling 6881842 >> 1761751552 0 8.58e1 256.00 >> ip4-input-no-checksum active 6881842 >> 1761751552 0 2.19e1 256.00 >> ip4-load-balance active 6881842 >> 1761751552 0 1.68e1 25 >> 6.00 >> ip4-lookup active 6881842 >> 1761751552 0 2.80e1 256.00 >> ip4-rewrite active 6881842 >> 1761751552 0 2.89e1 256.00 >> >> Any idea where ip4-load-balance is coming from? >> >> Ray K >> >> -=-=-=-=-=-=-=-=-=-=-=- >> Links: You receive all messages sent to this group. >> >> View/Reply Online (#11223): >> https://lists.fd.io/g/vpp-dev/message/11223 >> Mute This Topic: https://lists.fd.io/mt/28123777/675355 >> Group Owner: vpp-dev+ow...@lists.fd.io >> Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [m...@ashroe.eu] >> -=-=-=-=-=-=-=-=-=-=-=- >> > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > > View/Reply Online (#11226): > https://lists.fd.io/g/vpp-dev/message/11226 > Mute This Topic: https://lists.fd.io/mt/28123777/675355 > Group Owner: vpp-dev+ow...@lists.fd.io > Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [m...@ashroe.eu] > -=-=-=-=-=-=-=-=-=-=-=- >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#11253): https://lists.fd.io/g/vpp-dev/message/11253 Mute This Topic: https://lists.fd.io/mt/28123777/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
