Without zcount and ./zsend -i zc:eth7 -g 1 -c 1 -a, it starts off at 14.88/10Gbps. It stays like that until I start doing stuff in another window (ssh’d in) like running top or changing directories. At which point it dropped down to 14.5Mpps and seems to be hovering there now.
> On Jun 8, 2016, at 5:43 PM, Alfredo Cardigliano <[email protected]> wrote: > > It could be related to available memory bandwidth, do you see the same when > zcount is not running? > > Alfredo > >> On 08 Jun 2016, at 23:40, Jason Lixfeld <[email protected]> wrote: >> >> I did. zsend to one core, zcount to another, but I can’t seen to quote get >> up there. send sometimes has the tendency to start off strong, about >> 14.86Mpps, but slowly ramps down to about 14.25Mpps after about 15 seconds. >> >> On Jun 8, 2016, at 4:48 PM, Alfredo Cardigliano <[email protected]> wrote: >> >> Did you bind pfsend/zsend to a cpu core? >> >> Alfredo >> >> On 08 Jun 2016, at 22:44, Jason Lixfeld <[email protected]> wrote: >> >> I don’t seem to be able to transmit the full 14.88Mpps to get full linerate. >> Is there anything else I can tweak? I’ve added the -a option which has given >> gotten me a bit closer, but not all the way. >> >> My CPU is a 6 core Intel® Xeon® CPU E5-2620 v3 @ 2.40GHz >> >> Thanks once again! >> >> On Jun 8, 2016, at 4:21 PM, Alfredo Cardigliano <[email protected]> wrote: >> >> Please use RSS=1,1,1,1,1,1 as you have 6 ports. >> >> Alfredo >> >> On 08 Jun 2016, at 22:11, [email protected] wrote: >> >> Thanks Alfredo, >> >> If I’m reading this correctly, eth2 has 12 Tx and Rx queues while eth7 has 1 >> each: >> >> root@pgen:/home/jlixfeld/PF_RING/drivers/intel/ixgbe/ixgbe-4.1.5-zc/src# cat >> /proc/net/pf_ring/dev/eth2/info Name: eth2 Index: 86 Address: >> 00:0C:BD:08:80:98 Polling Mode: NAPI/ZC Type: Ethernet Family: Intel ixgbe >> 82599 Max # TX Queues: 12 # Used RX Queues: 12 Num RX Slots: 32768 Num TX >> Slots: 32768 >> root@pgen:/home/jlixfeld/PF_RING/drivers/intel/ixgbe/ixgbe-4.1.5-zc/src# cat >> /proc/net/pf_ring/dev/eth7/info Name: eth7 Index: 83 Address: >> 00:0C:BD:08:80:9D Polling Mode: NAPI/ZC Type: Ethernet Family: Intel ixgbe >> 82599 Max # TX Queues: 1 # Used RX Queues: 1 Num RX Slots: 32768 Num TX >> Slots: 32768 >> root@pgen:/home/jlixfeld/PF_RING/drivers/intel/ixgbe/ixgbe-4.1.5-zc/src# >> >> load_driver.sh seems to be set to disable multi queue so I’m not quite sure >> how this got this way, or how to correct it? >> >> root@pgen:/home/jlixfeld/PF_RING/drivers/intel/ixgbe/ixgbe-4.1.5-zc/src# >> grep -i rss load_driver.sh #insmod ./ixgbe.ko RSS=0,0,0,0 insmod ./ixgbe.ko >> RSS=1,1,1,1 #insmod ./ixgbe.ko RSS=1,1,1,1 low_latency_tx=1 #insmod >> ./ixgbe.ko MQ=1,1,1,1 RSS=16,16,16,16 #insmod ./ixgbe.ko RSS=1,1,1,1 >> FdirPballoc=3,3,3,3 #insmod ./ixgbe.ko RSS=1,1,1,1 numa_cpu_affinity=0,0,0,0 >> root@pgen:/home/jlixfeld/PF_RING/drivers/intel/ixgbe/ixgbe-4.1.5-zc/src# >> >> On Jun 8, 2016, at 3:55 PM, Alfredo Cardigliano <[email protected]> wrote: >> >> Hi Jason no problem, I was able to read something :-) Please check that both >> the interfaces are configured with a single RSS queue (take a look at >> /proc/net/pf_ring/dev/eth2/info) >> >> Alfredo >> >> On 08 Jun 2016, at 21:09, Jason Lixfeld <[email protected]> wrote: >> >> My gosh! I’m so sorry for the way this is formatted. My mailer insists that >> this message was sent in plain-text, not whatever the heck this is! >> >> I’m sorry this is so impossible to read :( >> >> On Jun 8, 2016, at 3:05 PM, [email protected] wrote: >> >> Hello, >> >> My first run through with poring. :) >> >> I’ve compiled the zc variant of pfring in an attempt to get linerate between >> two of the ports which are looped together. >> >> The NIC is a 6 port 82599 based one. >> >> root@pgen:/home/jlixfeld/PF_RING/drivers/intel/ixgbe/ixgbe-4.1.5-zc/src# >> ./load_driver.sh irqbalance: no process found Configuring eth6 IFACE CORE >> MASK -> FILE ======================= eth6 0 1 -> /proc/irq/87/smp_affinity >> Configuring eth7 IFACE CORE MASK -> FILE ======================= eth7 0 1 -> >> /proc/irq/89/smp_affinity Configuring eth2 IFACE CORE MASK -> FILE >> ======================= eth2 0 1 -> /proc/irq/96/smp_affinity eth2 1 2 -> >> /proc/irq/97/smp_affinity eth2 2 4 -> /proc/irq/98/smp_affinity eth2 3 8 -> >> /proc/irq/99/smp_affinity eth2 4 10 -> /proc/irq/100/smp_affinity eth2 5 20 >> -> /proc/irq/101/smp_affinity eth2 6 40 -> /proc/irq/102/smp_affinity eth2 7 >> 80 -> /proc/irq/103/smp_affinity eth2 8 100 -> /proc/irq/104/smp_affinity >> eth2 9 200 -> /proc/irq/105/smp_affinity eth2 10 400 -> >> /proc/irq/106/smp_affinity eth2 11 800 -> /proc/irq/107/smp_affinity >> Configuring eth3 IFACE CORE MASK -> FILE ======================= eth3 0 1 -> >> /proc/irq/109/smp_affinity eth3 1 2 -> /proc/irq/110/smp_affinity eth3 2 4 >> -> /proc/irq/111/smp_affinity eth3 3 8 -> /proc/irq/112/smp_affinity eth3 4 >> 10 -> /proc/irq/113/smp_affinity eth3 5 20 -> /proc/irq/114/smp_affinity >> eth3 6 40 -> /proc/irq/115/smp_affinity eth3 7 80 -> >> /proc/irq/116/smp_affinity eth3 8 100 -> /proc/irq/117/smp_affinity eth3 9 >> 200 -> /proc/irq/118/smp_affinity eth3 10 400 -> /proc/irq/119/smp_affinity >> eth3 11 800 -> /proc/irq/120/smp_affinity Configuring eth4 IFACE CORE MASK >> -> FILE ======================= eth4 0 1 -> /proc/irq/91/smp_affinity >> Configuring eth5 IFACE CORE MASK -> FILE ======================= eth5 0 1 -> >> /proc/irq/93/smp_affinity >> root@pgen:/home/jlixfeld/PF_RING/drivers/intel/ixgbe/ixgbe-4.1.5-zc/src# >> >> My issue is that I only get 10Gbps in one direction. If zc:eth7 is the >> sender, zc:eth2 only sees Rx @ 0.54Gbps: >> >> root@pgen:/home/jlixfeld/PF_RING/userland/examples_zc# ./zsend -i zc:eth7 -c >> 2 >> >> Absolute Stats: 111'707'057 pkts – 9'383'392'788 bytes Actual Stats: >> 13'983'520.42 pps – 9.40 Gbps [1133946996 bytes / 1.0 sec] >> >> root@pgen:/home/jlixfeld/PF_RING/userland/examples_zc# ./zcount -i zc:eth2 >> -c 1 >> >> Absolute Stats: 5'699'096 pkts (33'135'445 drops) – 478'724'064 bytes Actual >> Stats: 802'982.00 pps (4'629'316.93 drops) – 0.54 Gbps >> >> But, if zc:eth2 is the sender, zc:eth7 sees rates more in-line with what >> zc:eth2 is sending. >> >> root@pgen:/home/jlixfeld/PF_RING/userland/examples_zc# ./zsend -i zc:eth2 -c >> 2 >> >> Absolute Stats: 28'285'274 pkts – 2'375'963'016 bytes Actual Stats: >> 14'114'355.24 pps – 9.48 Gbps [1185800280 bytes / 1.0 sec] >> >> root@pgen:/home/jlixfeld/PF_RING/userland/examples_zc# ./zcount -i zc:eth7 >> -c 1 >> >> Absolute Stats: 28'007'460 pkts (0 drops) – 2'352'626'640 bytes Actual >> Stats: 14'044'642.54 pps (0.00 drops) – 9.44 Gbps >> >> I’ve done some reading, but I haven’t found anything that has pointed me >> towards a possible reason why this is happening. I’m wondering if anyone has >> any thoughts? >> >> Thanks! >> >> _______________________________________ Ntop-misc mailing list >> [email protected] >> https://u3172041.ct.sendgrid.net/wf/click?upn=UmVayPZKpIdftgoYx-2BU88-2B1jnUM7Rvx0OUKMS78f3Zn78u-2BsRXpXcIeS3oh8u5YEWIgu-2BvmYwr4GzDvZ-2FfPGPw-3D-3D_rhOy4YLBxOPOohPdjw4-2BtC7rnGr1J9pkUkX-2FFAaRhBOTfw29ZNkYUq90lhDJi-2B9CuxzIJw-2FLGOjZ9aHSbsPSY5J-2Ffpk6-2BFcVnqND69ZBuhYmQjZ-2Bq3mlcoeiYBh67t9KsJILTV9hvhbd1L8IixulO8MUefntlHu6ZBs-2BNCPIl1uh24zjUs36YBVlzFLfsRIwPrI2M0zDX2Wa01RpmD4UxWBnBK6uTTTmJ-2B9BjYX2Z34-3D >> >> _________________________________________ Ntop-misc mailing list >> [email protected] >> https://u3172041.ct.sendgrid.net/wf/click?upn=UmVayPZKpIdftgoYx-2BU88-2B1jnUM7Rvx0OUKMS78f3Zn78u-2BsRXpXcIeS3oh8u5YEWIgu-2BvmYwr4GzDvZ-2FfPGPw-3D-3D_rhOy4YLBxOPOohPdjw4-2BtC7rnGr1J9pkUkX-2FFAaRhBOTfw29ZNkYUq90lhDJi-2B9CuxzIJw-2FLGOjZ9aHSbsPSYxhd-2BRbjxOym6QCkXCjOjakdRI3eWBrJCqUixq8KAPZPx-2BkLhFONgmbI6mD-2FjzHC-2F7RcGHtH5MUzGvB2MoVD8-2FkYkdfe7I5QBHR5vI-2FwjGMb2ewGKIjFtf1lzDhRLAaDFAKPtg8aBIE3MlxxBDCx7jE-3D >> >> _________________________________________ Ntop-misc mailing list >> [email protected] >> https://u3172041.ct.sendgrid.net/wf/click?upn=UmVayPZKpIdftgoYx-2BU88-2B1jnUM7Rvx0OUKMS78f3Zn78u-2BsRXpXcIeS3oh8u5YEWIgu-2BvmYwr4GzDvZ-2FfPGPw-3D-3D_rhOy4YLBxOPOohPdjw4-2BtC7rnGr1J9pkUkX-2FFAaRhBOTfw29ZNkYUq90lhDJi-2B9CuxzIJw-2FLGOjZ9aHSbsPSY85UPhR4GSaI-2F-2FoL9Mud4BwtMKDr-2FWTl-2F8PB0gQCCGXgGzgJU5uN4onRl8ihuXvXFLqoQxk7CPkPuZU4OmndQRBYINdWpejhwPWp8yYmby4g6YWqLy1qvVTbp3LQtNd49VgMcu8rdQl96BpClctOwXA-3D >> >> ___________________________________________ Ntop-misc mailing list >> [email protected] >> https://u3172041.ct.sendgrid.net/wf/click?upn=UmVayPZKpIdftgoYx-2BU88-2B1jnUM7Rvx0OUKMS78f3Zn78u-2BsRXpXcIeS3oh8u5YEWIgu-2BvmYwr4GzDvZ-2FfPGPw-3D-3D_rhOy4YLBxOPOohPdjw4-2BtC7rnGr1J9pkUkX-2FFAaRhBOTfw29ZNkYUq90lhDJi-2B9CuxzIJw-2FLGOjZ9aHSbsPSYxGcFfKLabCWfIbDfSYxB78OMpJ-2FMm8-2Fp97dm25oYwN2Njp0r-2FAyjglzoCigTRXS0td5g-2BdKIksCMU83a8LgzzDr8WdwXZf9QKAQFwx5NyiRINOzl2269IYqcVUHySrgx-2BZSp0FYvIZtzhDam1UtKdw-3D >> >> ___________________________________________ Ntop-misc mailing list >> [email protected] >> https://u3172041.ct.sendgrid.net/wf/click?upn=UmVayPZKpIdftgoYx-2BU88-2B1jnUM7Rvx0OUKMS78f3Zn78u-2BsRXpXcIeS3oh8u5YEWIgu-2BvmYwr4GzDvZ-2FfPGPw-3D-3D_rhOy4YLBxOPOohPdjw4-2BtC7rnGr1J9pkUkX-2FFAaRhBOTfw29ZNkYUq90lhDJi-2B9CuxzIJw-2FLGOjZ9aHSbsPSYzvD47TuWHqsDkdAUU09bFCR18Kd4yd5VzQBImb4LlOSnL5A36XKq-2BNxcT4F0Kr7yc4hsZMVrBfuHDcPBLfPAXL2Dll3KTOeNU5KnP8ZPOZnkFbjH7MUSMYHsSzxvgSqXh92vro2UEjhY2wmZ0DTpMo-3D >> >> _____________________________________________ Ntop-misc mailing list >> [email protected] >> https://u3172041.ct.sendgrid.net/wf/click?upn=UmVayPZKpIdftgoYx-2BU88-2B1jnUM7Rvx0OUKMS78f3Zn78u-2BsRXpXcIeS3oh8u5YEWIgu-2BvmYwr4GzDvZ-2FfPGPw-3D-3D_rhOy4YLBxOPOohPdjw4-2BtC7rnGr1J9pkUkX-2FFAaRhBOTfw29ZNkYUq90lhDJi-2B9CuxzIJw-2FLGOjZ9aHSbsPSYzJjEW2kU6rlawAJygmCR5mFtfvfjdiFSjVvdRrE2XXhOTy9N-2B2T0wEZhhZjGT-2FyQswVbv2ypLLCYGEswuM7ofBr1mdDLjMSINU6U2scnBBaKX8hxQbew3reCNVK68tjtEmdnFDKXsMG-2FMzaORyJr2c-3D >> >> _____________________________________________ Ntop-misc mailing list >> [email protected] >> https://u3172041.ct.sendgrid.net/wf/click?upn=UmVayPZKpIdftgoYx-2BU88-2B1jnUM7Rvx0OUKMS78f3Zn78u-2BsRXpXcIeS3oh8u5YEWIgu-2BvmYwr4GzDvZ-2FfPGPw-3D-3D_rhOy4YLBxOPOohPdjw4-2BtC7rnGr1J9pkUkX-2FFAaRhBOTfw29ZNkYUq90lhDJi-2B9CuxzIJw-2FLGOjZ9aHSbsPSY1tIKBAfd7zTcbEK0nQR8MRk-2FDar2ORBCZo3LDJG-2B2bXoZbwzyFsy4KyZwTxx6TGy22aoV5fNZRF6-2BpmFf3Ab3gh6Cvg7ayjk2RLaTNiS7wn-2Fvp4KMXAuMktocWwVKdkX8Jq3LP90SEBdR1kQCtilOg-3D >> >> >> _______________________________________________ >> Ntop-misc mailing list >> [email protected] >> https://u3172041.ct.sendgrid.net/wf/click?upn=UmVayPZKpIdftgoYx-2BU88-2B1jnUM7Rvx0OUKMS78f3Zn78u-2BsRXpXcIeS3oh8u5YEWIgu-2BvmYwr4GzDvZ-2FfPGPw-3D-3D_rhOy4YLBxOPOohPdjw4-2BtC7rnGr1J9pkUkX-2FFAaRhBOTfw29ZNkYUq90lhDJi-2B9CuxzIJw-2FLGOjZ9aHSbsPSY0tMxSMCFlLPLDT8UK9hWXlgxblQsHwtaw8yhT-2BmI3gFewBR-2B2yniWPtxAG8GoVlbjalK5OGleKdyFuwN59KR2MOgBSPJHcTDaVvri0spBMJrcGy6uEqP2mQXSbPHMXFNXFQGIitwLhnoNqn3l-2BkvwA-3D > > _______________________________________________ > Ntop-misc mailing list > [email protected] > https://u3172041.ct.sendgrid.net/wf/click?upn=UmVayPZKpIdftgoYx-2BU88-2B1jnUM7Rvx0OUKMS78f3Zn78u-2BsRXpXcIeS3oh8u5YEWIgu-2BvmYwr4GzDvZ-2FfPGPw-3D-3D_rhOy4YLBxOPOohPdjw4-2BtC7rnGr1J9pkUkX-2FFAaRhBOTfw29ZNkYUq90lhDJi-2B9CuxzIJw-2FLGOjZ9aHSbsPSY9SeHUk-2BQxh51lTmOnrCYSJb6usytmeZQRRrXxHml8fQJn-2B2LUjg0zvhhor6oPmWx-2FIkgMtlQbcFMxFtJeTItiyKkwLHTBBUmW8uUUUSd8Nno7KwxXMBfckof5Rz62V7RWAgLIgW8UOI0j5XLecxJsc-3D
_______________________________________________ Ntop-misc mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop-misc
