It could be related to available memory bandwidth, do you see the same when zcount is not running?
Alfredo > On 08 Jun 2016, at 23:40, Jason Lixfeld <[email protected]> wrote: > > I did. zsend to one core, zcount to another, but I can’t seen to quote get up > there. send sometimes has the tendency to start off strong, about 14.86Mpps, > but slowly ramps down to about 14.25Mpps after about 15 seconds. > > On Jun 8, 2016, at 4:48 PM, Alfredo Cardigliano <[email protected]> wrote: > > Did you bind pfsend/zsend to a cpu core? > > Alfredo > > On 08 Jun 2016, at 22:44, Jason Lixfeld <[email protected]> wrote: > > I don’t seem to be able to transmit the full 14.88Mpps to get full linerate. > Is there anything else I can tweak? I’ve added the -a option which has given > gotten me a bit closer, but not all the way. > > My CPU is a 6 core Intel® Xeon® CPU E5-2620 v3 @ 2.40GHz > > Thanks once again! > > On Jun 8, 2016, at 4:21 PM, Alfredo Cardigliano <[email protected]> wrote: > > Please use RSS=1,1,1,1,1,1 as you have 6 ports. > > Alfredo > > On 08 Jun 2016, at 22:11, [email protected] wrote: > > Thanks Alfredo, > > If I’m reading this correctly, eth2 has 12 Tx and Rx queues while eth7 has 1 > each: > > root@pgen:/home/jlixfeld/PF_RING/drivers/intel/ixgbe/ixgbe-4.1.5-zc/src# cat > /proc/net/pf_ring/dev/eth2/info Name: eth2 Index: 86 Address: > 00:0C:BD:08:80:98 Polling Mode: NAPI/ZC Type: Ethernet Family: Intel ixgbe > 82599 Max # TX Queues: 12 # Used RX Queues: 12 Num RX Slots: 32768 Num TX > Slots: 32768 > root@pgen:/home/jlixfeld/PF_RING/drivers/intel/ixgbe/ixgbe-4.1.5-zc/src# cat > /proc/net/pf_ring/dev/eth7/info Name: eth7 Index: 83 Address: > 00:0C:BD:08:80:9D Polling Mode: NAPI/ZC Type: Ethernet Family: Intel ixgbe > 82599 Max # TX Queues: 1 # Used RX Queues: 1 Num RX Slots: 32768 Num TX > Slots: 32768 > root@pgen:/home/jlixfeld/PF_RING/drivers/intel/ixgbe/ixgbe-4.1.5-zc/src# > > load_driver.sh seems to be set to disable multi queue so I’m not quite sure > how this got this way, or how to correct it? > > root@pgen:/home/jlixfeld/PF_RING/drivers/intel/ixgbe/ixgbe-4.1.5-zc/src# grep > -i rss load_driver.sh #insmod ./ixgbe.ko RSS=0,0,0,0 insmod ./ixgbe.ko > RSS=1,1,1,1 #insmod ./ixgbe.ko RSS=1,1,1,1 low_latency_tx=1 #insmod > ./ixgbe.ko MQ=1,1,1,1 RSS=16,16,16,16 #insmod ./ixgbe.ko RSS=1,1,1,1 > FdirPballoc=3,3,3,3 #insmod ./ixgbe.ko RSS=1,1,1,1 numa_cpu_affinity=0,0,0,0 > root@pgen:/home/jlixfeld/PF_RING/drivers/intel/ixgbe/ixgbe-4.1.5-zc/src# > > On Jun 8, 2016, at 3:55 PM, Alfredo Cardigliano <[email protected]> wrote: > > Hi Jason no problem, I was able to read something :-) Please check that both > the interfaces are configured with a single RSS queue (take a look at > /proc/net/pf_ring/dev/eth2/info) > > Alfredo > > On 08 Jun 2016, at 21:09, Jason Lixfeld <[email protected]> wrote: > > My gosh! I’m so sorry for the way this is formatted. My mailer insists that > this message was sent in plain-text, not whatever the heck this is! > > I’m sorry this is so impossible to read :( > > On Jun 8, 2016, at 3:05 PM, [email protected] wrote: > > Hello, > > My first run through with poring. :) > > I’ve compiled the zc variant of pfring in an attempt to get linerate between > two of the ports which are looped together. > > The NIC is a 6 port 82599 based one. > > root@pgen:/home/jlixfeld/PF_RING/drivers/intel/ixgbe/ixgbe-4.1.5-zc/src# > ./load_driver.sh irqbalance: no process found Configuring eth6 IFACE CORE > MASK -> FILE ======================= eth6 0 1 -> /proc/irq/87/smp_affinity > Configuring eth7 IFACE CORE MASK -> FILE ======================= eth7 0 1 -> > /proc/irq/89/smp_affinity Configuring eth2 IFACE CORE MASK -> FILE > ======================= eth2 0 1 -> /proc/irq/96/smp_affinity eth2 1 2 -> > /proc/irq/97/smp_affinity eth2 2 4 -> /proc/irq/98/smp_affinity eth2 3 8 -> > /proc/irq/99/smp_affinity eth2 4 10 -> /proc/irq/100/smp_affinity eth2 5 20 > -> /proc/irq/101/smp_affinity eth2 6 40 -> /proc/irq/102/smp_affinity eth2 7 > 80 -> /proc/irq/103/smp_affinity eth2 8 100 -> /proc/irq/104/smp_affinity > eth2 9 200 -> /proc/irq/105/smp_affinity eth2 10 400 -> > /proc/irq/106/smp_affinity eth2 11 800 -> /proc/irq/107/smp_affinity > Configuring eth3 IFACE CORE MASK -> FILE ======================= eth3 0 1 -> > /proc/irq/109/smp_affinity eth3 1 2 -> /proc/irq/110/smp_affinity eth3 2 4 -> > /proc/irq/111/smp_affinity eth3 3 8 -> /proc/irq/112/smp_affinity eth3 4 10 > -> /proc/irq/113/smp_affinity eth3 5 20 -> /proc/irq/114/smp_affinity eth3 6 > 40 -> /proc/irq/115/smp_affinity eth3 7 80 -> /proc/irq/116/smp_affinity eth3 > 8 100 -> /proc/irq/117/smp_affinity eth3 9 200 -> /proc/irq/118/smp_affinity > eth3 10 400 -> /proc/irq/119/smp_affinity eth3 11 800 -> > /proc/irq/120/smp_affinity Configuring eth4 IFACE CORE MASK -> FILE > ======================= eth4 0 1 -> /proc/irq/91/smp_affinity Configuring > eth5 IFACE CORE MASK -> FILE ======================= eth5 0 1 -> > /proc/irq/93/smp_affinity > root@pgen:/home/jlixfeld/PF_RING/drivers/intel/ixgbe/ixgbe-4.1.5-zc/src# > > My issue is that I only get 10Gbps in one direction. If zc:eth7 is the > sender, zc:eth2 only sees Rx @ 0.54Gbps: > > root@pgen:/home/jlixfeld/PF_RING/userland/examples_zc# ./zsend -i zc:eth7 -c 2 > > Absolute Stats: 111'707'057 pkts – 9'383'392'788 bytes Actual Stats: > 13'983'520.42 pps – 9.40 Gbps [1133946996 bytes / 1.0 sec] > > root@pgen:/home/jlixfeld/PF_RING/userland/examples_zc# ./zcount -i zc:eth2 -c > 1 > > Absolute Stats: 5'699'096 pkts (33'135'445 drops) – 478'724'064 bytes Actual > Stats: 802'982.00 pps (4'629'316.93 drops) – 0.54 Gbps > > But, if zc:eth2 is the sender, zc:eth7 sees rates more in-line with what > zc:eth2 is sending. > > root@pgen:/home/jlixfeld/PF_RING/userland/examples_zc# ./zsend -i zc:eth2 -c 2 > > Absolute Stats: 28'285'274 pkts – 2'375'963'016 bytes Actual Stats: > 14'114'355.24 pps – 9.48 Gbps [1185800280 bytes / 1.0 sec] > > root@pgen:/home/jlixfeld/PF_RING/userland/examples_zc# ./zcount -i zc:eth7 -c > 1 > > Absolute Stats: 28'007'460 pkts (0 drops) – 2'352'626'640 bytes Actual Stats: > 14'044'642.54 pps (0.00 drops) – 9.44 Gbps > > I’ve done some reading, but I haven’t found anything that has pointed me > towards a possible reason why this is happening. I’m wondering if anyone has > any thoughts? > > Thanks! > > _______________________________________ Ntop-misc mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop-misc > <https://u3172041.ct.sendgrid.net/wf/click?upn=UmVayPZKpIdftgoYx-2BU88-2B1jnUM7Rvx0OUKMS78f3Zn78u-2BsRXpXcIeS3oh8u5YEWIgu-2BvmYwr4GzDvZ-2FfPGPw-3D-3D_rhOy4YLBxOPOohPdjw4-2BtC7rnGr1J9pkUkX-2FFAaRhBNGCYbL1pm-2BUp3pjkSSXmHO-2BSjQS-2FE3plG0DCCI1oibknxYcTKe9qE5VdQyoepofg0NN5t-2FrxushXJFLOiHmFBA01yoxh19bTQxGLXFvCKQhtZyLlStUPDDCmokdYeJbLWrs2Iam4wOwaAjH4zd9QkrPUXCZVdCUSnUg3d74dEF0TxXR5sWsA7E1pDibZaoADA-3D> > _________________________________________ Ntop-misc mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop-misc > <https://u3172041.ct.sendgrid.net/wf/click?upn=UmVayPZKpIdftgoYx-2BU88-2B1jnUM7Rvx0OUKMS78f3Zn78u-2BsRXpXcIeS3oh8u5YEWIgu-2BvmYwr4GzDvZ-2FfPGPw-3D-3D_rhOy4YLBxOPOohPdjw4-2BtC7rnGr1J9pkUkX-2FFAaRhBNGCYbL1pm-2BUp3pjkSSXmHO-2BSjQS-2FE3plG0DCCI1oibko-2F7KQ6J65kiwk3WUkLm9gv5hnAGOUtFlfWJ1hCWjjv2lQ3yGyDHqbahrw2nlOu-2Fwk1xqcBuNYwoQf0Qhww-2BeXoGcuTiRtbKdKvkTFb-2B4ESjNAb6LIVTQ5cZ2vX-2BWV6jKiENMPP4FU4e-2FgzR2NYDGqo-3D> > _________________________________________ Ntop-misc mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop-misc > <https://u3172041.ct.sendgrid.net/wf/click?upn=UmVayPZKpIdftgoYx-2BU88-2B1jnUM7Rvx0OUKMS78f3Zn78u-2BsRXpXcIeS3oh8u5YEWIgu-2BvmYwr4GzDvZ-2FfPGPw-3D-3D_rhOy4YLBxOPOohPdjw4-2BtC7rnGr1J9pkUkX-2FFAaRhBNGCYbL1pm-2BUp3pjkSSXmHO-2BSjQS-2FE3plG0DCCI1oibkmKL8C8cPtQ5dLi0-2F1Zy6i1eP10vcKMNp5qDOLaLpz8kJ-2BZC5fS6pKattY6apOes3Fs2RKpe5J9-2B0PkO-2B-2F27ggooSBB7dQ7y8MDgU2tkOZga0zQiATo7H9wgjON6Blz5M32Yay6mxLfZSMHxTqwMdwM-3D> > ___________________________________________ Ntop-misc mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop-misc > <https://u3172041.ct.sendgrid.net/wf/click?upn=UmVayPZKpIdftgoYx-2BU88-2B1jnUM7Rvx0OUKMS78f3Zn78u-2BsRXpXcIeS3oh8u5YEWIgu-2BvmYwr4GzDvZ-2FfPGPw-3D-3D_rhOy4YLBxOPOohPdjw4-2BtC7rnGr1J9pkUkX-2FFAaRhBNGCYbL1pm-2BUp3pjkSSXmHO-2BSjQS-2FE3plG0DCCI1oibks6hm7u8418mofijS0mUcSvQvv5FnO8qkx2eNPhg5m2EJGUHiB-2Fbch1Y1KB6FnVbzpf53-2BpAM-2BAHk8aMs6X3TUwr-2F2kxhxiyJrgiEK7SXX4p7-2BCOEJUn-2F7C5konsGtXe7vuY215D-2BOgGWoKcJbI-2B6jA-3D> > ___________________________________________ Ntop-misc mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop-misc > <https://u3172041.ct.sendgrid.net/wf/click?upn=UmVayPZKpIdftgoYx-2BU88-2B1jnUM7Rvx0OUKMS78f3Zn78u-2BsRXpXcIeS3oh8u5YEWIgu-2BvmYwr4GzDvZ-2FfPGPw-3D-3D_rhOy4YLBxOPOohPdjw4-2BtC7rnGr1J9pkUkX-2FFAaRhBNGCYbL1pm-2BUp3pjkSSXmHO-2BSjQS-2FE3plG0DCCI1oibkl0LxlIQb9dTYPF4vmb8eL-2BKHmMa4n29yzeX9Omdp4FRxaNqylp18qV7mEVR4tgcnYEbP3NAGNLmVkpzBf54zSPTdR042egG8PZ1NFuNe07WxOueRlqGCUDBmdpJLLQXWTcgKKH6Jy8d3hV6xzMh7ZI-3D> > _____________________________________________ Ntop-misc mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop-misc > <https://u3172041.ct.sendgrid.net/wf/click?upn=UmVayPZKpIdftgoYx-2BU88-2B1jnUM7Rvx0OUKMS78f3Zn78u-2BsRXpXcIeS3oh8u5YEWIgu-2BvmYwr4GzDvZ-2FfPGPw-3D-3D_rhOy4YLBxOPOohPdjw4-2BtC7rnGr1J9pkUkX-2FFAaRhBNGCYbL1pm-2BUp3pjkSSXmHO-2BSjQS-2FE3plG0DCCI1oibkggTxtnwUgQbvQfoUZ4PJ3iRRQrj7Cv76t4mq3kXAvhUfGo1nHWPwMVoNzYiq4zCLTpES6UtoOSgrTAAtU48bUAbpx7MYqLXnLgKJ-2Fe0bn6-2BKAEY3z2lBydmzywHZvMPtvEUmWddf-2FY-2BMahf8-2BPmDXk-3D> > _____________________________________________ Ntop-misc mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop-misc > <https://u3172041.ct.sendgrid.net/wf/click?upn=UmVayPZKpIdftgoYx-2BU88-2B1jnUM7Rvx0OUKMS78f3Zn78u-2BsRXpXcIeS3oh8u5YEWIgu-2BvmYwr4GzDvZ-2FfPGPw-3D-3D_rhOy4YLBxOPOohPdjw4-2BtC7rnGr1J9pkUkX-2FFAaRhBNGCYbL1pm-2BUp3pjkSSXmHO-2BSjQS-2FE3plG0DCCI1oibkkS5-2B5-2BlRRt4m06D-2BxFNDLmZTi9X9X2noegzhZfqTuMURUePrmRsu8xskSMGbxNAiMYVhH4LR0IoRTvB4XfWFg9iq7HkpT2ooPUzytJvOOihvstf2ahHU-2BxZhaIZb6GdTU1pqJOqT9UI82hZaOvcRmQ-3D> > _______________________________________________ > Ntop-misc mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop-misc
signature.asc
Description: Message signed with OpenPGP using GPGMail
_______________________________________________ Ntop-misc mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop-misc
