On Tue, Nov 30, 2010 at 5:20 PM, Luca Deri <[email protected]> wrote:
> Rui
> in my tests with 2 cores you can definitively capture at least 1 Gbit (1.48
> Mpps). How did you balance irq/threads/pfcount across cores? How did you
> start pfcount?
>
> Regards Luca
>
set irq affinity(the tool is from intel ixgbe driver)
./set_irq_affinity.sh eth4
no rx vectors found on eth4
no tx vectors found on eth4
eth4 mask=1 for /proc/irq/86/smp_affinity
eth4 mask=2 for /proc/irq/87/smp_affinity
eth4 mask=4 for /proc/irq/88/smp_affinity
eth4 mask=8 for /proc/irq/89/smp_affinity
eth4 mask=10 for /proc/irq/90/smp_affinity
eth4 mask=20 for /proc/irq/91/smp_affinity
eth4 mask=40 for /proc/irq/92/smp_affinity
eth4 mask=80 for /proc/irq/93/smp_affinity
eth4 mask=100 for /proc/irq/94/smp_affinity
eth4 mask=200 for /proc/irq/95/smp_affinity
eth4 mask=400 for /proc/irq/96/smp_affinity
eth4 mask=800 for /proc/irq/97/smp_affinity
eth4 mask=1000 for /proc/irq/98/smp_affinity
eth4 mask=2000 for /proc/irq/99/smp_affinity
eth4 mask=4000 for /proc/irq/100/smp_affinity
eth4 mask=8000 for /proc/irq/101/smp_affinity
cat /proc/interrupts |grep eth4
86: 1108 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 IR-PCI-MSI-edge
eth4-TxRx-0
87: 8 967 0 0 0
0 0 0 0 0 0 0
0 0 0 0 IR-PCI-MSI-edge
eth4-TxRx-1
88: 8 0 967 0 0
0 0 0 0 0 0 0
0 0 0 0 IR-PCI-MSI-edge
eth4-TxRx-2
89: 8 0 0 967 0
0 0 0 0 0 0 0
0 0 0 0 IR-PCI-MSI-edge
eth4-TxRx-3
90: 8 0 0 0 968
0 0 0 0 0 0 0
0 0 0 0 IR-PCI-MSI-edge
eth4-TxRx-4
91: 8 0 0 0 0
967 0 0 0 0 0 0
0 0 0 0 IR-PCI-MSI-edge
eth4-TxRx-5
92: 8 0 0 0 0
0 967 0 0 0 0 0
0 0 0 0 IR-PCI-MSI-edge
eth4-TxRx-6
93: 8 0 0 0 0
0 0 967 0 0 0 0
0 0 0 0 IR-PCI-MSI-edge
eth4-TxRx-7
94: 8 0 0 0 0
0 0 0 1009 0 0 0
0 0 0 0 IR-PCI-MSI-edge
eth4-TxRx-8
95: 65741 0 0 0 0
0 0 0 0 14100392 0 0
0 0 0 0 IR-PCI-MSI-edge
eth4-TxRx-9
96: 8 0 0 0 0
0 0 0 0 0 967 0
0 0 0 0 IR-PCI-MSI-edge
eth4-TxRx-10
97: 8 0 0 0 0
0 0 0 0 0 0 967
0 0 0 0 IR-PCI-MSI-edge
eth4-TxRx-11
98: 8 0 0 0 0
0 0 0 0 0 0 0
967 0 0 0 IR-PCI-MSI-edge
eth4-TxRx-12
99: 8 0 0 0 0
0 0 0 0 0 0 0
0 967 0 0 IR-PCI-MSI-edge
eth4-TxRx-13
100: 8 0 0 0 0
0 0 0 0 0 0 0
0 0 967 0 IR-PCI-MSI-edge
eth4-TxRx-14
101: 8 0 0 0 0
0 0 0 0 0 0 0
0 0 0 967 IR-PCI-MSI-edge
eth4-TxRx-15
102: 321857 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 IR-PCI-MSI-edge
eth4:lsc
I start pfcount like below:
./pfcount -e 1 -i e...@0 2> ./pfcount0.log &
./pfcount -e 1 -i e...@1 2> ./pfcount1.log &
./pfcount -e 1 -i e...@2 2> ./pfcount2.log &
./pfcount -e 1 -i e...@3 2> ./pfcount3.log &
./pfcount -e 1 -i e...@4 2> ./pfcount4.log &
./pfcount -e 1 -i e...@5 2> ./pfcount5.log &
./pfcount -e 1 -i e...@6 2> ./pfcount6.log &
./pfcount -e 1 -i e...@7 2> ./pfcount7.log &
./pfcount -e 1 -i e...@8 2> ./pfcount8.log &
./pfcount -e 1 -i e...@9 2> ./pfcount9.log &
./pfcount -e 1 -i e...@10 2> ./pfcount10.log &
./pfcount -e 1 -i e...@11 2> ./pfcount11.log &
./pfcount -e 1 -i e...@12 2> ./pfcount12.log &
./pfcount -e 1 -i e...@13 2> ./pfcount13.log &
./pfcount -e 1 -i e...@14 2> ./pfcount14.log &
./pfcount -e 1 -i e...@15 2> ./pfcount15.log &
today I test a pfring aware ixgbe driver with "transparent_mode=1" and
get a similar result(800Mbps)
>
> On 11/30/2010 08:28 AM, Rui wrote:
>
> hi luca:
> I have tuned some network parameters, such as irq affinity,
> wmem_default,tcp_window_scale,etc and observed the throughput had
> reached 6Gbps+
> but with imbalance traffic,pfcount performance is still bad (limited at
> 800Mbps)
> do you think it is normal? (is it too low?)
> what else I can do to improve this situation?
> I just think that general system optimization only increase the total
> throughput but won't be helpful for imbalance traffic?
> regards
> rui
>
>
> From: [email protected]
> Date: Sun, 28 Nov 2010 11:00:22 +0100
> To: [email protected]
> Subject: Re: [Ntop-misc] pfcount drop much when (800Mbps) falls at same
> RX-queue
> Rui
> if you out the card in promiscuous move (e.g. via pfcount) you drop
> traffic as traffic is not discarded by the NIC. Unless you have balance-able
> traffic (i.e. if you send always the same packet it will definitively go
> onto the same queue) all you observe is correct. Please read some of my
> papers that describe how to optimize the system. Unless you do that the
> performance would be poor
> Thanks Luca
> On Nov 27, 2010, at 3:52 AM, Rui wrote:
>
>
> hi
> today I did a test with pfcount,latest PF_RING and a intel 10G
> nic(ixgbe from intel,without PF_RING aware driver,transparent mode=0)
> I used a stress test equipment to generate the traffic, they have same
> src ip and dst ip, port, so will get same RSS hash result and fall at
> the same RX-queue.
> I found that performance is bad, pfcount will drop much if the
> traffic>800Mbps
> the performance is good If I didn't start pfcount. "ifconfig ethx"
> indicate no packet drop even that throughput reachs 6Gbps
> my machine(HP DL380 G6) has 16 cores, XEON 2.5Ghz, hyperthreading,the
> 10G nic was installed at a PCI-e 4X slot(I don't have 8X slot at the
> moment)
> any comment about this situation? is it normal?
> thanks
> rui
> _______________________________________________
> Ntop-misc mailing list
> [email protected]
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
> ---
> Keep looking, don't settle - Steve Jobs
> _______________________________________________
> Ntop-misc mailing list
> [email protected]
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
> _______________________________________________
> Ntop-misc mailing list
> [email protected]
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
> _______________________________________________
> Ntop-misc mailing list
> [email protected]
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
_______________________________________________
Ntop-misc mailing list
[email protected]
http://listgateway.unipi.it/mailman/listinfo/ntop-misc
