Rui
you mean you capture 800Mbit * 16 or 800Mbit in total?
note that splitting traffic across all the cores is a *bad* idea.
Please read my papers. I understand that's not simple to digest it
given that most of the people think in terms of "divide and conquer".
With multicore if you do that (in particular across two physical
processors) you invalidate teh caches, fill us the QPI bus etc etc. so
the worst you can do.
Cheers Luca
On 12/01/2010 11:07 AM, Rui wrote:
On Tue, Nov 30, 2010 at 5:20 PM, Luca Deri <[email protected]> wrote:
Rui
in my tests with 2 cores you can definitively capture at least 1 Gbit (1.48
Mpps). How did you balance irq/threads/pfcount across cores? How did you
start pfcount?
Regards Luca
set irq affinity(the tool is from intel ixgbe driver)
./set_irq_affinity.sh eth4
no rx vectors found on eth4
no tx vectors found on eth4
eth4 mask=1 for /proc/irq/86/smp_affinity
eth4 mask=2 for /proc/irq/87/smp_affinity
eth4 mask=4 for /proc/irq/88/smp_affinity
eth4 mask=8 for /proc/irq/89/smp_affinity
eth4 mask=10 for /proc/irq/90/smp_affinity
eth4 mask=20 for /proc/irq/91/smp_affinity
eth4 mask=40 for /proc/irq/92/smp_affinity
eth4 mask=80 for /proc/irq/93/smp_affinity
eth4 mask=100 for /proc/irq/94/smp_affinity
eth4 mask=200 for /proc/irq/95/smp_affinity
eth4 mask=400 for /proc/irq/96/smp_affinity
eth4 mask=800 for /proc/irq/97/smp_affinity
eth4 mask=1000 for /proc/irq/98/smp_affinity
eth4 mask=2000 for /proc/irq/99/smp_affinity
eth4 mask=4000 for /proc/irq/100/smp_affinity
eth4 mask=8000 for /proc/irq/101/smp_affinity
cat /proc/interrupts |grep eth4
86: 1108 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 IR-PCI-MSI-edge
eth4-TxRx-0
87: 8 967 0 0 0
0 0 0 0 0 0 0
0 0 0 0 IR-PCI-MSI-edge
eth4-TxRx-1
88: 8 0 967 0 0
0 0 0 0 0 0 0
0 0 0 0 IR-PCI-MSI-edge
eth4-TxRx-2
89: 8 0 0 967 0
0 0 0 0 0 0 0
0 0 0 0 IR-PCI-MSI-edge
eth4-TxRx-3
90: 8 0 0 0 968
0 0 0 0 0 0 0
0 0 0 0 IR-PCI-MSI-edge
eth4-TxRx-4
91: 8 0 0 0 0
967 0 0 0 0 0 0
0 0 0 0 IR-PCI-MSI-edge
eth4-TxRx-5
92: 8 0 0 0 0
0 967 0 0 0 0 0
0 0 0 0 IR-PCI-MSI-edge
eth4-TxRx-6
93: 8 0 0 0 0
0 0 967 0 0 0 0
0 0 0 0 IR-PCI-MSI-edge
eth4-TxRx-7
94: 8 0 0 0 0
0 0 0 1009 0 0 0
0 0 0 0 IR-PCI-MSI-edge
eth4-TxRx-8
95: 65741 0 0 0 0
0 0 0 0 14100392 0 0
0 0 0 0 IR-PCI-MSI-edge
eth4-TxRx-9
96: 8 0 0 0 0
0 0 0 0 0 967 0
0 0 0 0 IR-PCI-MSI-edge
eth4-TxRx-10
97: 8 0 0 0 0
0 0 0 0 0 0 967
0 0 0 0 IR-PCI-MSI-edge
eth4-TxRx-11
98: 8 0 0 0 0
0 0 0 0 0 0 0
967 0 0 0 IR-PCI-MSI-edge
eth4-TxRx-12
99: 8 0 0 0 0
0 0 0 0 0 0 0
0 967 0 0 IR-PCI-MSI-edge
eth4-TxRx-13
100: 8 0 0 0 0
0 0 0 0 0 0 0
0 0 967 0 IR-PCI-MSI-edge
eth4-TxRx-14
101: 8 0 0 0 0
0 0 0 0 0 0 0
0 0 0 967 IR-PCI-MSI-edge
eth4-TxRx-15
102: 321857 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 IR-PCI-MSI-edge
eth4:lsc
I start pfcount like below:
./pfcount -e 1 -i e...@0 2> ./pfcount0.log &
./pfcount -e 1 -i e...@1 2> ./pfcount1.log &
./pfcount -e 1 -i e...@2 2> ./pfcount2.log &
./pfcount -e 1 -i e...@3 2> ./pfcount3.log &
./pfcount -e 1 -i e...@4 2> ./pfcount4.log &
./pfcount -e 1 -i e...@5 2> ./pfcount5.log &
./pfcount -e 1 -i e...@6 2> ./pfcount6.log &
./pfcount -e 1 -i e...@7 2> ./pfcount7.log &
./pfcount -e 1 -i e...@8 2> ./pfcount8.log &
./pfcount -e 1 -i e...@9 2> ./pfcount9.log &
./pfcount -e 1 -i e...@10 2> ./pfcount10.log &
./pfcount -e 1 -i e...@11 2> ./pfcount11.log &
./pfcount -e 1 -i e...@12 2> ./pfcount12.log &
./pfcount -e 1 -i e...@13 2> ./pfcount13.log &
./pfcount -e 1 -i e...@14 2> ./pfcount14.log &
./pfcount -e 1 -i e...@15 2> ./pfcount15.log &
today I test a pfring aware ixgbe driver with "transparent_mode=1" and
get a similar result(800Mbps)
On 11/30/2010 08:28 AM, Rui wrote:
hi luca:
I have tuned some network parameters, such as irq affinity,
wmem_default,tcp_window_scale,etc and observed the throughput had
reached 6Gbps+
but with imbalance traffic,pfcount performance is still bad (limited at
800Mbps)
do you think it is normal? (is it too low?)
what else I can do to improve this situation?
I just think that general system optimization only increase the total
throughput but won't be helpful for imbalance traffic?
regards
rui
From: [email protected]
Date: Sun, 28 Nov 2010 11:00:22 +0100
To: [email protected]
Subject: Re: [Ntop-misc] pfcount drop much when (800Mbps) falls at same
RX-queue
Rui
if you out the card in promiscuous move (e.g. via pfcount) you drop
traffic as traffic is not discarded by the NIC. Unless you have balance-able
traffic (i.e. if you send always the same packet it will definitively go
onto the same queue) all you observe is correct. Please read some of my
papers that describe how to optimize the system. Unless you do that the
performance would be poor
Thanks Luca
On Nov 27, 2010, at 3:52 AM, Rui wrote:
hi
today I did a test with pfcount,latest PF_RING and a intel 10G
nic(ixgbe from intel,without PF_RING aware driver,transparent mode=0)
I used a stress test equipment to generate the traffic, they have same
src ip and dst ip, port, so will get same RSS hash result and fall at
the same RX-queue.
I found that performance is bad, pfcount will drop much if the
traffic>800Mbps
the performance is good If I didn't start pfcount. "ifconfig ethx"
indicate no packet drop even that throughput reachs 6Gbps
my machine(HP DL380 G6) has 16 cores, XEON 2.5Ghz, hyperthreading,the
10G nic was installed at a PCI-e 4X slot(I don't have 8X slot at the
moment)
any comment about this situation? is it normal?
thanks
rui
_______________________________________________
Ntop-misc mailing list
[email protected]
http://listgateway.unipi.it/mailman/listinfo/ntop-misc
---
Keep looking, don't settle - Steve Jobs
_______________________________________________
Ntop-misc mailing list
[email protected]
http://listgateway.unipi.it/mailman/listinfo/ntop-misc
_______________________________________________
Ntop-misc mailing list
[email protected]
http://listgateway.unipi.it/mailman/listinfo/ntop-misc
_______________________________________________
Ntop-misc mailing list
[email protected]
http://listgateway.unipi.it/mailman/listinfo/ntop-misc
_______________________________________________
Ntop-misc mailing list
[email protected]
http://listgateway.unipi.it/mailman/listinfo/ntop-misc
|
_______________________________________________
Ntop-misc mailing list
[email protected]
http://listgateway.unipi.it/mailman/listinfo/ntop-misc
