Daniel if you have 100k concurrent flows, -w 256000 will do the job Luca
On Apr 19, 2011, at 3:24 PM, Daniel Aschwanden wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > Hi all, > > I'm still struggling with the hash of nprobe (version 6.3.0). > > Luca suggested to increase the hash size, if there is a max. bucket > search over 8. > > I've in total around 550'000 active flow buckets and I see hardly any > difference in terms of max. bucket search, if I vary the hash size from > 2^17 to 2^23. > > ====================================================================== > with -w 131072, I get the following output from nprobe: > ====================================================================== > 19/Apr/2011 15:00:41 [nprobe.c:1459] --------------------------------- > 19/Apr/2011 15:00:41 [nprobe.c:1460] Average traffic: [118.154 K pps][73 > Mb/sec] > 19/Apr/2011 15:00:41 [nprobe.c:1465] Current traffic: [107.855 K pps][67 > Mb/sec] > 19/Apr/2011 15:00:41 [nprobe.c:1471] Current flow export rate: [58351.3 > flows/sec] > 19/Apr/2011 15:00:41 [nprobe.c:1474] Flow drops: [export queue too > long=0][too many flows=0] > 19/Apr/2011 15:00:41 [nprobe.c:1480] Flow Buckets: > [active=570228][allocated=570228][toBeExported=0][frags=0] > 19/Apr/2011 15:00:41 [nprobe.c:1516] Processed packets: 15478149 (max > bucket search: 16) > 19/Apr/2011 15:00:41 [nprobe.c:1519] Flow Collection: [collected pkts: > 0][processed flows: 0] > 19/Apr/2011 15:00:41 [nprobe.c:222] Packet stats: 15478229/0 pkts > rcvd/dropped [0.0%] [Last 323545/0 pkts rcvd/dropped] > 19/Apr/2011 15:00:43 [nprobe.c:1459] --------------------------------- > 19/Apr/2011 15:00:43 [nprobe.c:1460] Average traffic: [118.735 K pps][73 > Mb/sec] > 19/Apr/2011 15:00:43 [nprobe.c:1465] Current traffic: [156.785 K pps][97 > Mb/sec] > 19/Apr/2011 15:00:43 [nprobe.c:1471] Current flow export rate: [59780.5 > flows/sec] > 19/Apr/2011 15:00:43 [nprobe.c:1474] Flow drops: [export queue too > long=0][too many flows=0] > 19/Apr/2011 15:00:43 [nprobe.c:1480] Flow Buckets: > [active=562104][allocated=607792][toBeExported=45688][frags=0] > 19/Apr/2011 15:00:43 [nprobe.c:1516] Processed packets: 15791715 (max > bucket search: 15) > 19/Apr/2011 15:00:43 [nprobe.c:1519] Flow Collection: [collected pkts: > 0][processed flows: 0] > 19/Apr/2011 15:00:43 [nprobe.c:222] Packet stats: 15791810/0 pkts > rcvd/dropped [0.0%] [Last 313581/0 pkts rcvd/dropped] > > ====================================================================== > and with -w 8388608: > ====================================================================== > 19/Apr/2011 15:15:22 [nprobe.c:1459] --------------------------------- > 19/Apr/2011 15:15:22 [nprobe.c:1460] Average traffic: [117.322 K pps][72 > Mb/sec] > 19/Apr/2011 15:15:22 [nprobe.c:1465] Current traffic: [110.342 K pps][69 > Mb/sec] > 19/Apr/2011 15:15:22 [nprobe.c:1471] Current flow export rate: [64136.3 > flows/sec] > 19/Apr/2011 15:15:22 [nprobe.c:1474] Flow drops: [export queue too > long=0][too many flows=0] > 19/Apr/2011 15:15:22 [nprobe.c:1480] Flow Buckets: > [active=540968][allocated=540968][toBeExported=0][frags=0] > 19/Apr/2011 15:15:22 [nprobe.c:1516] Processed packets: 11614885 (max > bucket search: 19) > 19/Apr/2011 15:15:22 [nprobe.c:1519] Flow Collection: [collected pkts: > 0][processed flows: 0] > 19/Apr/2011 15:15:22 [nprobe.c:222] Packet stats: 11615012/0 pkts > rcvd/dropped [0.0%] [Last 331026/0 pkts rcvd/dropped] > 19/Apr/2011 15:15:25 [nprobe.c:1459] --------------------------------- > 19/Apr/2011 15:15:25 [nprobe.c:1460] Average traffic: [117.143 K pps][72 > Mb/sec] > 19/Apr/2011 15:15:25 [nprobe.c:1465] Current traffic: [111.232 K pps][69 > Mb/sec] > 19/Apr/2011 15:15:25 [nprobe.c:1471] Current flow export rate: [62210.0 > flows/sec] > 19/Apr/2011 15:15:25 [nprobe.c:1474] Flow drops: [export queue too > long=0][too many flows=0] > 19/Apr/2011 15:15:25 [nprobe.c:1480] Flow Buckets: > [active=521688][allocated=521688][toBeExported=0][frags=0] > 19/Apr/2011 15:15:25 [nprobe.c:1516] Processed packets: 11948651 (max > bucket search: 16) > 19/Apr/2011 15:15:25 [nprobe.c:1519] Flow Collection: [collected pkts: > 0][processed flows: 0] > 19/Apr/2011 15:15:25 [nprobe.c:222] Packet stats: 11948778/0 pkts > rcvd/dropped [0.0%] [Last 333766/0 pkts rcvd/dropped] > > What should I do to decrease the max. bucket search. Do I understand it > correctly, that the max. bucket search is just a list of the hash > collisions? If yes, a bigger hash size should normally lead to less > collisions if the hash is well distributed across the hash domain? > > I'm really glad about any remarks and suggestions. > > Cheers, > Dani Aschwanden > > - -- > daniel aschwanden > junior engineer > > open systems ag > raeffelstrasse 29 > ch-8045 zurich > t: +41 44 455 74 00 > f: +41 44 455 74 01 > [email protected] > > http://www.open.ch > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.10 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iF4EAREIAAYFAk2tjQ8ACgkQxIzmH53Qg6FomwD/WuGVICd8FBw8Mk3layzZuJQ4 > u5/DOunaoknh860fuhAA/0yit6ucLBjEI8M37A+uwvxV6MjZdyLR9bpqOnTenfI0 > =uIUz > -----END PGP SIGNATURE----- > _______________________________________________ > Ntop-misc mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop-misc --- If you can not measure it, you can not improve it - Lord Kelvin _______________________________________________ Ntop-misc mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop-misc
