-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi all, Hi Luca,
I have a further question regarding the memory management of nprobe: How is the memory management handled? I'm asking that question, because I'm experiencing the following: Firstly, nprobe is started and it consumes only about one percent of memory, because there isn't any severe traffic. Then I'm starting my traffic generators which generates around 100k flows per second (as Maurizio had outlined in the previous mail). This leads of course to a bigger memory usage. However, if the traffic generation has finished and there is no traffic again, the memory consumption stays as it was. So my question: why does nprobe not freeing the unused memory? I guess this memory growth is according to longer bucket lists, since the hash size stays constant. As mentioned before, I'm also experiencing very long lists (19-31 max. bucket search). Furthermore, if I run the traffic generation for a very long time, nprobes memory consumption is monotonically growing at a low pace till it runs out of memory. Some details of the configuration: nprobe v.6.3.0 ($Revision: 1947 $) for i686-pc-linux-gnu - -t=10 - -d=10 - -l=1 - -s=2 - -V=5 - -w=8388608 - -M=8000000 - -e=0 Thanks for your comments and explanation of that. Cheers, Dani On 04/20/2011 10:20 AM, Maurizio Molina wrote: > Hi Luca, > our tests are trying to put as much stress as we can on nprobe from a > flows-per-second point of view, and since we don't have dedicated HW > generators we generate 1 packet flows, TCP, of the smallest size > possible. The flows/s (and pk/s as well, since flows are 1 pkts long > flows) are in these tests 100K/s. There are six virtual machines > generating this traffic, so 6 different source IPs. Each of them chooses > randomly a destination port Xi (i=1,....,6), then generates 60,000 > packets incrementing each time deterministically the destination port by > one (Di, Di+1, Di+2, ...), then the loop repeats itself with every VM > incrementing deterministically Xi by one. > So, at the end, two nested loops on the src and dst ports. This means > actually low "entropy" (a lot of subsequent pkt headers will differ by > one bit only) but this is similar to what happens e.g. in a port > scanning. A "good" hash should distribute randomly also with 1 bit > variations, but I know this is difficult. What hash is nprobe using, BTW? > Here below, the probability of long lists that we expect with a > perfectly uniform hash distribution for that load, hash size of 2^23, > flows kept in the flow cache for 11 seconds (10 sec idle tout and 1 sec > scan cycle). Bucket sizes of 16 elements are clearly indicating an > uneven distribution of the hash. > Cheers, > Maurizio > > K P(list>k) > 0 1.00E+00 > 1 1.16E-01 > 2 7.54E-03 > 3 3.30E-04 > 4 1.08E-05 > 5 2.83E-07 > 6 6.19E-09 > 7 1.16E-10 > 8 1.90E-12 > 9 2.77E-14 > 10 3.63E-16 > 11 4.33E-18 > 12 4.73E-20 > 13 4.77E-22 > 14 4.47E-24 > 15 3.91E-26 > 16 3.20E-28 > 17 2.47E-30 > 18 1.80E-32 > 19 1.24E-34 > 20 8.15E-37 > > > > On 19.04.2011 18:07, Luca Deri wrote: >> Daniel >> it looks your traffic is not easy balance-able then. What kind of >> packets are you using? >> >> Luca >> _______________________________________________ Ntop-misc mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop-misc - -- daniel aschwanden junior engineer open systems ag raeffelstrasse 29 ch-8045 zurich t: +41 44 455 74 96 f: +41 44 455 74 01 [email protected] http://www.open.ch -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iF4EAREIAAYFAk25MSEACgkQxIzmH53Qg6EQSQEAghNTyKmejaAjZdhiI/25xrFV Ar3F3tUuRcgtW0OTSUgA/2c/npNbNQa8SASgW3lsHEnSJIZGER66QhZEh06o9wtx =Iy93 -----END PGP SIGNATURE----- _______________________________________________ Ntop-misc mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop-misc
