Hello, I’m new to nProbe and trying to figure out an issue I’m experiencing with missing flow records when using it in collector mode.
Specifically, I am running nProbe in collector mode on an ubuntu 14.04.2 box as follows: nprobe -i none -n none --collector-port 2055 -P ~/data/flow_data & A colleague is sending netflow records using a netflow generator on a different network. nProbe successfully collects and processes all the flows that are sent, but I am only seeing about half the number of netflow records in the dumped output files. We’ve run three separate tests using different nProbe options but in each case we’re missing a lot of records in the output: Test 1) 5110 sent, 5110 collected / processed by nProbe, 2340 output records Test 2) 2000 sent, 2000 collected / processed by nProbe, 1057 output records Test 3) 2000 sent, 2000 collected / processed by nProbe, 1055 output records In the above tests I tried different options for —flow-version. I also tried a separate test dumping to a SQLite file, which resulted in only 412 rows/records for 2000 collected/processed... The only possibilities I can think of are that nProbe is either merging some of the incoming flow records, or the missing records are not being written to file due to a particular configuration setting. Reading through the nProbe manual, I’m wondering if one or more of the following options could help: -F: frequency at which files are dumped to disk (default: 60 secs; our tests run for about 10 mins) -e: delay between flow exports (although doesn’t seem relevant to collector mode) -z: min TCP flow size (again doesn’t seem relevant to collector mode) —disable-cache: export flows immediately in collector mode, rather than put in the cache (promising?) Would appreciate any thoughts / suggestions on the above. I’m including an output log from one of the tests below (didn't run in full verbose mode, apologies…). Many thanks, Nick - 13/Jan/2016 01:06:15 [nprobe.c:3176] Valid nProbe license found 13/Jan/2016 01:06:15 [nprobe.c:4576] WARNING: The output interfaceId is set to 0: did you forget to use -Q perhaps ? 13/Jan/2016 01:06:15 [nprobe.c:4579] WARNING: The input interfaceId is set to 0: did you forget to use -u perhaps ? 13/Jan/2016 01:06:15 [nprobe.c:4671] Welcome to nProbe v.7.3.151217 ($Revision: 4740 $) for x86_64-unknown-linux-gnu with native PF_RING acceleration 13/Jan/2016 01:06:15 [nprobe.c:4681] Running on Ubuntu 14.04.2 LTS 13/Jan/2016 01:06:15 [nprobe.c:4706] [LICENSE] nProbe Edition: Standard [without PF_RING Acceleration] 13/Jan/2016 01:06:15 [nprobe.c:4736] [LICENSE] Maintenance is available until Thu Dec 29 17:48:53 2016 [351 days left] 13/Jan/2016 01:06:15 [nprobe.c:4775] Dumping flow files every 60 sec into directory /home/ubuntu/data/flow_data/NFv9_default_template 13/Jan/2016 01:06:15 [nprobe.c:6699] Welcome to nProbe v.7.3.151217 for x86_64-unknown-linux-gnu 13/Jan/2016 01:06:15 [nprobe.c:5951] WARNING: You selected v9/IPFIX without specifying a template (-T). 13/Jan/2016 01:06:15 [nprobe.c:5952] WARNING: The default template will be used 13/Jan/2016 01:06:15 [nprobe.c:5957] Using NetFlow Packet Payload Len: 1472 13/Jan/2016 01:06:15 [plugin.c:1009] 0 plugin(s) enabled 13/Jan/2016 01:06:15 [nprobe.c:6354] Each flow is 85 bytes long 13/Jan/2016 01:06:15 [nprobe.c:6355] The # packets per flow has been set to 16 13/Jan/2016 01:06:15 [nprobe.c:6374] Non IPv4/v6 traffic is discarded according to the template 13/Jan/2016 01:06:15 [util.c:431] GeoIP: loaded AS config file GeoIPASNum.dat 13/Jan/2016 01:06:15 [util.c:441] GeoIP: loaded AS IPv6 config file GeoIPASNumv6.dat 13/Jan/2016 01:06:15 [nprobe.c:5243] Using packet capture length 128 13/Jan/2016 01:06:15 [nprobe.c:7001] Not capturing packet from interface (collector mode) 13/Jan/2016 01:06:15 [collect.c:145] Flow collector listening on port 2055 (IPv4/v6) 13/Jan/2016 01:06:15 [nprobe.c:7213] nProbe started successfully 14/Jan/2016 00:38:02 [cache.c:1210] Redis Cache [0 total/0.0 get/sec][0 total/0.0 set/sec] 14/Jan/2016 00:38:02 [nprobe.c:394] Received shutdown request... [signal: 15] 14/Jan/2016 00:38:05 [cache.c:1210] Redis Cache [0 total/0.0 get/sec][0 total/0.0 set/sec] 14/Jan/2016 00:38:05 [nprobe.c:2503] Processed packets: 0 (max bucket search: 9) 14/Jan/2016 00:38:05 [nprobe.c:2486] Fragment queue length: 0 14/Jan/2016 00:38:05 [nprobe.c:2512] Flow export stats: [0 bytes/0 pkts][0 flows/0 pkts sent] 14/Jan/2016 00:38:05 [nprobe.c:2519] Flow collection: [collected pkts: 2000][processed flows: 2000] 14/Jan/2016 00:38:05 [nprobe.c:2522] Flow drop stats: [0 bytes/0 pkts][0 flows] 14/Jan/2016 00:38:05 [nprobe.c:2527] Total flow stats: [0 bytes/0 pkts][0 flows/0 pkts sent] _______________________________________________ Ntop-misc mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop-misc
