Hi Dan I have added a patch on nProbe for handling this case. The new ASA flows do not contain the number of packets (just bytes) and export the bytes to using the IN/OUT bytes like they should do. Cisco invented NetFlow and now they are re-inventing it.
Overnight a new nProbe package containing the fix will be build. Thanks for your support. Regards Luca On 08 Jul 2014, at 18:30, Dan Curfman <[email protected]> wrote: > Here is the nProbe startup log > > 02/Jul/2014 16:21:50 [nprobe.c:5943] ERROR: Invalid nProbe license > (/etc/nprobe.license) [Missing license file] > 02/Jul/2014 16:21:50 [nprobe.c:5953] ERROR: for 68B221397A05A201 > 02/Jul/2014 16:21:50 [nprobe.c:5958] ERROR: > *************************************************** > 02/Jul/2014 16:21:50 [nprobe.c:5959] ERROR: ** > ** > 02/Jul/2014 16:21:50 [nprobe.c:5960] ERROR: ** Switching to DEMO MODE due to > license error ** > 02/Jul/2014 16:21:50 [nprobe.c:5961] ERROR: ** > ** > 02/Jul/2014 16:21:50 [nprobe.c:5962] ERROR: ** Create your nProbe license at > ** > 02/Jul/2014 16:21:50 [nprobe.c:5963] ERROR: ** > http://www.nmon.net/mklicense/ ** > 02/Jul/2014 16:21:50 [nprobe.c:5964] ERROR: ** > ** > 02/Jul/2014 16:21:50 [nprobe.c:5965] ERROR: > *************************************************** > 02/Jul/2014 16:21:50 [nprobe.c:5985] ERROR: > *************************************************************** > 02/Jul/2014 16:21:50 [nprobe.c:5986] ERROR: * NOTE: This is a DEMO version > limited to 25000 flows export. * > 02/Jul/2014 16:21:50 [nprobe.c:5987] ERROR: > *************************************************************** > 02/Jul/2014 16:21:50 [plugin.c:161] No plugins found in ./plugins > 02/Jul/2014 16:21:50 [nprobe.c:4037] WARNING: The output interfaceId is set > to 0: did you forget to use -Q perhaps ? > 02/Jul/2014 16:21:50 [nprobe.c:4040] WARNING: The input interfaceId is set to > 0: did you forget to use -u perhaps ? > 02/Jul/2014 16:21:50 [nprobe.c:4100] Welcome to nprobe v.6.16.140702 > ($Revision: 4232 $) for x86_64-unknown-linux-gnu with native PF_RING > acceleration > 02/Jul/2014 16:21:50 [nprobe.c:4118] nProbe SystemId: 68B221397A05A201 > 02/Jul/2014 16:21:50 [nprobe.c:4133] Tracing enabled > 02/Jul/2014 16:21:50 [nprobe.c:4171] WARNING: -n parameter is missing. > 127.0.0.1:2055 will be used. > 02/Jul/2014 16:21:50 [nprobe.c:2678] Exporting flows towards 127.0.0.1:2055 > using UDP > 02/Jul/2014 16:21:50 [smtpPlugin.c:127] Initialized SMTP plugin > 02/Jul/2014 16:21:50 [mysqlPlugin.c:117] Initialized MySQL plugin > 02/Jul/2014 16:21:50 [plugins/rtpPlugin.c:118] Initializing RTP plugin > 02/Jul/2014 16:21:50 [sipPlugin.c:262] Initialized SIP plugin > 02/Jul/2014 16:21:50 [sipPlugin.c:291] Initialized SIP plugin > 02/Jul/2014 16:21:50 [gtpv2Plugin.c:126] Initialized GTPv2 plugin > 02/Jul/2014 16:21:50 [dbPlugin.c:78] Initializing DB plugin > 02/Jul/2014 16:21:50 [bgpPlugin.c:376] BGP plugin is disabled (--bgp-port has > not been specified) > 02/Jul/2014 16:21:50 [radiusPlugin.c:133] Initialized Radius plugin > 02/Jul/2014 16:21:50 [dnsPlugin.c:101] Initialized DNS plugin > 02/Jul/2014 16:21:50 [gtpv1Plugin.c:129] Initialized GTPv1 plugin > 02/Jul/2014 16:21:50 [gtpv0Plugin.c:103] Initialized GTPv0 plugin > 02/Jul/2014 16:21:50 [httpPlugin.c:490] Initialized HTTP plugin > 02/Jul/2014 16:21:50 [nflitePlugin.c:901] [NFLite] Initialized NetFlow-Lite > plugin > 02/Jul/2014 16:21:50 [plugin.c:225] 13 plugin(s) loaded [12 delete][11 > packet]. > 02/Jul/2014 16:21:50 [nprobe.c:6003] Welcome to nprobe v.6.16.140702 for > x86_64-unknown-linux-gnu > 02/Jul/2014 16:21:50 [nprobe.c:5228] Compiling flow templates... > 02/Jul/2014 16:21:50 [plugin.c:797] Scanning plugin SMTP Protocol > 02/Jul/2014 16:21:50 [plugin.c:797] Scanning plugin MySQL Plugin > 02/Jul/2014 16:21:50 [plugin.c:797] Scanning plugin RTP Plugin > 02/Jul/2014 16:21:50 [plugin.c:797] Scanning plugin SIP Plugin > 02/Jul/2014 16:21:50 [plugin.c:797] Scanning plugin GTPv2 Signaling Protocol > 02/Jul/2014 16:21:50 [plugin.c:797] Scanning plugin MySQL DB > 02/Jul/2014 16:21:50 [plugin.c:797] Scanning plugin BGP Update Listener > 02/Jul/2014 16:21:50 [plugin.c:797] Scanning plugin Radius Protocol > 02/Jul/2014 16:21:50 [plugin.c:797] Scanning plugin DNS Protocol > 02/Jul/2014 16:21:50 [plugin.c:797] Scanning plugin GTPv1 Signaling Protocol > 02/Jul/2014 16:21:50 [plugin.c:797] Scanning plugin GTPv0 Signaling Protocol > 02/Jul/2014 16:21:50 [plugin.c:797] Scanning plugin HTTP Protocol > 02/Jul/2014 16:21:50 [plugin.c:797] Scanning plugin Netflow-Lite Plugin > 02/Jul/2014 16:21:50 [plugin.c:931] 0 plugin(s) enabled > Error Opening file /usr/local/nprobe/GeoIPASNum.dat > 02/Jul/2014 16:21:50 [util.c:310] WARNING: Unable to load AS file > /usr/local/nprobe/GeoIPASNum.dat. AS support disabled > Error Opening file /usr/local/nprobe/GeoIPASNumv6.dat > 02/Jul/2014 16:21:50 [util.c:319] WARNING: Unable to load AS IPv6 file > /usr/local/nprobe/GeoIPASNumv6.dat. AS IPv6 support disabled > 02/Jul/2014 16:21:50 [nprobe.c:6179] IPv6 traffic will NOT be > exported/accounted by this probe > 02/Jul/2014 16:21:50 [nprobe.c:6180] due to configuration options (e.g. use > NetFlow v9) > 02/Jul/2014 16:21:50 [nprobe.c:6183] The flows hash has 131072 buckets > 02/Jul/2014 16:21:50 [nprobe.c:6185] Flows older than 120 seconds will be > exported > 02/Jul/2014 16:21:50 [nprobe.c:6188] Flows inactive for at least 30 seconds > will be exported > 02/Jul/2014 16:21:50 [nprobe.c:6191] Expired flows will not be queued for > more than 30 seconds > 02/Jul/2014 16:21:50 [nprobe.c:6198] Exported flows with engineType 0 and > engineId 14 > 02/Jul/2014 16:21:50 [nprobe.c:6220] TCP TOS will be ignored and set to 0. > 02/Jul/2014 16:21:50 [nprobe.c:6225] Flows ASs will not be computed > 02/Jul/2014 16:21:50 [nprobe.c:6238] After 1 flow packets are sent, we'll > delay at least 1 ms > 02/Jul/2014 16:21:50 [nprobe.c:6258] Flows will be emitted in NetFlow 5 format > 02/Jul/2014 16:21:50 [nprobe.c:6288] Flow input interface index is set to 0 > 02/Jul/2014 16:21:50 [nprobe.c:6294] Flow output interface index is set to 0 > 02/Jul/2014 16:21:50 [util.c:3601] Succesfully created ZMQ endpoint > tcp://*:5556 > 02/Jul/2014 16:21:50 [util.c:2751] nProbe changed user to 'nobody' > 02/Jul/2014 16:21:50 [plugin.c:760] Disabling plugin SMTP Protocol (no > template is using it) > 02/Jul/2014 16:21:50 [plugin.c:760] Disabling plugin MySQL Plugin (no > template is using it) > 02/Jul/2014 16:21:50 [plugin.c:760] Disabling plugin RTP Plugin (no template > is using it) > 02/Jul/2014 16:21:50 [plugin.c:760] Disabling plugin SIP Plugin (no template > is using it) > 02/Jul/2014 16:21:50 [plugin.c:760] Disabling plugin GTPv2 Signaling Protocol > (no template is using it) > 02/Jul/2014 16:21:50 [plugin.c:760] Disabling plugin MySQL DB (no template is > using it) > 02/Jul/2014 16:21:50 [plugin.c:760] Disabling plugin BGP Update Listener (no > template is using it) > 02/Jul/2014 16:21:50 [plugin.c:760] Disabling plugin Radius Protocol (no > template is using it) > 02/Jul/2014 16:21:50 [plugin.c:760] Disabling plugin DNS Protocol (no > template is using it) > 02/Jul/2014 16:21:50 [plugin.c:760] Disabling plugin GTPv1 Signaling Protocol > (no template is using it) > 02/Jul/2014 16:21:50 [plugin.c:760] Disabling plugin GTPv0 Signaling Protocol > (no template is using it) > 02/Jul/2014 16:21:50 [plugin.c:760] Disabling plugin HTTP Protocol (no > template is using it) > 02/Jul/2014 16:21:50 [plugin.c:760] Disabling plugin Netflow-Lite Plugin (no > template is using it) > 02/Jul/2014 16:21:50 [collect.c:96] Created UDP sockets > 02/Jul/2014 16:21:50 [collect.c:155] Flow collector listening on port 4444 > (IPv4/v6) > 02/Jul/2014 16:21:50 [nprobe.c:6427] Starting 1 packet fetch thread(s) > 02/Jul/2014 16:21:50 [engine.c:3146] Starting bucket dequeue thread > > Dan Curfman > Washington United Terminals > > > > -----Original Message----- > From: [email protected] > [mailto:[email protected]] On Behalf Of Dan Curfman > Sent: Tuesday, July 08, 2014 9:29 AM > To: [email protected] > Subject: [Ntop-misc] nProbe + ntopng + cisco ASA > > I am having the same problem that Pablo had in this post, I didn't see any > resolution posted. > > http://listgateway.unipi.it/mailman/private/ntop-misc/2014-January/004092.html > > I'm starting nProbe like this > > nprobe --zmq "tcp://*:5556" --collector-port 4444 -i none -b 2 > > These are the messages we are getting from nProbe. > > 02/Jul/2014 16:10:02 [collect.c:403] Received flow with invalid count > [sentPkts: 0][sentOctets: 0]: discarded [num_flows: 110] > 02/Jul/2014 16:10:02 [collect.c:403] Received flow with invalid count > [sentPkts: 0][sentOctets: 0]: discarded [num_flows: 110] > 02/Jul/2014 16:10:02 [collect.c:403] Received flow with invalid count > [sentPkts: 0][sentOctets: 0]: discarded [num_flows: 110] > > The same lines are repeated over and over, only with incrementing num_flows > count. > > 02/Jul/2014 16:10:07 [collect.c:403] Received flow with invalid count > [sentPkts: 0][sentOctets: 0]: discarded [num_flows: 123] > 02/Jul/2014 16:10:07 [collect.c:403] Received flow with invalid count > [sentPkts: 0][sentOctets: 0]: discarded [num_flows: 123] > 02/Jul/2014 16:10:07 [collect.c:403] Received flow with invalid count > [sentPkts: 0][sentOctets: 0]: discarded [num_flows: 123] > 02/Jul/2014 16:10:07 [collect.c:403] Received flow with invalid count > [sentPkts: 0][sentOctets: 0]: discarded [num_flows: 123] > 02/Jul/2014 16:10:07 [collect.c:403] Received flow with invalid count > [sentPkts: 0][sentOctets: 0]: discarded [num_flows: 123] > 02/Jul/2014 16:10:07 [collect.c:403] Received flow with invalid count > [sentPkts: 0][sentOctets: 0]: discarded [num_flows: 123] > 02/Jul/2014 16:10:07 [collect.c:403] Received flow with invalid count > [sentPkts: 0][sentOctets: 0]: discarded [num_flows: 123] > 02/Jul/2014 16:10:07 [collect.c:403] Received flow with invalid count > [sentPkts: 0][sentOctets: 0]: discarded [num_flows: 123] > 02/Jul/2014 16:10:07 [collect.c:403] Received flow with invalid count > [sentPkts: 0][sentOctets: 0]: discarded [num_flows: 123] > 02/Jul/2014 16:10:07 [collect.c:403] Received flow with invalid count > [sentPkts: 0][sentOctets: 0]: discarded [num_flows: 123] > 02/Jul/2014 16:10:07 [collect.c:403] Received flow with invalid count > [sentPkts: 0][sentOctets: 0]: discarded [num_flows: 123] > 02/Jul/2014 16:10:07 [collect.c:403] Received flow with invalid count > [sentPkts: 0][sentOctets: 0]: discarded [num_flows: 123] > ^C02/Jul/2014 16:10:07 [cache.c:1033] Redis Cache [0 total/0.0 get/sec][0 > total/0.0 set/sec] > 02/Jul/2014 16:10:07 [nprobe.c:386] Received shutdown request... > 02/Jul/2014 16:10:07 [nprobe.c:4232] nProbe is shutting down... > 02/Jul/2014 16:10:07 [nprobe.c:4268] Exporting pending buckets... > 02/Jul/2014 16:10:07 [nprobe.c:4289] Pending buckets have been exported... > 02/Jul/2014 16:10:07 [engine.c:3222] Export thread terminated [exportQueue=0] > 02/Jul/2014 16:10:07 [nprobe.c:4350] Flushing queued flows... > 02/Jul/2014 16:10:07 [nprobe.c:4353] Freeing memory... > 02/Jul/2014 16:10:07 [plugin.c:254] Terminating plugins. > 02/Jul/2014 16:10:07 [cache.c:1033] Redis Cache [0 total/0.0 get/sec][0 > total/0.0 set/sec] > 02/Jul/2014 16:10:07 [nprobe.c:4445] Still allocated 0 hash buckets > 02/Jul/2014 16:10:07 [nprobe.c:2187] Processed packets: 0 (max bucket search: > 0) > 02/Jul/2014 16:10:07 [nprobe.c:2170] Fragment queue length: 0 > 02/Jul/2014 16:10:07 [nprobe.c:2196] Flow export stats: [0 bytes/0 pkts][0 > flows/0 pkts sent] > 02/Jul/2014 16:10:07 [nprobe.c:2203] Flow collection: [collected pkts: > 123][processed flows: 1576] > 02/Jul/2014 16:10:07 [nprobe.c:2206] Flow drop stats: [0 bytes/0 pkts][0 > flows] > 02/Jul/2014 16:10:07 [nprobe.c:2211] Total flow stats: [0 bytes/0 pkts][0 > flows/0 pkts sent] > 02/Jul/2014 16:10:07 [nprobe.c:4458] Cleaning globals > 02/Jul/2014 16:10:07 [nprobe.c:4479] nProbe terminated. > > > The error shows it is discarding the flow data, but the summary shows it is > processing packets, but then nothing shows up in ntopng.. and if I add the > interface on nprobe to look for traffic then I start getting data in ntopng. > so it appears to be an issue with nprobe and the netflow.. but I'm not sure > what to do about it. > > I have a packet capture of the netflows from the ASA available to send to > anyone who might be able to help. > > Let me know if there is any other day that would be helpful in > troubleshooting this. I just installed from the stable repos on July 2nd, so > it should be the most current. > > Thanks, > Dan > > _______________________________________________ > Ntop-misc mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop-misc > _______________________________________________ > Ntop-misc mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop-misc _______________________________________________ Ntop-misc mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop-misc
