Pablo, please provide me a pcap file with (full packets size) the ASA template+flows so I can see what happens
Thanks Luca On 04 Jan 2014, at 14:15, Pablo Destéfanis <[email protected]> wrote: > I have found that every NetFlow packet arriving to nprobe elicits this > message: > > [collect.c:326] Received flow with invalid count [sentPkts: 0][sentOctets: > 0]: discarded > > This starts after the templates are received. > > I'm looking for information online, and finding next to none. When I look at > the counters in Wireshark, the numbers do make sense (in the interpreted > packets), so I am starting to think it is a something with the way nprobe > decodes the packets based on the ASA 5510 templates. > > Have you seen this before? Any ideas where to look? > > Thank you, > > Pablo > > > On Fri, Jan 3, 2014 at 5:02 PM, Pablo Destéfanis <[email protected]> > wrote: > Hello guys, > > I have done a bit more research to check if ASA was sending the template > info. I have set the template timeout at 5 minutes and also directed the > output to Wireshark. > I see first the uninterpreted flows, and then the interpreted NetFlow > packets, including source and destination address, pre/post NAT, etc. While I > did not search for the packets with templates, I'm assuming they were > detected by Wireshark, and then the output was reinterpreted. > > Tried with nrprobe, and flow collection shows processed flows after a while > (initially only shows collected packets) > Flow collection: [collected pkts: 889][processed flows: 12697] > > Still, when connecting from ntopng I get no information. The commands I'm > using are: > > nprobe --zmq "tcp://*:5556" -i none -n none --collector-port 2055 --city-list > /usr/local/nprobe/GeoLiteCity.dat > ntopng -i tcp://127.0.0.1:5556 -m 10.1.0.0/16 > > I see the ports open, and I see ntopng connected to nprobe: > > tcp 0 0 0.0.0.0:5556 0.0.0.0:* > LISTEN 14596/nprobe > tcp 0 0 0.0.0.0:3000 0.0.0.0:* > LISTEN 14602/ntopng > tcp 0 0 127.0.0.1:35085 127.0.0.1:5556 > ESTABLISHED 14602/ntopng > tcp 0 0 127.0.0.1:40789 127.0.0.1:6379 > ESTABLISHED 14602/ntopng > tcp 0 0 127.0.0.1:5556 127.0.0.1:35085 > ESTABLISHED 14596/nprobe > > Upon cancelling the nprobe I see this: > > 03/Jan/2014 14:41:31 [nprobe.c:369] Received shutdown request... > 03/Jan/2014 14:41:31 [cache.c:850] Redis Cache [0 total/0.0 get/sec][0 > total/0.0 set/sec] > 03/Jan/2014 14:41:31 [nprobe.c:2045] Processed packets: 0 (max bucket search: > 0) > 03/Jan/2014 14:41:31 [nprobe.c:2028] Fragment queue length: 0 > 03/Jan/2014 14:41:31 [nprobe.c:2054] Flow export stats: [0 bytes/0 pkts][0 > flows/0 pkts sent] > 03/Jan/2014 14:41:31 [nprobe.c:2061] Flow collection: [collected pkts: > 3963][processed flows: 67041] > 03/Jan/2014 14:41:31 [nprobe.c:2064] Flow drop stats: [0 bytes/0 pkts][0 > flows] > 03/Jan/2014 14:41:31 [nprobe.c:2069] Total flow stats: [0 bytes/0 pkts][0 > flows/0 pkts sent] > > Upon cancelling ntopng I see this: > > ^C03/Jan/2014 14:41:37 [main.cpp:37] Shutting down... > 03/Jan/2014 14:41:39 [ProtoStats.cpp:35] [IPv4] 0 B/0.00 Packets > 03/Jan/2014 14:41:39 [ProtoStats.cpp:35] [IPv6] 0 B/0.00 Packets > 03/Jan/2014 14:41:39 [ProtoStats.cpp:35] [ARP] 0 B/0.00 Packets > 03/Jan/2014 14:41:39 [ProtoStats.cpp:35] [MPLS] 0 B/0.00 Packets > 03/Jan/2014 14:41:39 [ProtoStats.cpp:35] [Other] 0 B/0.00 Packets > 03/Jan/2014 14:41:39 [Ntop.cpp:590] Interface [email protected]:5556 > [running: 0] > 03/Jan/2014 14:41:39 [main.cpp:55] Deleted PID /var/tmp/ntopng.pid [rc: 0] > 03/Jan/2014 14:41:40 [HTTPserver.cpp:374] HTTP server terminated > 03/Jan/2014 14:41:40 [AddressResolution.cpp:187] Address resolution stats [0 > resolved][0 failures] > > I am starting to believe that the issue is still with nprobe. Just in case, > I'm attaching a capture file before the templates (NF100-200.pcapng) and > after the ASA sent the templates (NF200-300.pcapng) > > Thank you for any ideas, > > Pablo > > > > > > > > > > > On Sat, Dec 14, 2013 at 1:07 PM, Luca Deri <[email protected]> wrote: > Hi Pablo > > On 13 Dec 2013, at 22:59, Pablo Destéfanis <[email protected]> wrote: > >> Hello, >> >> I have ntopng running fine capturing packets redirected from my ASA via a >> SNAP port. Since I cannot use this setup in a VM (now it is running in real >> hardware), I would like to use either ntopng or nprobe as a Netflow >> collector. >> >> I have confirmed that Netflow flows are coming from the ASA unit on UDP:2055 >> with tcpdump >> >> I was unable to find a way/command to start ntopng as a pure Netflow >> collector (listening to UDP:2055), tried the following options with >> different errors: >> >> ntopng -i "tcp://127.0.0.1:2055" [no info displayed in the web interface] >> ntopng -i "udp://127.0.0.1:2055" [ERROR: could not open pcap file: >> udp://127.0.0.1:2055: No such file or directory] >> >> For what I read at >> http://www.ntop.org/nprobe/why-nprobejsonzmq-instead-of-native-sflownetflow-support-in-ntopng/ >> it seems that ntopng will not consume NetFlow flows, but rather ZQM flows >> from nprobe. > yes > >> >> I have tried that setup with nprobe v.6.15.131213, and while I get it to >> listen (shows "Flow collector listening on port 2055 (IPv4/v6)"), and I see >> port TCP:5556 open, ntopng does not seem to consume/connect to nprobe, in >> spite of stating "Collecting flows..." after initialization. >> >> The web interface for ntopng reads "No packet has been received yet on >> interface [email protected]:5556", which is confusing, as I thought that >> actually ntopng would poll packets from nprobe as stated in the document >> referenced above > > I believe your ASA device is not sending flow templates but just flow data. > > Please check as this is a common problem with these devices > > Luca > >> >> Command line used: >> ntopng -i "nprobe-collector.lua@tcp://127.0.0.1:5556" >> nprobe --zmq "tcp://*:5556" -i none -n none --collector-port 2055 >> --city-list /usr/local/nprobe/GeoLiteCity.dat >> >> Note: just ran nprobe with -b 1 and apparently is not getting any data from >> the Netflow flows (checked after a few minutes). It collecting packets, but >> not processing any: >> >> 13/Dec/2013 16:39:25 [nprobe.c:2178] Average traffic: [0.00 pps][0 b/sec] >> 13/Dec/2013 16:39:25 [nprobe.c:2185] Current traffic: [0.00 pps][0 b/sec] >> 13/Dec/2013 16:39:25 [nprobe.c:2191] Current flow export rate: [0.0 >> flows/sec] >> 13/Dec/2013 16:39:25 [nprobe.c:2194] Flow drops: [export queue too >> long=0][too many flows=0] >> 13/Dec/2013 16:39:25 [nprobe.c:2198] Export Queue: 0/512000 [0.0 %] >> 13/Dec/2013 16:39:25 [nprobe.c:2203] Flow Buckets: >> [active=0][allocated=0][toBeExported=0] >> 13/Dec/2013 16:39:25 [cache.c:850] Redis Cache [0 total/0.0 get/sec][0 >> total/0.0 set/sec] >> 13/Dec/2013 16:39:25 [nprobe.c:2221] Collector Threads: [644 pkts@0] >> 13/Dec/2013 16:39:25 [nprobe.c:2045] Processed packets: 0 (max bucket >> search: 0) >> 13/Dec/2013 16:39:25 [nprobe.c:2028] Fragment queue length: 0 >> 13/Dec/2013 16:39:25 [nprobe.c:2054] Flow export stats: [0 bytes/0 pkts][0 >> flows/0 pkts sent] >> 13/Dec/2013 16:39:25 [nprobe.c:2061] Flow collection: [collected pkts: >> 644][processed flows: 0] <------- See here >> 13/Dec/2013 16:39:25 [nprobe.c:2064] Flow drop stats: [0 bytes/0 pkts][0 >> flows] >> 13/Dec/2013 16:39:25 [nprobe.c:2069] Total flow stats: [0 bytes/0 pkts][0 >> flows/0 pkts sent] >> >> Versions: >> nProbe.x86_64 6.15.131213-3810 >> @ntop >> ntopng.x86_64 1.1.1_7107-7107 >> @ntop >> ntopng-data.x86_64 1.1.1_7107-7107 >> @ntop >> pfring.x86_64 5.6.2-7113 >> @ntop >> ntopng-data.x86_64 _7113-7113 ntop >> >> Based on this: >> >> Is it possible to use ntopng to receive Netflow flows from an ASA unit? Can >> you provide an example command line? >> Is the nprobe/ntopng setup correct? Why would ntopng not collect any data? >> >> Thank you for your help, >> >> Pablo Destefanis >> >> ------ >> >> Startup screens follow: >> >> NPROBE >> >> [root@hqsys2 ntopng]# nprobe --zmq "tcp://*:5556" -i none -n none >> --collector-port 2055 --city-list /usr/local/nprobe/GeoLiteCity.dat >> 13/Dec/2013 16:41:19 [nprobe.c:5673] ERROR: Invalid or missing nProbe >> license (/etc/nprobe.license) >> 13/Dec/2013 16:41:19 [nprobe.c:5683] ERROR: for 6F087510910461D2 >> 13/Dec/2013 16:41:19 [nprobe.c:5688] ERROR: >> *************************************************** >> 13/Dec/2013 16:41:19 [nprobe.c:5689] ERROR: ** >> ** >> 13/Dec/2013 16:41:19 [nprobe.c:5690] ERROR: ** Switching to DEMO MODE due >> to license error ** >> 13/Dec/2013 16:41:19 [nprobe.c:5691] ERROR: ** >> ** >> 13/Dec/2013 16:41:19 [nprobe.c:5692] ERROR: ** Create your nProbe license >> at ** >> 13/Dec/2013 16:41:19 [nprobe.c:5693] ERROR: ** >> http://www.nmon.net/mklicense/ ** >> 13/Dec/2013 16:41:19 [nprobe.c:5694] ERROR: ** >> ** >> 13/Dec/2013 16:41:19 [nprobe.c:5695] ERROR: >> *************************************************** >> 13/Dec/2013 16:41:19 [nprobe.c:5715] ERROR: >> *************************************************************** >> 13/Dec/2013 16:41:19 [nprobe.c:5716] ERROR: * NOTE: This is a DEMO version >> limited to 25000 flows export. * >> 13/Dec/2013 16:41:19 [nprobe.c:5717] ERROR: >> *************************************************************** >> 13/Dec/2013 16:41:19 [plugin.c:161] No plugins found in ./plugins >> 13/Dec/2013 16:41:19 [nprobe.c:3628] Succesfully created ZMQ endpoint >> tcp://*:5556 >> 13/Dec/2013 16:41:19 [util.c:344] GeoIP: loaded cities config file >> /usr/local/nprobe/GeoLiteCity.dat >> 13/Dec/2013 16:41:19 [util.c:353] GeoIP: loaded IPv6 cities config file >> /usr/local/nprobe/GeoLiteCityv6.dat >> 13/Dec/2013 16:41:19 [nprobe.c:3795] WARNING: The output interfaceId is set >> to 0: did you forget to use -Q perhaps ? >> 13/Dec/2013 16:41:19 [nprobe.c:3798] WARNING: The input interfaceId is set >> to 0: did you forget to use -u perhaps ? >> 13/Dec/2013 16:41:19 [nprobe.c:3802] WARNING: You have specified --zmq and >> not specified -n. >> 13/Dec/2013 16:41:19 [nprobe.c:3803] WARNING: We believe you want to use >> just ZMQ and no netflow export >> 13/Dec/2013 16:41:19 [nprobe.c:3804] WARNING: Setting flow export to -n none >> 13/Dec/2013 16:41:19 [nprobe.c:3858] Welcome to nprobe v.6.15.131213 >> ($Revision: 3810 $) for x86_64-unknown-linux-gnu with native PF_RING >> acceleration >> 13/Dec/2013 16:41:19 [nprobe.c:3871] nProbe SystemId: 6F087510910461D2 >> 13/Dec/2013 16:41:19 [dbPlugin.c:78] Initializing DB plugin >> 13/Dec/2013 16:41:19 [nprobe.c:5733] Welcome to nprobe v.6.15.131213 for >> x86_64-unknown-linux-gnu >> 13/Dec/2013 16:41:19 [nprobe.c:4943] GEO-533LITE 20090701 Build 1 Copyright >> (c) 2007 MaxMind LLC All Rights Reserved >> 13/Dec/2013 16:41:19 [plugin.c:872] 0 plugin(s) enabled >> 13/Dec/2013 16:41:19 [util.c:308] GeoIP: loaded AS config file >> /usr/local/nprobe/GeoIPASNum.dat >> 13/Dec/2013 16:41:19 [util.c:317] GeoIP: loaded AS IPv6 config file >> /usr/local/nprobe/GeoIPASNumv6.dat >> 13/Dec/2013 16:41:19 [nprobe.c:4379] Using packet capture length 128 >> 13/Dec/2013 16:41:19 [nprobe.c:5909] IPv6 traffic will NOT be >> exported/accounted by this probe >> 13/Dec/2013 16:41:19 [nprobe.c:5910] due to configuration options (e.g. use >> NetFlow v9) >> 13/Dec/2013 16:41:19 [nprobe.c:6038] Not capturing packet from interface >> (collector mode) >> 13/Dec/2013 16:41:19 [collect.c:156] Flow collector listening on port 2055 >> (IPv4/v6) >> >> NTOPNG: >> [root@hqsys2 ~]# ntopng -i "tcp://127.0.0.1:5556" >> 13/Dec/2013 16:59:46 [Ntop.cpp:457] Setting local networks to >> 192.168.1.0/24,0.0.0.0/32,224.0.0.0/8,239.0.0.0/8,255.255.255.255/32,127.0.0.0/8 >> 13/Dec/2013 16:59:46 [Ntop.cpp:564] Registered interface >> [email protected]:5556 [id: 0] >> 13/Dec/2013 16:59:46 [Utils.cpp:238] User changed to nobody >> 13/Dec/2013 16:59:46 [main.cpp:147] PID stored in file /var/tmp/ntopng.pid >> 13/Dec/2013 16:59:46 [HTTPserver.cpp:363] HTTP server listening on port 3000 >> [/usr/local/share/ntopng/httpdocs][/usr/local/share/ntopng/scripts] >> 13/Dec/2013 16:59:46 [main.cpp:179] Using RRD version 1.4.7 >> 13/Dec/2013 16:59:46 [main.cpp:188] Working directory: /var/tmp/ntopng >> 13/Dec/2013 16:59:46 [main.cpp:190] Scripts/HTML pages directory: >> /usr/local/share/ntopng >> 13/Dec/2013 16:59:46 [Ntop.cpp:161] Welcome to ntopng x86_64 v.1.1.1 (r) - >> (C) 1998-13 ntop.org >> 13/Dec/2013 16:59:46 [Redis.cpp:47] Successfully connected to Redis >> 127.0.0.1:6379 >> 13/Dec/2013 16:59:46 [PeriodicActivities.cpp:53] Started periodic activities >> loop... >> 13/Dec/2013 16:59:46 [NetworkInterface.cpp:634] Started packet polling on >> interface [email protected]:5556... >> 13/Dec/2013 16:59:46 [CollectorInterface.cpp:100] Collecting flows... >> >> >> >> _______________________________________________ >> Ntop-misc mailing list >> [email protected] >> http://listgateway.unipi.it/mailman/listinfo/ntop-misc > > > _______________________________________________ > Ntop-misc mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop-misc > > > > _______________________________________________ > Ntop-misc mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop-misc
_______________________________________________ Ntop-misc mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop-misc
