Hi all, ASA (and other devices like PaloAlto) generate flows based on firewall events, like flow create, deletion etc. and do not send interim updates as flow devices do. ASA for instance sends only total octets.
Due to this if we take into account flow updates we need to make sure that the frequency of these updates is greater than the flow inactivity timeout in ntopng as otherwise we would generate incorrect values (i.e. we will account more traffic than the one that has really passed on the network). The flow delete instead is not affected by this problem as it is the very last message emitted for a given flow. This said, reading this thread it seems that you all have different option and this I have modified nProbe to implement the behaviour you expect. Shall you have further comments on this matter, please file ticket on github so we can keep a log of the discussion Regards Luca > On 6 Jul 2017, at 14:13, Marat Khalili <[email protected]> wrote: > > Hello Simone, > > I'm also very interested in this issue, observing same behaviour as Pelham. > Do you mean we need a custom fields configuration file to make this work? How > will it be different from your attempts where "events other than flow-delete > often contain to few attributes to properly update the flow"? > -- > > With Best Regards, > Marat Khalili > > On 06/07/17 14:54, Simone Mainardi wrote: >> Dear Pelham, >> >> Thanks for sharing the pcap. Currently, in nProbe we deliberately ignore ASA >> firewall events different from flow-delete. We have made this choice as we >> have seen that events other than flow-delete often contain to few attributes >> to properly update the flow. This is the reason why other templates are >> silently ignored by nprobe. >> >> If you want your nProbe to proxy all the templates received as-is, you may >> want to have a look at: >> http://www.ntop.org/nprobe/collecting-proprietary-flows-with-nprobe/ >> <http://www.ntop.org/nprobe/collecting-proprietary-flows-with-nprobe/> >> >> >> Simone >> >> Regards, >> Simone >> >> >> >> On Thu, Jul 6, 2017 at 3:55 AM, Pelham Whitmore >> <[email protected] <mailto:[email protected]>> >> wrote: >> Hey Simone, >> >> Thanks for the reply. I have configured the timeout values you mentioned in >> nprobe however it seemed to have no effect. >> I have generated a .pcap file that includes templates, flow create, update, >> and teardown events. >> >> .pcap file can be downloaded from here: >> https://files.aceinfo.net.au/index.php/s/bj0aFU3lpyUjqM4/download >> <https://files.aceinfo.net.au/index.php/s/bj0aFU3lpyUjqM4/download> >> >> One thing I did notcie from the packet capture is that flow update events >> are being listed as "Firewall Event: Unknown (5)" . >> I'm not sure if that is to be expected. >> >> >> Regards, >> Pelham Whitmore >> >> >> -----Original Message----- >> From: [email protected] >> <mailto:[email protected]> >> [mailto:[email protected] >> <mailto:[email protected]>] On Behalf Of >> [email protected] <mailto:[email protected]> >> Sent: 05 July 2017 8:00 PM >> To: [email protected] <mailto:[email protected]> >> Subject: Ntop Digest, Vol 158, Issue 1 >> >> Send Ntop mailing list submissions to >> [email protected] <mailto:[email protected]> >> >> To subscribe or unsubscribe via the World Wide Web, visit >> http://listgateway.unipi.it/mailman/listinfo/ntop >> <http://listgateway.unipi.it/mailman/listinfo/ntop> >> or, via email, send a message with subject or body 'help' to >> [email protected] >> <mailto:[email protected]> >> >> You can reach the person managing the list at >> [email protected] >> <mailto:[email protected]> >> >> When replying, please edit your Subject line so it is more specific than >> "Re: Contents of Ntop digest..." >> >> >> Today's Topics: >> >> 1. Re: Netflow (NSEL) updates from Cisco ASA (Simone Mainardi) >> 2. Re: Netflow (NSEL) updates from Cisco ASA (Simone Mainardi) >> >> >> ---------------------------------------------------------------------- >> >> Message: 1 >> Date: Tue, 4 Jul 2017 14:25:19 +0200 >> From: Simone Mainardi <[email protected] <mailto:[email protected]>> >> To: [email protected] <mailto:[email protected]> >> Cc: "[email protected] <mailto:[email protected]>" >> <[email protected] <mailto:[email protected]>> >> Subject: Re: [Ntop] Netflow (NSEL) updates from Cisco ASA >> Message-ID: >> <cajcxkcdhiqcnpxrwnx3sos6mcszrkmnanacbuxcjz96iynz...@mail.gmail.com >> <mailto:cajcxkcdhiqcnpxrwnx3sos6mcszrkmnanacbuxcjz96iynz...@mail.gmail.com>> >> Content-Type: text/plain; charset="utf-8" >> >> Pelham, >> >> ASA flow-updates are received and processed by nprobe. However, I am not >> sure they contain all the necessary information required to properly update >> flow statistics. Can you please generate and send a .pcap capture file of >> your ASA netflow (make sure it contains both templates and data records for >> flow-updates and flow-teardown) for our inspection? >> >> By the way, nprobe gives you a couple of configurable timeout that you can >> use to periodically export long-lived flows: >> >> [--lifetime-timeout|-t] <timeout> | It specifies the maximum (seconds) >> flow >> | lifetime [default=120] >> [--idle-timeout|-d] <timeout> | It specifies the maximum (seconds) >> flow >> | idle lifetime [default=30] >> >> Regards, >> Simone >> >> On Wed, Jun 28, 2017 at 2:38 AM, Pelham Whitmore < >> [email protected] <mailto:[email protected]>> >> wrote: >> >> > Hello, >> > >> > >> > >> > I have a Cisco ASA configured to send Netflow to an instance of >> > nprobe, and then on to ntopng. >> > >> > The configuration is working, however I have noticed that nprobe is >> > only emitting flows when it receives a flow-teardown event from the >> > ASA. This is causing inaccurate bandwidth reporting for long-lived >> > flows as the total byte count is being recorded as a single spike once the >> > flow is torn down. >> > >> > >> > >> > My understanding is that Cisco ASA netflow is very non-standard and >> > that this behaviour used to be expected on older version of ASA. >> > However, newer versions of ASA are capable of sending flow-update >> > events using a refresh-interval for active flows. When I run tcpdump >> > on my nprobe server I can see the flow-create and flow-update events >> > being sent from the ASA, however nprobe does not seem to use these >> > events, or act on them in any way. I have enabled verbose logging, but >> > can only see logs being generated for flow-teardown events, not >> > flow-create or flow-update. >> > >> > >> > >> > My question is, should I expect nprobe to use the flow-updates from >> > Cisco ASA for long-lived active flows, or is it normal for it to only >> > process flow-teardown events? >> > >> > >> > >> > Nprobe (dev build v.8.1.170626) is running in collector mode with the >> > following settings: >> > >> > >> > >> > --zmq="tcp://*:5559" >> > >> > --collector-port=2055 >> > >> > -i=none >> > >> > -n=none >> > >> > >> > >> > >> > >> > >> > >> > Regards, >> > >> > Pelham >> > >> > >> > >> > >> > >> > ------------------------------ >> > >> > This electronic mail is solely for the use of the addressee and may >> > contain information which is confidential or privileged. If you >> > receive this electronic mail in error, please delete it from your >> > system immediately and notify the sender by electronic mail. Any >> > opinion expressed in this email is not represented as the opinion of >> > Australian Communication Limited unless that is stated or apparent from >> > its terms. >> > ------------------------------ >> > >> > >> > _______________________________________________ >> > Ntop mailing list >> > [email protected] <mailto:[email protected]> >> > http://listgateway.unipi.it/mailman/listinfo/ntop >> > <http://listgateway.unipi.it/mailman/listinfo/ntop> >> > >> -------------- next part -------------- >> An HTML attachment was scrubbed... >> URL: >> <http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0002.htm >> >> <http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0002.htm>> >> >> ------------------------------ >> >> Message: 2 >> Date: Tue, 4 Jul 2017 14:25:19 +0200 >> From: Simone Mainardi <[email protected] <mailto:[email protected]>> >> To: [email protected] <mailto:[email protected]> >> Cc: "[email protected] <mailto:[email protected]>" >> <[email protected] <mailto:[email protected]>> >> Subject: Re: [Ntop] Netflow (NSEL) updates from Cisco ASA >> Message-ID: >> <cajcxkcdhiqcnpxrwnx3sos6mcszrkmnanacbuxcjz96iynz...@mail.gmail.com >> <mailto:cajcxkcdhiqcnpxrwnx3sos6mcszrkmnanacbuxcjz96iynz...@mail.gmail.com>> >> Content-Type: text/plain; charset="utf-8" >> >> Pelham, >> >> ASA flow-updates are received and processed by nprobe. However, I am not >> sure they contain all the necessary information required to properly update >> flow statistics. Can you please generate and send a .pcap capture file of >> your ASA netflow (make sure it contains both templates and data records for >> flow-updates and flow-teardown) for our inspection? >> >> By the way, nprobe gives you a couple of configurable timeout that you can >> use to periodically export long-lived flows: >> >> [--lifetime-timeout|-t] <timeout> | It specifies the maximum (seconds) >> flow >> | lifetime [default=120] >> [--idle-timeout|-d] <timeout> | It specifies the maximum (seconds) >> flow >> | idle lifetime [default=30] >> >> Regards, >> Simone >> >> On Wed, Jun 28, 2017 at 2:38 AM, Pelham Whitmore < >> [email protected] <mailto:[email protected]>> >> wrote: >> >> > Hello, >> > >> > >> > >> > I have a Cisco ASA configured to send Netflow to an instance of >> > nprobe, and then on to ntopng. >> > >> > The configuration is working, however I have noticed that nprobe is >> > only emitting flows when it receives a flow-teardown event from the >> > ASA. This is causing inaccurate bandwidth reporting for long-lived >> > flows as the total byte count is being recorded as a single spike once the >> > flow is torn down. >> > >> > >> > >> > My understanding is that Cisco ASA netflow is very non-standard and >> > that this behaviour used to be expected on older version of ASA. >> > However, newer versions of ASA are capable of sending flow-update >> > events using a refresh-interval for active flows. When I run tcpdump >> > on my nprobe server I can see the flow-create and flow-update events >> > being sent from the ASA, however nprobe does not seem to use these >> > events, or act on them in any way. I have enabled verbose logging, but >> > can only see logs being generated for flow-teardown events, not >> > flow-create or flow-update. >> > >> > >> > >> > My question is, should I expect nprobe to use the flow-updates from >> > Cisco ASA for long-lived active flows, or is it normal for it to only >> > process flow-teardown events? >> > >> > >> > >> > Nprobe (dev build v.8.1.170626) is running in collector mode with the >> > following settings: >> > >> > >> > >> > --zmq="tcp://*:5559" >> > >> > --collector-port=2055 >> > >> > -i=none >> > >> > -n=none >> > >> > >> > >> > >> > >> > >> > >> > Regards, >> > >> > Pelham >> > >> > >> > >> > >> > >> > ------------------------------ >> > >> > This electronic mail is solely for the use of the addressee and may >> > contain information which is confidential or privileged. If you >> > receive this electronic mail in error, please delete it from your >> > system immediately and notify the sender by electronic mail. Any >> > opinion expressed in this email is not represented as the opinion of >> > Australian Communication Limited unless that is stated or apparent from >> > its terms. >> > ------------------------------ >> > >> > >> > _______________________________________________ >> > Ntop mailing list >> > [email protected] <mailto:[email protected]> >> > http://listgateway.unipi.it/mailman/listinfo/ntop >> > <http://listgateway.unipi.it/mailman/listinfo/ntop> >> > >> -------------- next part -------------- >> An HTML attachment was scrubbed... >> URL: >> <http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0003.htm >> >> <http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0003.htm>> >> >> ------------------------------ >> >> _______________________________________________ >> Ntop mailing list >> [email protected] <mailto:[email protected]> >> http://listgateway.unipi.it/mailman/listinfo/ntop >> <http://listgateway.unipi.it/mailman/listinfo/ntop> >> >> End of Ntop Digest, Vol 158, Issue 1 >> ************************************ >> This electronic mail is solely for the use of the addressee and may contain >> information which is confidential or privileged. >> If you receive this electronic mail in error, please delete it from your >> system immediately and notify the sender by electronic mail. >> Any opinion expressed in this email is not represented as the opinion of >> Australian Communication Exchange Limited unless that is stated or >> apparent from its terms. >> _______________________________________________ >> Ntop mailing list >> [email protected] <mailto:[email protected]> >> http://listgateway.unipi.it/mailman/listinfo/ntop >> <http://listgateway.unipi.it/mailman/listinfo/ntop> >> >> >> >> _______________________________________________ >> Ntop mailing list >> [email protected] <mailto:[email protected]> >> http://listgateway.unipi.it/mailman/listinfo/ntop >> <http://listgateway.unipi.it/mailman/listinfo/ntop> > _______________________________________________ > Ntop mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop
_______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop
