Re: [Ntop] Netflow (NSEL) updates from Cisco ASA

Sat, 15 Jul 2017 10:14:18 -0700 

Hello Luca,
I've opened an issue on github as you requested: https://github.com/ntop/ntopng/issues/1359 . Please contact me off-list if you need full .pcap or other data. 

--

With Best Regards,
Marat Khalili

On 13/07/17 10:27, Marat Khalili wrote:



ASA (and other devices like PaloAlto) generate flows based on 
firewall events, like flow create, deletion etc. and do not send 
interim updates as flow devices do. ASA for instance sends only total 
octets. 

According to documentation 
<https://www.cisco.com/c/en/us/td/docs/security/asa/special/netflow/guide/asa_netflow.html#pgfId-1324322> 
there are Flow-Update events sent every minute by default.

Shall you have further comments on this matter, please file ticket on 
github so we can keep a log of the discussion

Ok, I'll re-test it my environment and open the ticket with what I'm 
seeing.

--

With Best Regards,
Marat Khalili

On 13/07/17 01:26, Luca Deri wrote:

Hi all,

ASA (and other devices like PaloAlto) generate flows based on 
firewall events, like flow create, deletion etc. and do not send 
interim updates as flow devices do. ASA for instance sends only total 
octets.



Due to this if we take into account flow updates we need to make sure 
that the frequency of these updates is greater than the flow 
inactivity timeout in ntopng as otherwise we would generate incorrect 
values (i.e. we will account more traffic than the one that has 
really passed on the network). The flow delete instead is not 
affected by this problem as it is the very last message emitted for a 
given flow.



This said, reading this thread it seems that you all have different 
option and this I have modified nProbe to implement the behaviour you 
expect. Shall you have further comments on this matter, please file 
ticket on github so we can keep a log of the discussion


Regards Luca


On 6 Jul 2017, at 14:13, Marat Khalili <[email protected] 
<mailto:[email protected]>> wrote:


Hello Simone,


I'm also very interested in this issue, observing same behaviour as 
Pelham. Do you mean we need a custom fields configuration file to 
make this work? How will it be different from your attempts where 
"events other than flow-delete often contain to few attributes to 
properly update the flow"?


--

With Best Regards,
Marat Khalili

On 06/07/17 14:54, Simone Mainardi wrote:

Dear Pelham,


Thanks for sharing the pcap. Currently, in nProbe we deliberately 
ignore ASA firewall events different from flow-delete. We have made 
this choice as we have seen that events other than flow-delete 
often contain to few attributes to properly update the flow. This 
is the reason why other templates are silently ignored by nprobe.



If you want your nProbe to proxy all the templates received as-is, 
you may want to have a look at: 
http://www.ntop.org/nprobe/collecting-proprietary-flows-with-nprobe/



Simone

Regards,
Simone




On Thu, Jul 6, 2017 at 3:55 AM, Pelham Whitmore 
<[email protected] 
<mailto:[email protected]>> wrote:


    Hey Simone,

    Thanks for the reply. I have configured the timeout values you
    mentioned in nprobe however it seemed to have no effect.
    I have generated a .pcap file that includes templates, flow
    create, update, and teardown events.

    .pcap file can be downloaded from here:
    https://files.aceinfo.net.au/index.php/s/bj0aFU3lpyUjqM4/download
    <https://files.aceinfo.net.au/index.php/s/bj0aFU3lpyUjqM4/download>

    One thing I did notcie from the packet capture is that flow
    update events are being listed as "Firewall Event: Unknown (5)" .
    I'm not sure if that is to be expected.


    Regards,
    Pelham Whitmore


    -----Original Message-----
    From: [email protected]
    <mailto:[email protected]>
    [mailto:[email protected]
    <mailto:[email protected]>] On Behalf Of
    [email protected]
    <mailto:[email protected]>
    Sent: 05 July 2017 8:00 PM
    To: [email protected] <mailto:[email protected]>
    Subject: Ntop Digest, Vol 158, Issue 1

    Send Ntop mailing list submissions to
    [email protected] <mailto:[email protected]>

    To subscribe or unsubscribe via the World Wide Web, visit
    http://listgateway.unipi.it/mailman/listinfo/ntop
    <http://listgateway.unipi.it/mailman/listinfo/ntop>
    or, via email, send a message with subject or body 'help' to
    [email protected]
    <mailto:[email protected]>

    You can reach the person managing the list at
    [email protected]
    <mailto:[email protected]>

    When replying, please edit your Subject line so it is more
    specific than "Re: Contents of Ntop digest..."


    Today's Topics:

       1. Re: Netflow (NSEL) updates from Cisco ASA (Simone Mainardi)
       2. Re: Netflow (NSEL) updates from Cisco ASA (Simone Mainardi)


    ----------------------------------------------------------------------

    Message: 1
    Date: Tue, 4 Jul 2017 14:25:19 +0200
    From: Simone Mainardi <[email protected]
    <mailto:[email protected]>>
    To: [email protected] <mailto:[email protected]>
    Cc: "[email protected]
    <mailto:[email protected]>" <[email protected]
    <mailto:[email protected]>>
    Subject: Re: [Ntop] Netflow (NSEL) updates from Cisco ASA
    Message-ID:

           
    <cajcxkcdhiqcnpxrwnx3sos6mcszrkmnanacbuxcjz96iynz...@mail.gmail.com

    <mailto:cajcxkcdhiqcnpxrwnx3sos6mcszrkmnanacbuxcjz96iynz...@mail.gmail.com>>
    Content-Type: text/plain; charset="utf-8"

    Pelham,

    ASA flow-updates are received and processed by nprobe. However,
    I am not sure they contain all the necessary information
    required to properly update flow statistics. Can you please
    generate and send a .pcap capture file of your ASA netflow
    (make sure it contains both templates and data records for
    flow-updates and flow-teardown) for our inspection?

    By the way, nprobe gives you a couple of configurable timeout
    that you can use to periodically export long-lived flows:

    [--lifetime-timeout|-t] <timeout>  | It specifies the maximum
    (seconds)
    flow
                                        | lifetime [default=120]
    [--idle-timeout|-d] <timeout>  | It specifies the maximum (seconds)
    flow
                                        | idle lifetime [default=30]

    Regards,
    Simone

    On Wed, Jun 28, 2017 at 2:38 AM, Pelham Whitmore <
    [email protected]
    <mailto:[email protected]>> wrote:

    > Hello,
    >
    >
    >
    > I have a Cisco ASA configured to send Netflow to an instance of
    > nprobe, and then on to ntopng.
    >
    > The configuration is working, however I have noticed that
    nprobe is
    > only emitting flows when it receives a flow-teardown event
    from the
    > ASA. This is causing inaccurate bandwidth reporting for
    long-lived
    > flows as the total byte count is being recorded as a single
    spike once the flow is torn down.
    >
    >
    >
    > My understanding is that Cisco ASA netflow is very
    non-standard and
    > that this behaviour used to be expected on older version of ASA.
    > However, newer versions of ASA are capable of sending flow-update
    > events using a refresh-interval for active flows. When I run
    tcpdump
    > on my nprobe server I can see the flow-create and flow-update
    events
    > being sent from the ASA, however nprobe does not seem to use
    these
    > events, or act on them in any way. I have enabled verbose
    logging, but
    > can only see logs being generated for flow-teardown events,
    not flow-create or flow-update.
    >
    >
    >
    > My question is, should I expect nprobe to use the
    flow-updates from
    > Cisco ASA for long-lived active flows, or is it normal for it
    to only
    > process flow-teardown events?
    >
    >
    >
    > Nprobe (dev build v.8.1.170626)  is running in collector mode
    with the
    > following settings:
    >
    >
    >
    > --zmq="tcp://*:5559"
    >
    > --collector-port=2055
    >
    > -i=none
    >
    > -n=none
    >
    >
    >
    >
    >
    >
    >
    > Regards,
    >
    > Pelham
    >
    >
    >
    >
    >
    > ------------------------------
    >
    > This electronic mail is solely for the use of the addressee
    and may
    > contain information which is confidential or privileged. If you
    > receive this electronic mail in error, please delete it from your
    > system immediately and notify the sender by electronic mail. Any
    > opinion expressed in this email is not represented as the
    opinion of
    > Australian Communication Limited unless that is stated or
    apparent from its terms.
    > ------------------------------
    >
    >
    > _______________________________________________
    > Ntop mailing list
    > [email protected] <mailto:[email protected]>
    > http://listgateway.unipi.it/mailman/listinfo/ntop
    <http://listgateway.unipi.it/mailman/listinfo/ntop>
    >
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL:
    
<http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0002.htm
    
<http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0002.htm>>

    ------------------------------

    Message: 2
    Date: Tue, 4 Jul 2017 14:25:19 +0200
    From: Simone Mainardi <[email protected]
    <mailto:[email protected]>>
    To: [email protected] <mailto:[email protected]>
    Cc: "[email protected]
    <mailto:[email protected]>" <[email protected]
    <mailto:[email protected]>>
    Subject: Re: [Ntop] Netflow (NSEL) updates from Cisco ASA
    Message-ID:

           
    <cajcxkcdhiqcnpxrwnx3sos6mcszrkmnanacbuxcjz96iynz...@mail.gmail.com

    <mailto:cajcxkcdhiqcnpxrwnx3sos6mcszrkmnanacbuxcjz96iynz...@mail.gmail.com>>
    Content-Type: text/plain; charset="utf-8"

    Pelham,

    ASA flow-updates are received and processed by nprobe. However,
    I am not sure they contain all the necessary information
    required to properly update flow statistics. Can you please
    generate and send a .pcap capture file of your ASA netflow
    (make sure it contains both templates and data records for
    flow-updates and flow-teardown) for our inspection?

    By the way, nprobe gives you a couple of configurable timeout
    that you can use to periodically export long-lived flows:

    [--lifetime-timeout|-t] <timeout>  | It specifies the maximum
    (seconds)
    flow
                                        | lifetime [default=120]
    [--idle-timeout|-d] <timeout>  | It specifies the maximum (seconds)
    flow
                                        | idle lifetime [default=30]

    Regards,
    Simone

    On Wed, Jun 28, 2017 at 2:38 AM, Pelham Whitmore <
    [email protected]
    <mailto:[email protected]>> wrote:

    > Hello,
    >
    >
    >
    > I have a Cisco ASA configured to send Netflow to an instance of
    > nprobe, and then on to ntopng.
    >
    > The configuration is working, however I have noticed that
    nprobe is
    > only emitting flows when it receives a flow-teardown event
    from the
    > ASA. This is causing inaccurate bandwidth reporting for
    long-lived
    > flows as the total byte count is being recorded as a single
    spike once the flow is torn down.
    >
    >
    >
    > My understanding is that Cisco ASA netflow is very
    non-standard and
    > that this behaviour used to be expected on older version of ASA.
    > However, newer versions of ASA are capable of sending flow-update
    > events using a refresh-interval for active flows. When I run
    tcpdump
    > on my nprobe server I can see the flow-create and flow-update
    events
    > being sent from the ASA, however nprobe does not seem to use
    these
    > events, or act on them in any way. I have enabled verbose
    logging, but
    > can only see logs being generated for flow-teardown events,
    not flow-create or flow-update.
    >
    >
    >
    > My question is, should I expect nprobe to use the
    flow-updates from
    > Cisco ASA for long-lived active flows, or is it normal for it
    to only
    > process flow-teardown events?
    >
    >
    >
    > Nprobe (dev build v.8.1.170626)  is running in collector mode
    with the
    > following settings:
    >
    >
    >
    > --zmq="tcp://*:5559"
    >
    > --collector-port=2055
    >
    > -i=none
    >
    > -n=none
    >
    >
    >
    >
    >
    >
    >
    > Regards,
    >
    > Pelham
    >
    >
    >
    >
    >
    > ------------------------------
    >
    > This electronic mail is solely for the use of the addressee
    and may
    > contain information which is confidential or privileged. If you
    > receive this electronic mail in error, please delete it from your
    > system immediately and notify the sender by electronic mail. Any
    > opinion expressed in this email is not represented as the
    opinion of
    > Australian Communication Limited unless that is stated or
    apparent from its terms.
    > ------------------------------
    >
    >
    > _______________________________________________
    > Ntop mailing list
    > [email protected] <mailto:[email protected]>
    > http://listgateway.unipi.it/mailman/listinfo/ntop
    <http://listgateway.unipi.it/mailman/listinfo/ntop>
    >
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL:
    
<http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0003.htm
    
<http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0003.htm>>

    ------------------------------

    _______________________________________________
    Ntop mailing list
    [email protected] <mailto:[email protected]>
    http://listgateway.unipi.it/mailman/listinfo/ntop
    <http://listgateway.unipi.it/mailman/listinfo/ntop>

    End of Ntop Digest, Vol 158, Issue 1
    ************************************
    This electronic mail is solely for the use of the addressee and
    may contain information which is confidential or privileged.
    If you receive this electronic mail in error, please delete it
    from your system immediately and notify the sender by
    electronic mail.
    Any opinion expressed in this email is not represented as the
    opinion of Australian Communication Exchange Limited unless
    that is stated or
    apparent from its terms.
    _______________________________________________
    Ntop mailing list
    [email protected] <mailto:[email protected]>
    http://listgateway.unipi.it/mailman/listinfo/ntop
    <http://listgateway.unipi.it/mailman/listinfo/ntop>




_______________________________________________
Ntop mailing list
[email protected]
http://listgateway.unipi.it/mailman/listinfo/ntop


_______________________________________________
Ntop mailing list
[email protected] <mailto:[email protected]>
http://listgateway.unipi.it/mailman/listinfo/ntop




_______________________________________________
Ntop mailing list
[email protected]
http://listgateway.unipi.it/mailman/listinfo/ntop




_______________________________________________
Ntop mailing list
[email protected]
http://listgateway.unipi.it/mailman/listinfo/ntop



_______________________________________________
Ntop mailing list
[email protected]
http://listgateway.unipi.it/mailman/listinfo/ntop





















					Reply via email to