Just for the archives -
The default template timeout in XR is 1800 seconds. I guess this is too long
for nfSen before it gives up and assumes that the exporter is not sampling?
Dropping this timeout to 30 seconds started to produce template data:
!
flow exporter-map fem
version v9
template timeout 30
!
This seems to reflect in nfdump -E
[ario@monitor01 live]$ nfdump -E bfr01-hudson/2013/01/11/nfcapd.201301111455
Exporters:
SysID: 1, IP: 10.219.49.11, version: 9, ID: 2049, Sequence failures: 5,
packets: 49123, flows: 1194104
Sampler for Exporter SysID: 1, Sampler: id: 1, mode: 2, interval: 10
[ario@monitor01 live]$ nfdump -E bfr01-mowat/2013/01/11/nfcapd.201301111455
Exporters:
SysID: 1, IP: 10.219.49.1, version: 9, ID: 2081, Sequence failures: 1,
packets: 24356, flows: 577625
Sampler for Exporter SysID: 1, Sampler: id: 1, mode: 2, interval: 10
[ario@monitor01 live]$ nfdump -E bfr01-front/2013/01/11/nfcapd.201301111455
Exporters:
SysID: 1, IP: 10.219.49.2, version: 9, ID: 2065, Sequence failures: 21,
packets: 17015, flows: 407730
Sampler for Exporter SysID: 1, Sampler: id: 1, mode: 2, interval: 10
SysID: 2, IP: 10.219.49.2, version: 9, ID: 2081, Sequence failures: 20,
packets: 20304, flows: 487401
Sampler for Exporter SysID: 2, Sampler: id: 1, mode: 2, interval: 10
[ario@monitor01 live]$
A couple of questions gleaned from this output -
1. What are the sequence failures indicative of?
2. Why does one of my routers have two SysIDs? It is configured identically
to the other two.
On 2013-01-05, at 3:08 AM, Peter Haag <[email protected]> wrote:
> Hi Jason,
> Thanks for the output and the packet dump.
> Sampling is sometimes a bit tricky. There are various possibilities
> to announce sampling and different vendors use different models.
> From your packet dump, you have a couple of data records and one
> template record. This template record is required in order to decode
> the data records. In the template record you see the announcement of
> FLOW_SAMPLER_ID (48). This is the reference to the sampler, as you
> may have several samplers in the same box. This reference points
> to sampler data, which is announced in option templates and option
> data records. For a cisco box, this may look like in debug mode of
> nfcapd:
>
> [0] Option Template ID: 257
> Scope length: 4 Option length: 12
> Scope field Type: 1, length 0
> Option field Type: 48, length 1
> Option field Type: 49, length 1
> Option field Type: 50, length 4
> [0] Sampling information found
> Allocate new sampling info from template 257
> Process_v9: New sampler: ID 0, mode: 1, interval: 2
>
> Now nfcapd can link the sampler id to the sampler data.
> To make the long story short, for some reason your box does not send
> these option template/data records. The reference to sampler ID 1
> is still missing at this point.
> So check at what interval the templates are refreshed or if they are
> sent at all.
>
> A note for nfcapd -s 1000:
> In this form nfcapd takes this as a start value for sampling. If the
> device announces at some point a sampling rate on its own, this new
> sampling rate is taken. In order to force nfcapd unconditionally to
> take 1000 as a sampling rate, regardless wht's announced, use
> -s -1000 ( negative number )
>
> Hope this helps
>
> - Peter
>
> On 4/1/13 5:43 PM, Jason Lixfeld wrote:
>>
>> On 2013-01-04, at 11:24 AM, "Mark D. Nagel" <[email protected]> wrote:
>>
>>> OK, that looks right. You might want to capture and post some raw flows
>>> with tcpdump so they can be examined to see if the sampler information
>>> is really present in the resulting datagrams. There was a similar
>>> thread on this for JunOS
>>> (http://blog.gmane.org/gmane.network.nfsen.general/month=20110101). It
>>> sounds like IOS-XR may not be sending the sampler info, but I'd love to
>>> see those raw datagrams (just a few) to see if that is really true. As
>>> Peter mentioned, you can add the "-s 1000" option to the source
>>> definition to force nfcapd to impose that rate on the exported data
>>> (also discussed in that thread).
>>
>> Ya, I've added the -s 1000 to nfsen.conf and that is working no problem, but
>> I'm curious as to why the sampling isn't working otherwise. Here are a
>> couple of flows and a template:
>>
>> No. Time VLAN Source Destination
>> Protocol Length Info
>> TCP Win Value TCP Win Scale TCP Win Size MPLS Label
>> 1 0.000000 10.219.49.1 10.219.51.130
>> CFLOW 126 total: 1 (v9) record
>>
>>
>> Frame 1: 126 bytes on wire (1008 bits), 126 bytes captured (1008 bits)
>> WTAP_ENCAP: 1
>> Arrival Time: Jan 4, 2013 11:30:25.199867000 EST
>> [Time shift for this packet: 0.000000000 seconds]
>> Epoch Time: 1357317025.199867000 seconds
>> [Time delta from previous captured frame: 0.000000000 seconds]
>> [Time delta from previous displayed frame: 0.000000000 seconds]
>> [Time since reference or first frame: 0.000000000 seconds]
>> Frame Number: 1
>> Frame Length: 126 bytes (1008 bits)
>> Capture Length: 126 bytes (1008 bits)
>> [Frame is marked: False]
>> [Frame is ignored: False]
>> [Protocols in frame: eth:ip:udp:cflow]
>> [Coloring Rule Name: UDP]
>> [Coloring Rule String: udp]
>> Ethernet II, Src: Cisco_ba:ba:40 (f4:ac:c1:ba:ba:40), Dst: Vmware_a5:70:ae
>> (00:0c:29:a5:70:ae)
>> Destination: Vmware_a5:70:ae (00:0c:29:a5:70:ae)
>> Address: Vmware_a5:70:ae (00:0c:29:a5:70:ae)
>> .... ..0. .... .... .... .... = LG bit: Globally unique address
>> (factory default)
>> .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
>> Source: Cisco_ba:ba:40 (f4:ac:c1:ba:ba:40)
>> Address: Cisco_ba:ba:40 (f4:ac:c1:ba:ba:40)
>> .... ..0. .... .... .... .... = LG bit: Globally unique address
>> (factory default)
>> .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
>> Type: IP (0x0800)
>> Internet Protocol Version 4, Src: 10.219.49.1 (10.219.49.1), Dst:
>> 10.219.51.130 (10.219.51.130)
>> Version: 4
>> Header length: 20 bytes
>> Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00:
>> Not-ECT (Not ECN-Capable Transport))
>> 0000 00.. = Differentiated Services Codepoint: Default (0x00)
>> .... ..00 = Explicit Congestion Notification: Not-ECT (Not
>> ECN-Capable Transport) (0x00)
>> Total Length: 112
>> Identification: 0x8194 (33172)
>> Flags: 0x00
>> 0... .... = Reserved bit: Not set
>> .0.. .... = Don't fragment: Not set
>> ..0. .... = More fragments: Not set
>> Fragment offset: 0
>> Time to live: 254
>> Protocol: UDP (17)
>> Header checksum: 0xc0af [correct]
>> [Good: True]
>> [Bad: False]
>> Source: 10.219.49.1 (10.219.49.1)
>> Destination: 10.219.51.130 (10.219.51.130)
>> [Source GeoIP: Unknown]
>> [Destination GeoIP: Unknown]
>> User Datagram Protocol, Src Port: 22919 (22919), Dst Port: 4901 (4901)
>> Source port: 22919 (22919)
>> Destination port: 4901 (4901)
>> Length: 92
>> Checksum: 0x0000 (none)
>> [Good Checksum: False]
>> [Bad Checksum: False]
>> Cisco NetFlow/IPFIX
>> Version: 9
>> Count: 1
>> SysUptime: 245079420
>> Timestamp: Jan 4, 2013 11:30:25.000000000 EST
>> CurrentSecs: 1357317025
>> FlowSequence: 491924
>> SourceId: 2081
>> FlowSet 1
>> FlowSet Id: (Data) (260)
>> FlowSet Length: 64
>> Flow 1
>> Packets: 2
>> Octets: 104
>> SrcAddr: mail.zulualphakilo.com (75.98.195.34)
>> DstAddr: c-76-115-189-181.hsd1.or.comcast.net (76.115.189.181)
>> InputInt: 67
>> OutputInt: 86
>> [Duration: 7.312000000 seconds]
>> StartTime: 245056.596000000 seconds
>> EndTime: 245063.908000000 seconds
>> SrcPort: 46874
>> DstPort: 26698
>> SrcAS: 0
>> DstAS: 7922
>> BGPNextHop: 209.29.130.241 (209.29.130.241)
>> SrcMask: 30
>> DstMask: 11
>> Protocol: 6
>> TCP Flags: 0x10
>> IP ToS: 0x00
>> Direction: Egress (1)
>> Forwarding Status: Forward: Forwarded (Unknown)
>> 01.. .... = ForwdStat: Forward (1)
>> ..00 0000 = ForwdCode: Forwarded (Unknown) (0)
>> SamplerID: 1
>> Padding (3 bytes)
>>
>> 0000 00 0c 29 a5 70 ae f4 ac c1 ba ba 40 08 00 45 00 ..)[email protected].
>> 0010 00 70 81 94 00 00 fe 11 c0 af 0a db 31 01 0a db .p..........1...
>> 0020 33 82 59 87 13 25 00 5c 00 00 00 09 00 01 0e 9b 3.Y..%.\........
>> 0030 9d 7c 50 e7 03 a1 00 07 81 94 00 00 08 21 01 04 .|P..........!..
>> 0040 00 40 00 00 00 02 00 00 00 68 4b 62 c3 22 4c 73 [email protected]."Ls
>> 0050 bd b5 00 00 00 43 00 00 00 56 0e 9b 60 e4 0e 9b .....C...V..`...
>> 0060 44 54 b7 1a 68 4a 00 00 00 00 00 00 1e f2 d1 1d DT..hJ..........
>> 0070 82 f1 1e 0b 06 10 00 01 40 00 01 00 00 00 ........@.....
>>
>> No. Time VLAN Source Destination
>> Protocol Length Info
>> TCP Win Value TCP Win Scale TCP Win Size MPLS Label
>> 3 2.004253 10.219.49.1 10.219.51.130
>> CFLOW 126 total: 1 (v9) record
>>
>>
>> Frame 3: 126 bytes on wire (1008 bits), 126 bytes captured (1008 bits)
>> WTAP_ENCAP: 1
>> Arrival Time: Jan 4, 2013 11:30:27.204120000 EST
>> [Time shift for this packet: 0.000000000 seconds]
>> Epoch Time: 1357317027.204120000 seconds
>> [Time delta from previous captured frame: 0.000016000 seconds]
>> [Time delta from previous displayed frame: 2.004253000 seconds]
>> [Time since reference or first frame: 2.004253000 seconds]
>> Frame Number: 3
>> Frame Length: 126 bytes (1008 bits)
>> Capture Length: 126 bytes (1008 bits)
>> [Frame is marked: False]
>> [Frame is ignored: False]
>> [Protocols in frame: eth:ip:udp:cflow]
>> [Coloring Rule Name: UDP]
>> [Coloring Rule String: udp]
>> Ethernet II, Src: Cisco_ba:ba:40 (f4:ac:c1:ba:ba:40), Dst: Vmware_a5:70:ae
>> (00:0c:29:a5:70:ae)
>> Destination: Vmware_a5:70:ae (00:0c:29:a5:70:ae)
>> Address: Vmware_a5:70:ae (00:0c:29:a5:70:ae)
>> .... ..0. .... .... .... .... = LG bit: Globally unique address
>> (factory default)
>> .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
>> Source: Cisco_ba:ba:40 (f4:ac:c1:ba:ba:40)
>> Address: Cisco_ba:ba:40 (f4:ac:c1:ba:ba:40)
>> .... ..0. .... .... .... .... = LG bit: Globally unique address
>> (factory default)
>> .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
>> Type: IP (0x0800)
>> Internet Protocol Version 4, Src: 10.219.49.1 (10.219.49.1), Dst:
>> 10.219.51.130 (10.219.51.130)
>> Version: 4
>> Header length: 20 bytes
>> Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00:
>> Not-ECT (Not ECN-Capable Transport))
>> 0000 00.. = Differentiated Services Codepoint: Default (0x00)
>> .... ..00 = Explicit Congestion Notification: Not-ECT (Not
>> ECN-Capable Transport) (0x00)
>> Total Length: 112
>> Identification: 0x8196 (33174)
>> Flags: 0x00
>> 0... .... = Reserved bit: Not set
>> .0.. .... = Don't fragment: Not set
>> ..0. .... = More fragments: Not set
>> Fragment offset: 0
>> Time to live: 254
>> Protocol: UDP (17)
>> Header checksum: 0xc0ad [correct]
>> [Good: True]
>> [Bad: False]
>> Source: 10.219.49.1 (10.219.49.1)
>> Destination: 10.219.51.130 (10.219.51.130)
>> [Source GeoIP: Unknown]
>> [Destination GeoIP: Unknown]
>> User Datagram Protocol, Src Port: 22919 (22919), Dst Port: 4901 (4901)
>> Source port: 22919 (22919)
>> Destination port: 4901 (4901)
>> Length: 92
>> Checksum: 0x0000 (none)
>> [Good Checksum: False]
>> [Bad Checksum: False]
>> Cisco NetFlow/IPFIX
>> Version: 9
>> Count: 1
>> SysUptime: 245081424
>> Timestamp: Jan 4, 2013 11:30:27.000000000 EST
>> CurrentSecs: 1357317027
>> FlowSequence: 491926
>> SourceId: 2081
>> FlowSet 1
>> FlowSet Id: (Data) (260)
>> FlowSet Length: 64
>> Flow 1
>> Packets: 2
>> Octets: 3000
>> SrcAddr: www.keek.com (66.207.211.183)
>> DstAddr: cds56.sin.llnw.net (117.121.249.76)
>> InputInt: 54
>> OutputInt: 86
>> [Duration: 6.316000000 seconds]
>> StartTime: 245050.515000000 seconds
>> EndTime: 245056.831000000 seconds
>> SrcPort: 80
>> DstPort: 30653
>> SrcAS: 0
>> DstAS: 38621
>> BGPNextHop: 209.29.130.241 (209.29.130.241)
>> SrcMask: 28
>> DstMask: 22
>> Protocol: 6
>> TCP Flags: 0x10
>> IP ToS: 0x00
>> Direction: Egress (1)
>> Forwarding Status: Forward: Forwarded (Unknown)
>> 01.. .... = ForwdStat: Forward (1)
>> ..00 0000 = ForwdCode: Forwarded (Unknown) (0)
>> SamplerID: 1
>> Padding (3 bytes)
>>
>> 0000 00 0c 29 a5 70 ae f4 ac c1 ba ba 40 08 00 45 00 ..)[email protected].
>> 0010 00 70 81 96 00 00 fe 11 c0 ad 0a db 31 01 0a db .p..........1...
>> 0020 33 82 59 87 13 25 00 5c 00 00 00 09 00 01 0e 9b 3.Y..%.\........
>> 0030 a5 50 50 e7 03 a3 00 07 81 96 00 00 08 21 01 04 .PP..........!..
>> 0040 00 40 00 00 00 02 00 00 0b b8 42 cf d3 b7 75 79 [email protected]
>> 0050 f9 4c 00 00 00 36 00 00 00 56 0e 9b 45 3f 0e 9b .L...6...V..E?..
>> 0060 2c 93 00 50 77 bd 00 00 00 00 00 00 96 dd d1 1d ,..Pw...........
>> 0070 82 f1 1c 16 06 10 00 01 40 00 01 00 00 00 ........@.....
>>
>> No. Time VLAN Source Destination
>> Protocol Length Info
>> TCP Win Value TCP Win Scale TCP Win Size MPLS Label
>> 30 4.012722 10.219.49.1 10.219.51.130
>> CFLOW 154 total: 1 (v9) record
>>
>>
>> Frame 30: 154 bytes on wire (1232 bits), 154 bytes captured (1232 bits)
>> WTAP_ENCAP: 1
>> Arrival Time: Jan 4, 2013 11:30:29.212589000 EST
>> [Time shift for this packet: 0.000000000 seconds]
>> Epoch Time: 1357317029.212589000 seconds
>> [Time delta from previous captured frame: 0.000003000 seconds]
>> [Time delta from previous displayed frame: 2.008469000 seconds]
>> [Time since reference or first frame: 4.012722000 seconds]
>> Frame Number: 30
>> Frame Length: 154 bytes (1232 bits)
>> Capture Length: 154 bytes (1232 bits)
>> [Frame is marked: False]
>> [Frame is ignored: False]
>> [Protocols in frame: eth:ip:udp:cflow]
>> [Coloring Rule Name: UDP]
>> [Coloring Rule String: udp]
>> Ethernet II, Src: Cisco_ba:ba:40 (f4:ac:c1:ba:ba:40), Dst: Vmware_a5:70:ae
>> (00:0c:29:a5:70:ae)
>> Destination: Vmware_a5:70:ae (00:0c:29:a5:70:ae)
>> Address: Vmware_a5:70:ae (00:0c:29:a5:70:ae)
>> .... ..0. .... .... .... .... = LG bit: Globally unique address
>> (factory default)
>> .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
>> Source: Cisco_ba:ba:40 (f4:ac:c1:ba:ba:40)
>> Address: Cisco_ba:ba:40 (f4:ac:c1:ba:ba:40)
>> .... ..0. .... .... .... .... = LG bit: Globally unique address
>> (factory default)
>> .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
>> Type: IP (0x0800)
>> Internet Protocol Version 4, Src: 10.219.49.1 (10.219.49.1), Dst:
>> 10.219.51.130 (10.219.51.130)
>> Version: 4
>> Header length: 20 bytes
>> Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00:
>> Not-ECT (Not ECN-Capable Transport))
>> 0000 00.. = Differentiated Services Codepoint: Default (0x00)
>> .... ..00 = Explicit Congestion Notification: Not-ECT (Not
>> ECN-Capable Transport) (0x00)
>> Total Length: 140
>> Identification: 0x81b1 (33201)
>> Flags: 0x00
>> 0... .... = Reserved bit: Not set
>> .0.. .... = Don't fragment: Not set
>> ..0. .... = More fragments: Not set
>> Fragment offset: 0
>> Time to live: 254
>> Protocol: UDP (17)
>> Header checksum: 0xc076 [correct]
>> [Good: True]
>> [Bad: False]
>> Source: 10.219.49.1 (10.219.49.1)
>> Destination: 10.219.51.130 (10.219.51.130)
>> [Source GeoIP: Unknown]
>> [Destination GeoIP: Unknown]
>> User Datagram Protocol, Src Port: 22919 (22919), Dst Port: 4901 (4901)
>> Source port: 22919 (22919)
>> Destination port: 4901 (4901)
>> Length: 120
>> Checksum: 0x0000 (none)
>> [Good Checksum: False]
>> [Bad Checksum: False]
>> Cisco NetFlow/IPFIX
>> Version: 9
>> Count: 1
>> SysUptime: 245083432
>> Timestamp: Jan 4, 2013 11:30:29.000000000 EST
>> CurrentSecs: 1357317029
>> FlowSequence: 491953
>> SourceId: 2081
>> FlowSet 1
>> FlowSet Id: Data Template (V9) (0)
>> FlowSet Length: 92
>> Template (Id = 260, Count = 21)
>> Template Id: 260
>> Field Count: 21
>> Field (1/21): PKTS
>> Type: PKTS (2)
>> Length: 4
>> Field (2/21): BYTES
>> Type: BYTES (1)
>> Length: 4
>> Field (3/21): IP_SRC_ADDR
>> Type: IP_SRC_ADDR (8)
>> Length: 4
>> Field (4/21): IP_DST_ADDR
>> Type: IP_DST_ADDR (12)
>> Length: 4
>> Field (5/21): INPUT_SNMP
>> Type: INPUT_SNMP (10)
>> Length: 4
>> Field (6/21): OUTPUT_SNMP
>> Type: OUTPUT_SNMP (14)
>> Length: 4
>> Field (7/21): LAST_SWITCHED
>> Type: LAST_SWITCHED (21)
>> Length: 4
>> Field (8/21): FIRST_SWITCHED
>> Type: FIRST_SWITCHED (22)
>> Length: 4
>> Field (9/21): L4_SRC_PORT
>> Type: L4_SRC_PORT (7)
>> Length: 2
>> Field (10/21): L4_DST_PORT
>> Type: L4_DST_PORT (11)
>> Length: 2
>> Field (11/21): SRC_AS
>> Type: SRC_AS (16)
>> Length: 4
>> Field (12/21): DST_AS
>> Type: DST_AS (17)
>> Length: 4
>> Field (13/21): BGP_NEXT_HOP
>> Type: BGP_NEXT_HOP (18)
>> Length: 4
>> Field (14/21): SRC_MASK
>> Type: SRC_MASK (9)
>> Length: 1
>> Field (15/21): DST_MASK
>> Type: DST_MASK (13)
>> Length: 1
>> Field (16/21): PROTOCOL
>> Type: PROTOCOL (4)
>> Length: 1
>> Field (17/21): TCP_FLAGS
>> Type: TCP_FLAGS (6)
>> Length: 1
>> Field (18/21): IP_TOS
>> Type: IP_TOS (5)
>> Length: 1
>> Field (19/21): DIRECTION
>> Type: DIRECTION (61)
>> Length: 1
>> Field (20/21): FORWARDING_STATUS
>> Type: FORWARDING_STATUS (89)
>> Length: 1
>> Field (21/21): FLOW_SAMPLER_ID
>> Type: FLOW_SAMPLER_ID (48)
>> Length: 2
>>
>> 0000 00 0c 29 a5 70 ae f4 ac c1 ba ba 40 08 00 45 00 ..)[email protected].
>> 0010 00 8c 81 b1 00 00 fe 11 c0 76 0a db 31 01 0a db .........v..1...
>> 0020 33 82 59 87 13 25 00 78 00 00 00 09 00 01 0e 9b 3.Y..%.x........
>> 0030 ad 28 50 e7 03 a5 00 07 81 b1 00 00 08 21 00 00 .(P..........!..
>> 0040 00 5c 01 04 00 15 00 02 00 04 00 01 00 04 00 08 .\..............
>> 0050 00 04 00 0c 00 04 00 0a 00 04 00 0e 00 04 00 15 ................
>> 0060 00 04 00 16 00 04 00 07 00 02 00 0b 00 02 00 10 ................
>> 0070 00 04 00 11 00 04 00 12 00 04 00 09 00 01 00 0d ................
>> 0080 00 01 00 04 00 01 00 06 00 01 00 05 00 01 00 3d ...............=
>> 0090 00 01 00 59 00 01 00 30 00 02 ...Y...0..
>>
>>
>> ------------------------------------------------------------------------------
>> Master HTML5, CSS3, ASP.NET, MVC, AJAX, Knockout.js, Web API and
>> much more. Get web development skills now with LearnDevNow -
>> 350+ hours of step-by-step video tutorials by Microsoft MVPs and experts.
>> SALE $99.99 this month only -- learn more at:
>> http://p.sf.net/sfu/learnmore_122812
>> _______________________________________________
>> Nfsen-discuss mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
>>
>
> --
> Be nice to your netflow data. Use NfSen and nfdump :)
------------------------------------------------------------------------------
Master HTML5, CSS3, ASP.NET, MVC, AJAX, Knockout.js, Web API and
much more. Get web development skills now with LearnDevNow -
350+ hours of step-by-step video tutorials by Microsoft MVPs and experts.
SALE $99.99 this month only -- learn more at:
http://p.sf.net/sfu/learnmore_122812
_______________________________________________
Nfsen-discuss mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
