No prob Aaron, glad to hear it's behaving as [sort-of] expected! How much you want to sample is ultimately one of those "it depends" situations. Line-rate netflow for millions of flows will generate a lot of data and in some cases may not be supported by the exporting hardware itself regardless of traffic rate. I believe the config guide for Cisco's ASR9k line explicitly states that only sampled netflow is supported; there is no support for "full-mode" sampling -- though I've never personally tested that.
You are missing out on some packet data, though. If for example you send a single packet across your network and it is not the one sampled in that random-out-of-512 packet interval, you will not see data for the flow in nfsen. That's not a deficiency in nfsen/nfdump/etc, it's just packet-sampling on the exporting router working as intended. nfdump can't process what it doesn't receive to begin with. : ) One samples to get an approximation of traffic; a higher sample rate will provide more granularity at the expense of more CPU on the exporting router, more bandwidth used to send the flow data to the collector, and storage space and processing burdens on the collector (among other things I'm liable to miss). A lower sample rate means you miss out on some data, but you can get away with using smaller-scale export/collector hardware. Given enough money and hardware, I suppose there's no limit to what could conceivably be exported, retained, and processed, but I've personally never had the need to design for line-rate netflow in large environments and cannot comment authoritatively. Regards, Ron On Thu, Jul 11, 2013 at 1:52 PM, Aaron <[email protected]> wrote: > sampler-map sm > random 1 out-of 512 > > that helps immensely! Thanks Ron! > > I see about 7 gbps on my mrtg inet graphs... > I see about 14 mbps on my nfsen graphs... > You just taught me that this is because of the sampler on my routers which is > 1/512... > 14 * 512 = 7,168 > > Thanks Ron > > Also, is there a reason why I should use sampler 1 of 512 ? what if I didn't > use a sampler at all? Would this mean I would rcv a lot more netflow > samples? Is this a bad thing? Am I missing out on some netflow info by > sampling or does this just cut down on the exported netflow info via my > transit network ? > > Aaron > > -----Original Message----- > From: Ron Arsenault [mailto:[email protected]] > Sent: Thursday, July 11, 2013 9:27 AM > To: Aaron > Cc: Vinicius Esteves; [email protected] > Subject: Re: [Nfsen-discuss] nfsen bps any protocol graph > > Hi Aaron, > > Do you have sampling configured on your netflow exporters? Netflow apps will > only have data for flows that they see, and sampling will reduce that value > significantly. For example, my aggregate inbound traffic is just shy of 20 > Gbps, but as I only sample 1-out-of-6000 random packets, my nfsen bps graph > peaks at around 3 Mbps. > > SNMP-based apps a la MRTG behave differently; since they're just graphing > interface counters/OIDs (instead of extrapolating based on netflow samples), > they have a better glimpse into raw bits flowing across your links. > > Does this help? > > Regards, > > Ron > > On Thu, Jul 11, 2013 at 9:41 AM, Aaron <[email protected]> wrote: >> On the http://ipaddress/nfsen/nfsen.php when i click on the “Bits/s >> any protocol” graph I see that the peak usage 21:00 – 22:00 is only >> about 13 M (I think that means 13 Mbps)….. this is not what my mrtg >> shows….my mrtg shows that my dual 10 gig uplinks usage is about 7 gbps >> around that time of the night. Is this nfsen “Bits/s any protocol” >> graph supposed to be a literal representation of the traffic that >> flows through those dual 10 gbps internet uplinks? I will say this… >> mrtg that I speak of is monitoring the same (2) 10 gig interfaces that >> I have my netflow exporting from to nfsen….so this should be an apples >> to apples comparison as far as I can see…. Lemme know what yall think please. >> >> >> >> >> >> Aaron >> >> >> >> From: Vinicius Esteves [mailto:[email protected]] >> Sent: Wednesday, July 10, 2013 7:37 PM >> To: Aaron >> Cc: [email protected] >> Subject: Re: [Nfsen-discuss] nfsen bps any protocol graph >> >> >> >> Even accounting traffic of both directions ? >> >> >> >> 2013/7/10 Aaron <[email protected]> >> >> Why is it that my nfsen bps any protocol graph doesn’t match anywhere >> close to what my mrtg bps graph shows of my actual internet uplink >> utilization ? >> >> >> >> Aaron >> >> >> ---------------------------------------------------------------------- >> -------- See everything from the browser to the database with >> AppDynamics Get end-to-end visibility with application monitoring from >> AppDynamics Isolate bottlenecks and diagnose root cause in seconds. >> Start your free trial of AppDynamics Pro today! >> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.c >> lktrk _______________________________________________ >> Nfsen-discuss mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/nfsen-discuss >> >> >> >> >> ---------------------------------------------------------------------- >> -------- See everything from the browser to the database with >> AppDynamics Get end-to-end visibility with application monitoring from >> AppDynamics Isolate bottlenecks and diagnose root cause in seconds. >> Start your free trial of AppDynamics Pro today! >> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.c >> lktrk _______________________________________________ >> Nfsen-discuss mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/nfsen-discuss >> > ------------------------------------------------------------------------------ See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk _______________________________________________ Nfsen-discuss mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
