Re: Sflow billing or usage calculation software
On Tue, Apr 16, 2019 at 8:35 PM Deepak Jain wrote: > Now I know I'm pushing my luck... but do certain vendors more fully > embrace sFlow than others? maybe one of the whitebox vendors if not one > of the majors? > > Hacking support into something isn't the worse thing in the world, but > if there is any experience on this to leverage off of, that is helpful. > Unfortunately, there isn't a publicly available list showing how well or completely vendors have implemented the sFlow specifications: https://sflow.org/developers/specifications.php I have been working on an sFlow test suite to try and address this problem: https://blog.sflow.com/2015/11/sflow-test.html The source code for the tests is on GitHub: https://github.com/sflow-rt/sflow-test The easiest way to run the software is using Docker: https://hub.docker.com/r/sflow/sflow-test The goal is to compile a list of equipment and network operating systems that pass the tests and publish the results on sFlow.org. Failed tests can be passed to vendors to help them improve their implementations. In addition to identifying feature support, there are also stress tests to ensure accurate results under production workloads (rapid detection of DDoS etc). Involvement of operators would be great. If there are tests that are missing from the suite, please submit an enhancement request, or even better, a pull request, on GitHub. If you have a test lab and can run the tests on your own hardware, please share the results. The open source Host sFlow agent, https://sflow.net/, has been ported to a number of white box network operating systems and provides an opportunity for the community to extend sFlow functionality and address issues in the white box ecosystem. Operator involvement in this project would be most welcome. Peter
Re: Sflow billing or usage calculation software
Thanks for the pointers and suggestions! Now I know I'm pushing my luck... but do certain vendors more fully embrace sFlow than others? maybe one of the whitebox vendors if not one of the majors? Hacking support into something isn't the worse thing in the world, but if there is any experience on this to leverage off of, that is helpful. TIA, Deepak On 4/16/2019 6:30 AM, Yang Yu wrote: On Tue, Apr 16, 2019 at 2:14 AM Nick Morrison wrote: Actually the sflow standard is flexible, and there are many fields widely available, including input interface and output interface, vlan/vxlan/mpls headers, etc. The sending device just needs to support the fields. Vendor support for sFlow extended data types seems to be very limited and there are quite a few caveats on when the data is missing/inaccurate. RFC5472 Section 4.2 Using IPFIX for Billing (Reliability Limitations) might be applicable to sFlow as well. https://tools.ietf.org/html/rfc5472#section-4.2 Yang
Re: Sflow billing or usage calculation software
>On Tue, Apr 16, 2019 at 2:14 AM Nick Morrison wrote: > > Actually the sflow standard is flexible, and there are many fields widely > available, including input interface and output interface, vlan/vxlan/mpls > headers, etc. The sending device just needs to support the fields. Vendor support for sFlow extended data types seems to be very limited and there are quite a few caveats on when the data is missing/inaccurate. RFC5472 Section 4.2 Using IPFIX for Billing (Reliability Limitations) might be applicable to sFlow as well. https://tools.ietf.org/html/rfc5472#section-4.2 Yang
Re: Sflow billing or usage calculation software
> On 16. Apr 2019, at 00:21, Deepak Jain wrote: > > I'm only aware of Sflow being IP/protocol/etc aware. Actually the sflow standard is flexible, and there are many fields widely available, including input interface and output interface, vlan/vxlan/mpls headers, etc. The sending device just needs to support the fields. To give you an idea of some of the fields you can query from a TS server for example, here’s a description of one of the tables in the database: https://inmon.com/sentinel_help/8.0/help/en/report/api_view_traffic.shtml Browse sflow.org for more fun info, including ideas for running sflow agents on your hypervisors (for eg correlating CPU and RAM usage per VM) etc :-) Nick
RE: Sflow billing or usage calculation software
(I'm out of practice with mailing lists, apologies in advance) Dove tailing on this request... not sure its worth another thread. Is there a good Sflow-way or Sflow+something way to link all the traffic flow from a physical port for this kind (or any kind) of inspection? One way would be to suck down all the IP configs (and learned addresses ala BGP) and perform complex analysis of the Sflow database. I'm hoping there is something more intuitive... so you could say port 5 on switch xxx has this % TCP traffic vs this % UDP traffic (for example). I'm only aware of Sflow being IP/protocol/etc aware. thanks in advance, Deepak -Original Message- From: NANOG [mailto:nanog-bounces+deepak=ai@nanog.org] On Behalf Of Nick Morrison Sent: Sunday, April 14, 2019 8:09 AM To: Tony C Cc: nanog Subject: Re: Sflow billing or usage calculation software On 14. Apr 2019, at 13:28, Tony C wrote: > > Please keep the suggestions coming. I’ve had good results using Traffic Sentinel from Inmon. It’s got a nice queriable database backend and you don’t have to do much manual setup to get good results. The UI feels a bit 1995, but it works, and the API is practical and useful. It’s quite fast, too. They can probably give you trial licenses to see if it works for you. Nick
Re: Sflow billing or usage calculation software
On 14. Apr 2019, at 13:28, Tony C wrote: > > Please keep the suggestions coming. I’ve had good results using Traffic Sentinel from Inmon. It’s got a nice queriable database backend and you don’t have to do much manual setup to get good results. The UI feels a bit 1995, but it works, and the API is practical and useful. It’s quite fast, too. They can probably give you trial licenses to see if it works for you. Nick
RE: Sflow billing or usage calculation software
Hi Tom The 50Mbps is just an example, it’s the function we are more after. The pmacct option looks interesting I wonder if this can be integrated with IPAM I also found the UTM5 ISP Billing System which looks interesting anyone using it ? or not using it for specific reasons ? Please keep the suggestions coming. Tony From: Tom Beecher [mailto:beec...@beecher.cc] Sent: Sunday, April 14, 2019 9:24 AM To: Peter Phaal Cc: Tony C ; nanog Subject: Re: Sflow billing or usage calculation software I’m curious what the service is that 50Mbps avg over a 24 hr window is an investigative threshold. On Sat, Apr 13, 2019 at 17:57 Peter Phaal mailto:peter.ph...@gmail.com> > wrote: Tony, You might find the following article useful in identifying features to consider when evaluating sFlow analyzers: https://blog.sflow.com/2009/05/choosing-sflow-analyzer.html The following white paper discusses accuracy of packet sampling for usage accounting: https://inmon.com/pdf/sFlowBilling.pdf Peter On Sat, Apr 13, 2019 at 2:07 PM Tony C mailto:tonyc...@iinet.net.au> > wrote: Hi All I am looking for Sflow analytical software that can tell me automatically over say a period of 24 hours (or any time period I select) the average mbit of data consumed for any IP address within our entire AS. (Without configuring a rule or billing group for each IP address or customer within our network) The purpose is to help quickly work out IP addressees which are using more bandwidth (in or out) than what we consider to be acceptable usage. For example, I would like to review a report or be automatically alerted to any IP address using more than an average of 50mbit within the past 24 hour plus have the capability to review data say over a month. Any names of software of suggestions would be great which I can investigate, happy to look at both commercial software and open source or if you have a Sflow billing solution for data consumption which is simple and easy to use please let me know Thanks Tony
Re: Sflow billing or usage calculation software
I’m curious what the service is that 50Mbps avg over a 24 hr window is an investigative threshold. On Sat, Apr 13, 2019 at 17:57 Peter Phaal wrote: > Tony, > > You might find the following article useful in identifying features to > consider when evaluating sFlow analyzers: > https://blog.sflow.com/2009/05/choosing-sflow-analyzer.html > > The following white paper discusses accuracy of packet sampling for usage > accounting: > https://inmon.com/pdf/sFlowBilling.pdf > > > Peter > > > > On Sat, Apr 13, 2019 at 2:07 PM Tony C wrote: > >> Hi All >> >> I am looking for Sflow analytical software that can tell me automatically >> over say a period of 24 hours (or any time period I select) the average >> mbit of data consumed for any IP address within our entire AS. >> >> (Without configuring a rule or billing group for each IP address or >> customer within our network) >> >> The purpose is to help quickly work out IP addressees which are using >> more bandwidth (in or out) than what we consider to be acceptable usage. >> >> For example, I would like to review a report or be automatically alerted >> to any IP address using more than an average of 50mbit within the past 24 >> hour plus have the capability to review data say over a month. >> >> Any names of software of suggestions would be great which I can >> investigate, happy to look at both commercial software and open source or >> if you have a Sflow billing solution for data consumption which is simple >> and easy to use please let me know >> >> Thanks >> >> Tony >> >
Re: Sflow billing or usage calculation software
Tony, You might find the following article useful in identifying features to consider when evaluating sFlow analyzers: https://blog.sflow.com/2009/05/choosing-sflow-analyzer.html The following white paper discusses accuracy of packet sampling for usage accounting: https://inmon.com/pdf/sFlowBilling.pdf Peter On Sat, Apr 13, 2019 at 2:07 PM Tony C wrote: > Hi All > > I am looking for Sflow analytical software that can tell me automatically > over say a period of 24 hours (or any time period I select) the average > mbit of data consumed for any IP address within our entire AS. > > (Without configuring a rule or billing group for each IP address or > customer within our network) > > The purpose is to help quickly work out IP addressees which are using more > bandwidth (in or out) than what we consider to be acceptable usage. > > For example, I would like to review a report or be automatically alerted > to any IP address using more than an average of 50mbit within the past 24 > hour plus have the capability to review data say over a month. > > Any names of software of suggestions would be great which I can > investigate, happy to look at both commercial software and open source or > if you have a Sflow billing solution for data consumption which is simple > and easy to use please let me know > > Thanks > > Tony >
RE: Sflow billing or usage calculation software
Tony, Take a look at pmacct, it will be able to handle your needs with a number of modifications. The section I linked below should give you a good starting point. Change the traffic dump to a MySQL database, add some indexes, craft some SQL queries, then you're off to the races. As for billing notifications, a cron script would need to calculate the usages, and alert based on your set thresholds. http://wiki.pmacct.net/OfficialExamples - XVII. Using pmacct as traffic/event logger For added bonus points, combine it with a BGP feed, and know where your traffic is going outbound, that way intelligent routing changes can be made much quicker. -- Ryan Hamel Network Administrator ryan.ha...@quadranet.com | +1 (888) 578-2372 QuadraNet Enterprises, LLC. | Dedicated Servers, Colocation, Cloud From: NANOG On Behalf Of Tony C Sent: Friday, April 12, 2019 8:22 PM To: nanog@nanog.org Subject: Sflow billing or usage calculation software Hi All I am looking for Sflow analytical software that can tell me automatically over say a period of 24 hours (or any time period I select) the average mbit of data consumed for any IP address within our entire AS. (Without configuring a rule or billing group for each IP address or customer within our network) The purpose is to help quickly work out IP addressees which are using more bandwidth (in or out) than what we consider to be acceptable usage. For example, I would like to review a report or be automatically alerted to any IP address using more than an average of 50mbit within the past 24 hour plus have the capability to review data say over a month. Any names of software of suggestions would be great which I can investigate, happy to look at both commercial software and open source or if you have a Sflow billing solution for data consumption which is simple and easy to use please let me know Thanks Tony
