From experience, sflows are horribly inaccurate for DDoS detection, since the volume could disrupt the control plane and render the process useless, thus not giving data to the external system to act upon it. You can't get any better than mirroring your inbound transit, and sampling the output to a sensor.
I have also spent some time on a spare server running netmap and influxdb, it simply could not keep up, nor could I write any useful queries against it. Which is why I'm deciding on using pf_ring ft (flow table) and just let it calculate all the data to be dumped into a MySQL memory table, and also harvesting ntop's NDPI protocol decoding, to enhance the detection rules. My ideal setup is to break things down into dedicated programs like flow exporter, detection, and response. I also want to make clear to Michel, that colo'ing a router at an ISP is no different than plugging it into your local router, your uplink will get saturated beyond what it can physically handle with only the ACLs protecting the other side, but if your clients are also receiving traffic on the same uplink as the attack, it's a denial of service to them. -----Original Message----- From: NANOG <nanog-boun...@nanog.org> On Behalf Of H I Baysal Sent: Friday, August 31, 2018 2:09 AM To: Michel Py <michel...@tsisemi.com>; Aaron Gould <aar...@gvtc.com>; mic...@arneill-py.sacramento.ca.us Cc: Nanog@nanog.org Subject: Re: automatic rtbh trigger using flow data Most of the solutions mentioned are paid, or fastnetmon is partially paid. And the thing you want is paid i believe.... Nice tool though, not saying anything against it. However.... My personal view is, as long as you can store your flow info in a timeseries database (like influxdb and NOT SQL LIKE!!!!!!!) you can do whatever you want with the (raw) data. And create custom triggers for different calculations. Flows are on the fly and are coming in constantly, you could have a calculation like group by srcip and whatever protocol you want or just srcip, and make a calculation for every x seconds or minutes. As i mentioned the flow data is a constant stream, so you could have it triggered as fast as you want. (and the nice thing is, with sflow, you also get as path, peer as, localpref,community (if enabled). You could group by anything.. :) I admit it takes a bit more time to setup but the outcome is amazing ;) (especially if you graph it then with grafana) And in your case it would be a script that does a influxdb command to make the calculations and if the outcome shows an IP meeting the thresholds you have set in the calculation, you trigger a script that adjusts the route to be announced to your upstream with the correct (rtbh) community. ( as i mentioned, as long as you have the "raw" flows, you can do anything ) Good luck, whatever you choose :) On 31-08-18 02:14, Michel Py wrote: >> Aaron Gould wrote : >> I'm really surprised that you all are doing this based on source ip, >> simply because I thought the distribution of botnet members around the world >> we're so extensive that I never really thought it possible to filter based >> on sources, if so I'd like to see the list too. > I emailed you. For years I ran it at home on a Cisco 1841, 100,000 BGP > prefixes is nothing these days. I am not surprised that Joe pushes that to > some CPEs. > >> Even so, this would not stop the attacks from hitting my front door, >> my side of my Internet uplink...when paying for a 30 gigs CIR and >> paying double for megabits per second over that, up to the ceiling of 100 >> gig every bit that hits my front door over 30 gig would cost me extra, >> remotely triggering based on my victim IP address inside my network would be >> my solution to saving money. > I agree. If you want to get a real use of source blacklisting, to save > bandwidth, you probably went to rent a U in a rack at your upstream(s) to > block it there. > I never did it past 1GE, and I have never measured seriously the bandwidth it > would save, would be curious to know. > I think the two approaches are complementary to each other though. > > Michel. > > > On Aug 30, 2018, at 6:43 PM, Michel Py <michel...@tsisemi.com> wrote: > >>> Joe Maimon wrote : >>> I use a bunch of scripts plus a supervisory sqlite3 database process >>> all injecting into quagga >> I have the sqlite part planned, today I'm using a flat file :-( I >> know :-( >> >>> Also aimed at attacker sources. I feed it with honeypots and live servers, >>> hooked into fail2ban and using independent host scripts. Not very >>> sophisticated, the remotes use ssh executed commands to add/delete. I also >>> setup a promiscuous ebgp RR so I can extend my umbrella to CPE with diverse >>> connectivity. >> I would like to have your feed. How many attacker prefixes do you currently >> have ? >> >>> Using flow data, that sounds like an interesting direction to take this >>> into, so thank you! >> The one thing we can share here is the attacker prefixes. The victim >> prefixes are unique to each of us but I expect our attacker prefixes to be >> very close. >> >> Michel. >> >> TSI Disclaimer: This message and any files or text attached to it are >> intended only for the recipients named above and contain information that >> may be confidential or privileged. If you are not the intended recipient, >> you must not forward, copy, use or otherwise disclose this communication or >> the information contained herein. In the event you have received this >> message in error, please notify the sender immediately by replying to this >> message, and then delete all copies of it from your system. Thank you!...
