Re: automatic rtbh trigger using flow data

[H I Baysal]

I also looked at PMACCT, however I have to say that I am way more 
pleased with the current setup.
PMACCT relies on bgp session etc, while you actually get that info from 
your (s)flow. No need for lookup.


And I really didn't like the idea of SQL like DB, I know i could push it 
to influx. But i thought this could be done way simpler and i think i 
did exactly that.

I store the "raw" info for 7 days, as detailed as possible.
then downsample for a week again, a month and a year.

And the picture rendering api function of grafana makes it possible for 
me to do for example /dst AS# or /peer #AS or /as-path AS#  in a 
chatbot, and grafana actually pushes the graph to telegram in our case, 
but it could be any bot.


So at peering conferences, i can see with 1 message how much traffic we 
do either to a peer, destination or if traffic passes your AS.

I don't see any (pre-made) system beating this :P



On 01-09-18 01:51, Paweł Małachowski wrote:

On Fri, Aug 31, 2018 at 11:09:19AM +0200, H I Baysal wrote:


My personal view is, as long as you can store your flow info in a
timeseries database (like influxdb and NOT SQL LIKE!!!) you can do
whatever you want with the (raw) data. And create custom triggers for
different calculations.

For one of our customers I've deployed good old pmacct + MySQL
(using memory engine) backend for DDoS detection purposes.
It has some drawbacks (e.g. one has to frequently delete old
records to keep tables fit and fast) but it allows asking complex
SQL queries against these short term data (e.g. different detection
logic per subnets) or precompute with triggers.


Flows are on the fly and are coming in constantly, you could have a
calculation like group by srcip and whatever protocol you want or just
srcip,

Beware of high cardinality issues when facing random src IP floods.

BTW, once again pmacct (with some glue) is nice for feeding flow
data into time series database. It can pre aggregate and pre filter
low volume flows to reduce storage requirements.

Re: automatic rtbh trigger using flow data

[Paweł Małachowski]

On Fri, Aug 31, 2018 at 11:09:19AM +0200, H I Baysal wrote:

> My personal view is, as long as you can store your flow info in a 
> timeseries database (like influxdb and NOT SQL LIKE!!!) you can do 
> whatever you want with the (raw) data. And create custom triggers for 
> different calculations.

For one of our customers I've deployed good old pmacct + MySQL
(using memory engine) backend for DDoS detection purposes.
It has some drawbacks (e.g. one has to frequently delete old
records to keep tables fit and fast) but it allows asking complex
SQL queries against these short term data (e.g. different detection
logic per subnets) or precompute with triggers.

> Flows are on the fly and are coming in constantly, you could have a 
> calculation like group by srcip and whatever protocol you want or just 
> srcip,

Beware of high cardinality issues when facing random src IP floods.

BTW, once again pmacct (with some glue) is nice for feeding flow
data into time series database. It can pre aggregate and pre filter
low volume flows to reduce storage requirements.


-- 
Paweł Małachowski

Re: automatic rtbh trigger using flow data

[Baldur Norddahl]

I would redirect the packet to a VRF with one global drop UDP ACL. That
scales perfectly. There is probably many ways to implement such a feature.


søn. 2. sep. 2018 11.07 skrev Ryan Hamel :

> Baldur,
>
>
>
> Modifying the routing table with a next-hop change from a community, is
> different than having a line card filtering packets at layer 4, of course
> most if not all carriers will support it. Instead of doing normal TCAM
> route lookups, you’re getting into packet inspection territory, which is
> something completely different.
>
>
>
> Just quickly reading the ASR 9K documentation, it can only support 3K
> rules per system. Juniper – 8K, Alcatel-Lucent – 512. That’s pretty low
> considering I can put many /32s into a routing table very easily and
> without hassle.
>
>
>
> As I said before, no ISP is going to offer such filtering services for
> free when DDoS mitigation is a cash cow.
>
>
>
> Ryan Hamel
>
>
>
> *From:* NANOG  *On Behalf Of *Baldur Norddahl
> *Sent:* Sunday, September 02, 2018 1:42 AM
> *To:* nanog@nanog.org
> *Subject:* Re: automatic rtbh trigger using flow data
>
>
>
> This is not true. Some of our transits do RTBH for free. For example
> Cogent.
>
>
>
> They will not do FlowSpec. Maybe their equipment can not do it or for some
> other reason.
>
>
>
> However RTBH is a simple routing hack that can be implemented on any
> router. The traffic is dropped right at the edge and is never transported
> on the transit provider network. In that sense it also protects the transit
> network.
>
>
>
> RTBH only for UDP would also be a very simple hack on many routers.
>
>
>
> It might not be FlowSpec, but it may have most of the benefit, in a much
> simplified way.
>
>
>
> Regards
>
>
>
> Baldur
>
>
>
>
>
> søn. 2. sep. 2018 02.39 skrev Ryan Hamel :
>
> No ISP is in the business of filtering traffic unless the client pays the
> hefty fee since someone still has to tank the attack.
>
>
>
> I also don’t think there is destination prefix IP filtering in flowspec,
> which could seriously cause problems.
>
>
>
> *From:* NANOG  *On Behalf Of *Baldur Norddahl
> *Sent:* Saturday, September 01, 2018 5:18 PM
> *To:* nanog@nanog.org
> *Subject:* Re: automatic rtbh trigger using flow data
>
>
>
>
>
> fre. 31. aug. 2018 17.16 skrev Hugo Slabbert :
>
>
>
> I would love an upstream that accepts flowspec routes to get granular
> about
> drops and to basically push "stateless ACLs" upstream.
>
> _keeps dreaming_
>
>
>
>
>
> We just need a signal to drop UDP for a prefix. The same as RTBH but only
> for UDP. This would prevent all volumetric attacks without the end user
> being cut off completely.
>
>
>
> Besides from some games, VPN and VoIP, they would have an almost
> completely normal internet experience. DNS would go through the ISP servers
> and only be affected if the user is using a third party service.
>
>
>
> Regards
>
>
>
> Baldur
>
>
>
>

RE: automatic rtbh trigger using flow data

[Ryan Hamel]

Baldur,

Modifying the routing table with a next-hop change from a community, is 
different than having a line card filtering packets at layer 4, of course most 
if not all carriers will support it. Instead of doing normal TCAM route 
lookups, you’re getting into packet inspection territory, which is something 
completely different.

Just quickly reading the ASR 9K documentation, it can only support 3K rules per 
system. Juniper – 8K, Alcatel-Lucent – 512. That’s pretty low considering I can 
put many /32s into a routing table very easily and without hassle.

As I said before, no ISP is going to offer such filtering services for free 
when DDoS mitigation is a cash cow.

Ryan Hamel

From: NANOG  On Behalf Of Baldur Norddahl
Sent: Sunday, September 02, 2018 1:42 AM
To: nanog@nanog.org
Subject: Re: automatic rtbh trigger using flow data

This is not true. Some of our transits do RTBH for free. For example Cogent.

They will not do FlowSpec. Maybe their equipment can not do it or for some 
other reason.

However RTBH is a simple routing hack that can be implemented on any router. 
The traffic is dropped right at the edge and is never transported on the 
transit provider network. In that sense it also protects the transit network.

RTBH only for UDP would also be a very simple hack on many routers.

It might not be FlowSpec, but it may have most of the benefit, in a much 
simplified way.

Regards

Baldur


søn. 2. sep. 2018 02.39 skrev Ryan Hamel 
mailto:ryan.ha...@quadranet.com>>:
No ISP is in the business of filtering traffic unless the client pays the hefty 
fee since someone still has to tank the attack.

I also don’t think there is destination prefix IP filtering in flowspec, which 
could seriously cause problems.

From: NANOG mailto:nanog-boun...@nanog.org>> On Behalf 
Of Baldur Norddahl
Sent: Saturday, September 01, 2018 5:18 PM
To: nanog@nanog.org<mailto:nanog@nanog.org>
Subject: Re: automatic rtbh trigger using flow data


fre. 31. aug. 2018 17.16 skrev Hugo Slabbert 
mailto:h...@slabnet.com>>:


I would love an upstream that accepts flowspec routes to get granular about
drops and to basically push "stateless ACLs" upstream.

_keeps dreaming_


We just need a signal to drop UDP for a prefix. The same as RTBH but only for 
UDP. This would prevent all volumetric attacks without the end user being cut 
off completely.

Besides from some games, VPN and VoIP, they would have an almost completely 
normal internet experience. DNS would go through the ISP servers and only be 
affected if the user is using a third party service.

Regards

Baldur

Re: automatic rtbh trigger using flow data

[Baldur Norddahl]

This is not true. Some of our transits do RTBH for free. For example Cogent.

They will not do FlowSpec. Maybe their equipment can not do it or for some
other reason.

However RTBH is a simple routing hack that can be implemented on any
router. The traffic is dropped right at the edge and is never transported
on the transit provider network. In that sense it also protects the transit
network.

RTBH only for UDP would also be a very simple hack on many routers.

It might not be FlowSpec, but it may have most of the benefit, in a much
simplified way.

Regards

Baldur


søn. 2. sep. 2018 02.39 skrev Ryan Hamel :

> No ISP is in the business of filtering traffic unless the client pays the
> hefty fee since someone still has to tank the attack.
>
>
>
> I also don’t think there is destination prefix IP filtering in flowspec,
> which could seriously cause problems.
>
>
>
> *From:* NANOG  *On Behalf Of *Baldur Norddahl
> *Sent:* Saturday, September 01, 2018 5:18 PM
> *To:* nanog@nanog.org
> *Subject:* Re: automatic rtbh trigger using flow data
>
>
>
>
>
> fre. 31. aug. 2018 17.16 skrev Hugo Slabbert :
>
>
>
> I would love an upstream that accepts flowspec routes to get granular
> about
> drops and to basically push "stateless ACLs" upstream.
>
> _keeps dreaming_
>
>
>
>
>
> We just need a signal to drop UDP for a prefix. The same as RTBH but only
> for UDP. This would prevent all volumetric attacks without the end user
> being cut off completely.
>
>
>
> Besides from some games, VPN and VoIP, they would have an almost
> completely normal internet experience. DNS would go through the ISP servers
> and only be affected if the user is using a third party service.
>
>
>
> Regards
>
>
>
> Baldur
>
>
>

RE: automatic rtbh trigger using flow data

[Michel Py]

> Roland Dobbins wrote :
> I'm well aware of what's mentioned in the Arbor report, thanks!

I would not have guessed :P


> Ryan Hamel wrote :
> No ISP is in the business of filtering traffic unless the client pays the 
> hefty fee since someone still has to tank the atack.

I agree. In the end, it tends to favor who has the biggest one.
I meant the biggest bandwidth, of course.

I the foreseeable future, no blacklist system is going to replace what Arbor 
and consorts can provide : a pipe big enough to either route the DDOS attack to 
null0 or even better route it to somewhere it can be analyzed further.

Michel.

TSI Disclaimer:  This message and any files or text attached to it are intended 
only for the recipients named above and contain information that may be 
confidential or privileged. If you are not the intended recipient, you must not 
forward, copy, use or otherwise disclose this communication or the information 
contained herein. In the event you have received this message in error, please 
notify the sender immediately by replying to this message, and then delete all 
copies of it from your system. Thank you!...

Re: automatic rtbh trigger using flow data

[Hugo Slabbert]

On Sun 2018-Sep-02 00:39:40 +, Ryan Hamel  wrote:

No ISP is in the business of filtering traffic unless the client pays the 
hefty fee since someone still has to tank the attack.


If I can tag an RTBH community on a /32, what's the additional lost revenue 
in letting me be more granular and get down to the specific flows I want 
dropped?


"drop all traffic to x/32" would drop *more* traffic than "drop any traffic 
from address y to x/32, protocol TCP, port n".


I also don’t think there is destination prefix IP filtering in flowspec, 
which could seriously cause problems.


What now?  Unless I'm misunderstanding what you're saying, it's right in 
the spec[1]:


   A flow specification NLRI must be validated such that it is
   considered feasible if and only if:

   a) The originator of the flow specification matches the originator of
  the best-match unicast route for the destination prefix embedded
  in the flow specification.

   b) There are no more specific unicast routes, when compared with the
  flow destination prefix, that have been received from a different
  neighboring AS than the best-match unicast route, which has been
  determined in step a).

   By originator of a BGP route, we mean either the BGP originator path
   attribute, as used by route reflection, or the transport address of
   the BGP peer, if this path attribute is not present.

   The underlying concept is that the neighboring AS that advertises the
   best unicast route for a destination is allowed to advertise flow-
   spec information that conveys a more or equally specific destination
   prefix.  Thus, as long as there are no more specific unicast routes,
   received from a different neighboring AS, which would be affected by
   that filtering rule.

   The neighboring AS is the immediate destination of the traffic
   described by the flow specification.  If it requests these flows to
   be dropped, that request can be honored without concern that it
   represents a denial of service in itself.  Supposedly, the traffic is
   being dropped by the downstream autonomous system, and there is no
   added value in carrying the traffic to it.

--
Hugo Slabbert   | email, xmpp/jabber: h...@slabnet.com
pgp key: B178313E   | also on Signal

[1] https://tools.ietf.org/html/rfc5575



From: NANOG  On Behalf Of Baldur Norddahl
Sent: Saturday, September 01, 2018 5:18 PM
To: nanog@nanog.org
Subject: Re: automatic rtbh trigger using flow data


fre. 31. aug. 2018 17.16 skrev Hugo Slabbert 
mailto:h...@slabnet.com>>:


I would love an upstream that accepts flowspec routes to get granular about
drops and to basically push "stateless ACLs" upstream.

_keeps dreaming_


We just need a signal to drop UDP for a prefix. The same as RTBH but only for 
UDP. This would prevent all volumetric attacks without the end user being cut 
off completely.

Besides from some games, VPN and VoIP, they would have an almost completely 
normal internet experience. DNS would go through the ISP servers and only be 
affected if the user is using a third party service.

Regards

Baldur



signature.asc
Description: Digital signature

Re: automatic rtbh trigger using flow data

[Hugo Slabbert]

On Sun 2018-Sep-02 10:09:32 +0700, Roland Dobbins  wrote:



On 1 Sep 2018, at 1:43, Hugo Slabbert wrote:

Generally on the TCP side you can try SYN or ACK floods, but you're 
not going to get an amplified reflection.


Actually, TCP reflection/amplification has been on the increase; the 
attacker is guaranteed at least 4:1 amplification in most 
circumstances, the number of reflectors/amplifiers is for all 
practical purposes infinite, and they're mostly legitimate, 
non-broken services/applications.


Fair.  I guess in terms of common reflect/amp vector at $dayjob we just see 
UDP-based significantly more frequently on large volumetric attacks given 
the amp factor on some vectors is so huge.


Some relevant reading I need to revisit:
https://www.usenix.org/sites/default/files/conference/protected-files/woot14_slides_kuhrer.pdf
https://www.usenix.org/system/files/conference/woot14/woot14-kuhrer.pdf

And as always, it's important to note that with all 
reflection/amplification attacks, the root of the issue is the lack of 
universal source-address validation (SAV).  Without the ability to spoof, 
there would be no reflection/amplification attacks.


ACK, pun intended.


---
Roland Dobbins 


--
Hugo Slabbert   | email, xmpp/jabber: h...@slabnet.com
pgp key: B178313E   | also on Signal


signature.asc
Description: Digital signature

Re: automatic rtbh trigger using flow data

[Roland Dobbins]

On 1 Sep 2018, at 1:43, Hugo Slabbert wrote:

Generally on the TCP side you can try SYN or ACK floods, but you're 
not going to get an amplified reflection.


Actually, TCP reflection/amplification has been on the increase; the 
attacker is guaranteed at least 4:1 amplification in most circumstances, 
the number of reflectors/amplifiers is for all practical purposes 
infinite, and they're mostly legitimate, non-broken 
services/applications.


And as always, it's important to note that with all 
reflection/amplification attacks, the root of the issue is the lack of 
universal source-address validation (SAV).  Without the ability to 
spoof, there would be no reflection/amplification attacks.


---
Roland Dobbins 

Re: automatic rtbh trigger using flow data

[Roland Dobbins]

On 1 Sep 2018, at 1:20, Lotia, Pratik M wrote:

Arbor report mentions volumetric attacks using DNS, NTP form 75+% of 
the attacks.


I'm well aware of what's mentioned in the Arbor report, thanks!

;>


Then QoSing certain ports and protocols is the best way to start with.


The point is that when applying broad policies of this nature, one must 
be very conservative, else one can cause larger problems on a macro 
scale.  Internet ateriosclerosis is a significant issue.


---
Roland Dobbins 

Re: automatic rtbh trigger using flow data

[Roland Dobbins]

On 1 Sep 2018, at 1:35, Aaron Gould wrote:

I may mark internet-sourced-udp with a certain marking dscp/exp so 
that as it travels through my internet
network, it will be the first to get dropped (? Wred ? work well for 
udp?) during congestion when an attack gets through


You can use flow telemetry analysis to look at the UDP non-initial 
fragments destined for any access networks under your control; you'll 
likely see that they comprise a tiny portion of the overall traffic mix, 
and they're most commonly associated with large DNS answers.


Once you've determined the numbers, you can police down the non-initial 
fragments destined for the access networks you control (don't do this on 
transit traffic!) to whatever small percentage makes the most sense, 
with a bit of extra headroom.  1% of link bandwidth works for several 
operators.


In that QoS policy, you exempt well-known/well-run open DNS recursor 
farms like Google DNS, OpenDNS, et. al. (and possibly your own, 
depending on your topology, etc.), and any other legitimate source CIDRs 
which generate appreciable amounts of non-initial fragments.


When a reflection/amplification attack which involves non-initial 
fragments hits, the QoS policy will sink a significant proportion of the 
attack.  It doesn't help with your peering links, but keeps the traffic 
off your core and off the large network(s).


Again, don't apply this across-the-board; only do it for access networks 
within your span of administrative control.


* btw, what can you experts tell me about tcp-based volumetric 
attacks...


TCP reflection/amplification.

---
Roland Dobbins 

RE: automatic rtbh trigger using flow data

[Ryan Hamel]

No ISP is in the business of filtering traffic unless the client pays the hefty 
fee since someone still has to tank the attack.

I also don’t think there is destination prefix IP filtering in flowspec, which 
could seriously cause problems.

From: NANOG  On Behalf Of Baldur Norddahl
Sent: Saturday, September 01, 2018 5:18 PM
To: nanog@nanog.org
Subject: Re: automatic rtbh trigger using flow data


fre. 31. aug. 2018 17.16 skrev Hugo Slabbert 
mailto:h...@slabnet.com>>:


I would love an upstream that accepts flowspec routes to get granular about
drops and to basically push "stateless ACLs" upstream.

_keeps dreaming_


We just need a signal to drop UDP for a prefix. The same as RTBH but only for 
UDP. This would prevent all volumetric attacks without the end user being cut 
off completely.

Besides from some games, VPN and VoIP, they would have an almost completely 
normal internet experience. DNS would go through the ISP servers and only be 
affected if the user is using a third party service.

Regards

Baldur

Re: automatic rtbh trigger using flow data

[Baldur Norddahl]

fre. 31. aug. 2018 17.16 skrev Hugo Slabbert :

>
>
> I would love an upstream that accepts flowspec routes to get granular
> about
> drops and to basically push "stateless ACLs" upstream.
>
> _keeps dreaming_
>
>
>
We just need a signal to drop UDP for a prefix. The same as RTBH but only
for UDP. This would prevent all volumetric attacks without the end user
being cut off completely.

Besides from some games, VPN and VoIP, they would have an almost completely
normal internet experience. DNS would go through the ISP servers and only
be affected if the user is using a third party service.

Regards

Baldur

Re: automatic rtbh trigger using flow data

[Hugo Slabbert]

On Fri 2018-Aug-31 13:35:29 -0500, Aaron Gould  wrote:


* btw, what can you experts tell me about tcp-based volumetric attacks...
please help me to understand... does tcp have an inherent inability to
ramp-up to massive speeds/loads with it's sliding window and
must-rcv-ack-before sending more segments ??  I ask since I heard this years
ago about tcp and I wonder if this is why


UDP, depending on the application, can be reflected and amplified.  
Generally on the TCP side you can try SYN or ACK floods, but you're not 
going to get an amplified reflection.


--
Hugo Slabbert   | email, xmpp/jabber: h...@slabnet.com
pgp key: B178313E   | also on Signal



signature.asc
Description: Digital signature

RE: automatic rtbh trigger using flow data

[Aaron Gould]

(I think this is all about volumetric attacks btw...it's my belief that
slow-and-low attacks are continually occurring and are going largely
unnoticed...i'll speak for myself)

Few years ago we began seeing certain ports used as attack vectors, thus we
began our internet boundary policers for these ports... as time went on, we
add to that list of ports.  Some ports as we know, like dns, and I think ntp
from time to time (dang, sorry, lol) are used in amplification, and so, we
can't police legit ports too slowly or real stuff is affected... so that's
what Roland probably meant by "judiciously"

We also have inside this set of qos tools at the internet boundary, an
ever-growing acl that we call "repeat victims"...  we have grown to
understand that, if a customer ip address is attack once, it's likely it
will be attacked again...

There are new attacked ports all the time, so sometimes, an attack gets
through... which is causing me to think about an overall UDP limit on my
internet boundary ports... since most attacks are udp-based*furthermore,
along with that overarching udp limit, I may mark internet-sourced-udp with
a certain marking dscp/exp so that as it travels through my internet
network, it will be the first to get dropped (? Wred ? work well for udp?)
during congestion when an attack gets through

-Aaron

* btw, what can you experts tell me about tcp-based volumetric attacks...
please help me to understand... does tcp have an inherent inability to
ramp-up to massive speeds/loads with it's sliding window and
must-rcv-ack-before sending more segments ??  I ask since I heard this years
ago about tcp and I wonder if this is why 




-Original Message-
From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Roland Dobbins
Sent: Friday, August 31, 2018 12:13 PM
To: NANOG list
Subject: Re: automatic rtbh trigger using flow data


On 31 Aug 2018, at 23:53, Lotia, Pratik M wrote:

> Instead of rtbh I would suggest blocking/rate limiting common ports 
> used in DDoS attacks.

This isn't an 'instead of', it's an 'in addition to'.  And it must be 
done judiciously; many operators doing this have concentrated on common 
port-pairs observed in UDP reflection/amplification attacks.

It's important to understand that any kind of packet of any 
protocol/ports (if such concepts apply on the protocol in question) can 
be used to launch DDoS attacks.

We've many tools in the toolbox, and should use them in a 
situationally-appropriate manner.  And when we're using techniques like 
QoSing down certain ports/protocols, we must err on the side of caution, 
lest we cause larger problems than the attacks themselves.

---
Roland Dobbins 

RE: automatic rtbh trigger using flow data

[Lotia, Pratik M]

>many operators doing this have concentrated on common 
>port-pairs observed in UDP reflection/amplification attacks.

Yes, because that's a great starting point.

> And when we're using techniques like 
>QoSing down certain ports/protocols, we must err on the side of caution,

Arbor report mentions volumetric attacks using DNS, NTP form 75+% of the 
attacks. Then QoSing certain ports and protocols is the best way to start with.

~Pratik Lotia  



-Original Message-
From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Roland Dobbins
Sent: Friday, August 31, 2018 11:13 AM
To: NANOG list
Subject: Re: automatic rtbh trigger using flow data


On 31 Aug 2018, at 23:53, Lotia, Pratik M wrote:

> Instead of rtbh I would suggest blocking/rate limiting common ports 
> used in DDoS attacks.

This isn't an 'instead of', it's an 'in addition to'.  And it must be 
done judiciously; many operators doing this have concentrated on common 
port-pairs observed in UDP reflection/amplification attacks.

It's important to understand that any kind of packet of any 
protocol/ports (if such concepts apply on the protocol in question) can 
be used to launch DDoS attacks.

We've many tools in the toolbox, and should use them in a 
situationally-appropriate manner.  And when we're using techniques like 
QoSing down certain ports/protocols, we must err on the side of caution, 
lest we cause larger problems than the attacks themselves.

---
Roland Dobbins 
E-MAIL CONFIDENTIALITY NOTICE: 
The contents of this e-mail message and any attachments are intended solely for 
the addressee(s) and may contain confidential and/or legally privileged 
information. If you are not the intended recipient of this message or if this 
message has been addressed to you in error, please immediately alert the sender 
by reply e-mail and then delete this message and any attachments. If you are 
not the intended recipient, you are notified that any use, dissemination, 
distribution, copying, or storage of this message or any attachment is strictly 
prohibited.

RE: automatic rtbh trigger using flow data

[Michel Py]

> Ryan Hamel wrote :
> I also want to make clear to Michel, that colo'ing a router at an ISP is no 
> different than plugging it into your local router, your uplink  will get 
> saturated beyond what it can physically handle with only the ACLs protecting 
> the other side, but if your clients are also receiving traffic on the same 
> uplink as the attack, it's a denial of service to them.

I understand, but there still is a difference : Bandwith inside the colo is 
cheaper than the WAN link; conceptually you could have multiple fat interfaces 
at the colo and hope to filter the unwanted traffic before it hits where it 
generally hurts : on the WAN link. Would save money only if the transit is 
billed separately from the WAN link traffic, and every situation is different.

Michel.

-Original Message-
From: NANOG  On Behalf Of H I Baysal
Sent: Friday, August 31, 2018 2:09 AM
To: Michel Py ; Aaron Gould ; 
mic...@arneill-py.sacramento.ca.us
Cc: Nanog@nanog.org
Subject: Re: automatic rtbh trigger using flow data

Most of the solutions mentioned are paid, or fastnetmon is partially paid. And 
the thing you want is paid i believe
Nice tool though, not saying anything against it. However

My personal view is, as long as you can store your flow info in a timeseries 
database (like influxdb and NOT SQL LIKE!!!) you can do whatever you want 
with the (raw) data. And create custom triggers for different calculations.

Flows are on the fly and are coming in constantly, you could have a calculation 
like group by srcip and whatever protocol you want or just srcip, and make a 
calculation for every x seconds or minutes. As i mentioned the flow data is a 
constant stream, so you could have it triggered as fast as you want.

(and the nice thing is, with sflow, you also get as path, peer as, 
localpref,community (if enabled). You could group by anything.. :)

I admit it takes a bit more time to setup but the outcome is amazing ;) 
(especially if you graph it then with grafana) And in your case it would be a 
script that does a influxdb command to make the calculations and if the outcome 
shows an IP meeting the thresholds you have set in the calculation, you trigger 
a script that adjusts the route to be announced to your upstream with the 
correct
(rtbh) community.
( as i mentioned, as long as you have the "raw" flows, you can do anything )


Good luck, whatever you choose :)


On 31-08-18 02:14, Michel Py wrote:
>> Aaron Gould wrote :
>> I'm really surprised that you all are doing this based on source ip, 
>> simply because I thought the distribution of botnet members around the world 
>> we're so extensive that I never really thought it possible to filter based 
>> on sources, if so I'd like to see the list too.
> I emailed you. For years I ran it at home on a Cisco 1841, 100,000 BGP 
> prefixes is nothing these days. I am not surprised that Joe pushes that to 
> some CPEs.
>
>> Even so, this would not stop the attacks from hitting my front door, 
>> my side of my Internet uplink...when paying for a 30 gigs CIR and 
>> paying double for megabits per second over that, up to the ceiling of 100 
>> gig every bit that hits my front door over 30 gig would cost me extra, 
>> remotely triggering based on my victim IP address inside my network would be 
>> my solution to saving money.
> I agree. If you want to get a real use of source blacklisting, to save 
> bandwidth, you probably went to rent a U in a rack at your upstream(s) to 
> block it there.
> I never did it past 1GE, and I have never measured seriously the bandwidth it 
> would save, would be curious to know.
> I think the two approaches are complementary to each other though.
>
> Michel.
>
>
> On Aug 30, 2018, at 6:43 PM, Michel Py  wrote:
>
>>> Joe Maimon wrote :
>>> I use a bunch of scripts plus a supervisory sqlite3 database process 
>>> all injecting into quagga
>> I have the sqlite part planned, today I'm using a flat file :-( I 
>> know :-(
>>
>>> Also aimed at attacker sources. I feed it with honeypots and live servers, 
>>> hooked into fail2ban and using independent host scripts. Not very 
>>> sophisticated, the remotes use ssh executed commands to add/delete. I also 
>>> setup a promiscuous ebgp RR so I can extend my umbrella to CPE with diverse 
>>> connectivity.
>> I would like to have your feed. How many attacker prefixes do you currently 
>> have ?
>>
>>> Using flow data, that sounds like an interesting direction to take this 
>>> into, so thank you!
>> The one thing we can share here is the attacker prefixes. The victim 
>> prefixes are unique to each of us but I expect our attacker prefixes to be 
>> very close.
>>
>> Michel.
>>
&g

Re: automatic rtbh trigger using flow data

[Roland Dobbins]

On 31 Aug 2018, at 23:53, Lotia, Pratik M wrote:

Instead of rtbh I would suggest blocking/rate limiting common ports 
used in DDoS attacks.


This isn't an 'instead of', it's an 'in addition to'.  And it must be 
done judiciously; many operators doing this have concentrated on common 
port-pairs observed in UDP reflection/amplification attacks.


It's important to understand that any kind of packet of any 
protocol/ports (if such concepts apply on the protocol in question) can 
be used to launch DDoS attacks.


We've many tools in the toolbox, and should use them in a 
situationally-appropriate manner.  And when we're using techniques like 
QoSing down certain ports/protocols, we must err on the side of caution, 
lest we cause larger problems than the attacks themselves.


---
Roland Dobbins 

RE: automatic rtbh trigger using flow data

[Lotia, Pratik M]

Instead of rtbh I would suggest blocking/rate limiting common ports used in 
DDoS attacks. That will block 90% of the DDoS attacks. We recently open sourced 
a BGP Flowspec based tool for DDoS Mitigation. It applies Flowspec rules per 
victim IP Addr.
https://github.com/racompton/docker-auto-flowspec


~Pratik Lotia 


-Original Message-
From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of H I Baysal
Sent: Friday, August 31, 2018 3:09 AM
To: Michel Py; Aaron Gould; mic...@arneill-py.sacramento.ca.us
Cc: Nanog@nanog.org
Subject: Re: automatic rtbh trigger using flow data

Most of the solutions mentioned are paid, or fastnetmon is partially 
paid. And the thing you want is paid i believe
Nice tool though, not saying anything against it. However

My personal view is, as long as you can store your flow info in a 
timeseries database (like influxdb and NOT SQL LIKE!!!) you can do 
whatever you want with the (raw) data. And create custom triggers for 
different calculations.

Flows are on the fly and are coming in constantly, you could have a 
calculation like group by srcip and whatever protocol you want or just 
srcip,
and make a calculation for every x seconds or minutes. As i mentioned 
the flow data is a constant stream, so you could have it triggered as 
fast as you want.

(and the nice thing is, with sflow, you also get as path, peer as, 
localpref,community (if enabled). You could group by anything.. :)

I admit it takes a bit more time to setup but the outcome is amazing ;) 
(especially if you graph it then with grafana)
And in your case it would be a script that does a influxdb command to 
make the calculations and if the outcome shows an IP meeting the 
thresholds you have set in the calculation, you trigger a script that 
adjusts the route to be announced to your upstream with the correct 
(rtbh) community.
( as i mentioned, as long as you have the "raw" flows, you can do anything )


Good luck, whatever you choose :)


On 31-08-18 02:14, Michel Py wrote:
>> Aaron Gould wrote :
>> I'm really surprised that you all are doing this based on source ip, simply 
>> because I thought the distribution of botnet members around
>> the world we're so extensive that I never really thought it possible to 
>> filter based on sources, if so I'd like to see the list too.
> I emailed you. For years I ran it at home on a Cisco 1841, 100,000 BGP 
> prefixes is nothing these days. I am not surprised that Joe pushes that to 
> some CPEs.
>
>> Even so, this would not stop the attacks from hitting my front door, my side 
>> of my Internet uplink...when paying for a 30 gigs CIR
>> and paying double for megabits per second over that, up to the ceiling of 
>> 100 gig every bit that hits my front door over 30 gig
>> would cost me extra, remotely triggering based on my victim IP address 
>> inside my network would be my solution to saving money.
> I agree. If you want to get a real use of source blacklisting, to save 
> bandwidth, you probably went to rent a U in a rack at your upstream(s) to 
> block it there.
> I never did it past 1GE, and I have never measured seriously the bandwidth it 
> would save, would be curious to know.
> I think the two approaches are complementary to each other though.
>
> Michel.
>
>
> On Aug 30, 2018, at 6:43 PM, Michel Py  wrote:
>
>>> Joe Maimon wrote :
>>> I use a bunch of scripts plus a supervisory sqlite3 database process all 
>>> injecting into quagga
>> I have the sqlite part planned, today I'm using a flat file :-( I know :-(
>>
>>> Also aimed at attacker sources. I feed it with honeypots and live servers, 
>>> hooked into fail2ban and using independent host scripts. Not very 
>>> sophisticated, the remotes use ssh executed commands to add/delete. I also 
>>> setup a promiscuous ebgp RR so I can extend my umbrella to CPE with diverse 
>>> connectivity.
>> I would like to have your feed. How many attacker prefixes do you currently 
>> have ?
>>
>>> Using flow data, that sounds like an interesting direction to take this 
>>> into, so thank you!
>> The one thing we can share here is the attacker prefixes. The victim 
>> prefixes are unique to each of us but I expect our attacker prefixes to be 
>> very close.
>>
>> Michel.
>>
>> TSI Disclaimer:  This message and any files or text attached to it are 
>> intended only for the recipients named above and contain information that 
>> may be confidential or privileged. If you are not the intended recipient, 
>> you must not forward, copy, use or otherwise disclose this communication or 
>> the information contained herein. In the event you have received this 
>> message in error, please notify the sender 

Re: automatic rtbh trigger using flow data

[Roland Dobbins]

On 31 Aug 2018, at 22:15, Hugo Slabbert wrote:

I would love an upstream that accepts flowspec routes to get granular 
about drops and to basically push "stateless ACLs" upstream.





---
Roland Dobbins 

Re: automatic rtbh trigger using flow data

[Roland Dobbins]

On 31 Aug 2018, at 16:33, Ryan Hamel wrote:

From experience, sflows are horribly inaccurate for DDoS detection, 
since the volume could disrupt the control plane and render the 
process useless, thus not giving data to the external system to act 
upon it.


On the contrary, flow telemetry in general works quite well for DDoS 
detection/classification/traceback, and is widely utilized for such 
purposes; it has been for many years.


I'm not a big fan of s/Flow comparatively speaking, but it and NetFlow, 
IPFIX, et. al. have proven themselves over the years, assuming that the 
flow export parameters on the exporting devices are configured 
correctly, and the collection/analysis systems are configured optimally.


Flow telemetry is management-plane, not control-plane.  Implementing 
network infrastructure self-protection BCPs such as iACLs is definitely 
recommended in general.



---
Roland Dobbins 

Re: automatic rtbh trigger using flow data

[Hugo Slabbert]

On Fri 2018-Aug-31 06:59:29 +0700, Roland Dobbins  wrote:


On 31 Aug 2018, at 6:47, Aaron Gould wrote:

I'm really surprised that you all are doing this based on source 
ip, simply because I thought the distribution of botnet members 
around the world we're so extensive that I never really thought it 
possible to filter based on sources, i


Using S/RTBH to drop attack sources has been a valid and useful 
mitigation tactic for close to 20 years.  Any kind of modern router 
scales up to large numbers of sources; and note that S/RTBH isn't 
limited to /32s.


It's discussed in this .pdf preso:




I would love an upstream that accepts flowspec routes to get granular about 
drops and to basically push "stateless ACLs" upstream.


_keeps dreaming_

--
Hugo Slabbert   | email, xmpp/jabber: h...@slabnet.com
pgp key: B178313E   | also on Signal



signature.asc
Description: Digital signature

Re: automatic rtbh trigger using flow data

[H I Baysal]

I think your experience has to do more with your setup than sflow or 
influxdb...


my sflow data, that i push into influxdb. is 1 on 1 accurate with the 
interface utilization ( even on group by per source ip )

Arista performs fine with sflow Don't know what brand you used.
I'm getting 300 mbit of constant (sflow) data in influxdb


'''
it simply could not keep up, nor could I write any useful queries against it
'''

again, i really think this has to do with the setup or the queries you 
tried to do. (and (i think ) you tried sflow exporter on a server, i 
think the topic was from a appliance as an exporter)


If you still want to give influxdb+ sflow a try, i'm more than willing 
to help you out ;)



On 31-08-18 11:33, Ryan Hamel wrote:

 From experience, sflows are horribly inaccurate for DDoS detection, since the 
volume could disrupt the control plane and render the process useless, thus not 
giving data to the external system to act upon it. You can't get any better 
than mirroring your inbound transit, and sampling the output to a sensor.

I have also spent some time on a spare server running netmap and influxdb, it 
simply could not keep up, nor could I write any useful queries against it. 
Which is why I'm deciding on using pf_ring ft (flow table) and just let it 
calculate all the data to be dumped into a MySQL memory table, and also 
harvesting ntop's NDPI protocol decoding, to enhance the detection rules. My 
ideal setup is to break things down into dedicated programs like flow exporter, 
detection, and response.

I also want to make clear to Michel, that colo'ing a router at an ISP is no 
different than plugging it into your local router, your uplink will get 
saturated beyond what it can physically handle with only the ACLs protecting 
the other side, but if your clients are also receiving traffic on the same 
uplink as the attack, it's a denial of service to them.

-Original Message-
From: NANOG  On Behalf Of H I Baysal
Sent: Friday, August 31, 2018 2:09 AM
To: Michel Py ; Aaron Gould ; 
mic...@arneill-py.sacramento.ca.us
Cc: Nanog@nanog.org
Subject: Re: automatic rtbh trigger using flow data

Most of the solutions mentioned are paid, or fastnetmon is partially paid. And 
the thing you want is paid i believe
Nice tool though, not saying anything against it. However

My personal view is, as long as you can store your flow info in a timeseries 
database (like influxdb and NOT SQL LIKE!!!) you can do whatever you want 
with the (raw) data. And create custom triggers for different calculations.

Flows are on the fly and are coming in constantly, you could have a calculation 
like group by srcip and whatever protocol you want or just srcip, and make a 
calculation for every x seconds or minutes. As i mentioned the flow data is a 
constant stream, so you could have it triggered as fast as you want.

(and the nice thing is, with sflow, you also get as path, peer as, 
localpref,community (if enabled). You could group by anything.. :)

I admit it takes a bit more time to setup but the outcome is amazing ;) 
(especially if you graph it then with grafana) And in your case it would be a 
script that does a influxdb command to make the calculations and if the outcome 
shows an IP meeting the thresholds you have set in the calculation, you trigger 
a script that adjusts the route to be announced to your upstream with the 
correct
(rtbh) community.
( as i mentioned, as long as you have the "raw" flows, you can do anything )


Good luck, whatever you choose :)


On 31-08-18 02:14, Michel Py wrote:

Aaron Gould wrote :
I'm really surprised that you all are doing this based on source ip,
simply because I thought the distribution of botnet members around the world 
we're so extensive that I never really thought it possible to filter based on 
sources, if so I'd like to see the list too.

I emailed you. For years I ran it at home on a Cisco 1841, 100,000 BGP prefixes 
is nothing these days. I am not surprised that Joe pushes that to some CPEs.


Even so, this would not stop the attacks from hitting my front door,
my side of my Internet uplink...when paying for a 30 gigs CIR and
paying double for megabits per second over that, up to the ceiling of 100 gig 
every bit that hits my front door over 30 gig would cost me extra, remotely 
triggering based on my victim IP address inside my network would be my solution 
to saving money.

I agree. If you want to get a real use of source blacklisting, to save 
bandwidth, you probably went to rent a U in a rack at your upstream(s) to block 
it there.
I never did it past 1GE, and I have never measured seriously the bandwidth it 
would save, would be curious to know.
I think the two approaches are complementary to each other though.

Michel.


On Aug 30, 2018, at 6:43 PM, Michel Py  wrote:


Joe Maimon wrote :
I use a bunch of scripts plus a supervisory sqlite3 database process
all injecting into quagga

I have the sqlite part planned,

RE: automatic rtbh trigger using flow data

[Ryan Hamel]

From experience, sflows are horribly inaccurate for DDoS detection, since the 
volume could disrupt the control plane and render the process useless, thus not 
giving data to the external system to act upon it. You can't get any better 
than mirroring your inbound transit, and sampling the output to a sensor.

I have also spent some time on a spare server running netmap and influxdb, it 
simply could not keep up, nor could I write any useful queries against it. 
Which is why I'm deciding on using pf_ring ft (flow table) and just let it 
calculate all the data to be dumped into a MySQL memory table, and also 
harvesting ntop's NDPI protocol decoding, to enhance the detection rules. My 
ideal setup is to break things down into dedicated programs like flow exporter, 
detection, and response.

I also want to make clear to Michel, that colo'ing a router at an ISP is no 
different than plugging it into your local router, your uplink will get 
saturated beyond what it can physically handle with only the ACLs protecting 
the other side, but if your clients are also receiving traffic on the same 
uplink as the attack, it's a denial of service to them.

-Original Message-
From: NANOG  On Behalf Of H I Baysal
Sent: Friday, August 31, 2018 2:09 AM
To: Michel Py ; Aaron Gould ; 
mic...@arneill-py.sacramento.ca.us
Cc: Nanog@nanog.org
Subject: Re: automatic rtbh trigger using flow data

Most of the solutions mentioned are paid, or fastnetmon is partially paid. And 
the thing you want is paid i believe
Nice tool though, not saying anything against it. However

My personal view is, as long as you can store your flow info in a timeseries 
database (like influxdb and NOT SQL LIKE!!!) you can do whatever you want 
with the (raw) data. And create custom triggers for different calculations.

Flows are on the fly and are coming in constantly, you could have a calculation 
like group by srcip and whatever protocol you want or just srcip, and make a 
calculation for every x seconds or minutes. As i mentioned the flow data is a 
constant stream, so you could have it triggered as fast as you want.

(and the nice thing is, with sflow, you also get as path, peer as, 
localpref,community (if enabled). You could group by anything.. :)

I admit it takes a bit more time to setup but the outcome is amazing ;) 
(especially if you graph it then with grafana) And in your case it would be a 
script that does a influxdb command to make the calculations and if the outcome 
shows an IP meeting the thresholds you have set in the calculation, you trigger 
a script that adjusts the route to be announced to your upstream with the 
correct
(rtbh) community.
( as i mentioned, as long as you have the "raw" flows, you can do anything )


Good luck, whatever you choose :)


On 31-08-18 02:14, Michel Py wrote:
>> Aaron Gould wrote :
>> I'm really surprised that you all are doing this based on source ip, 
>> simply because I thought the distribution of botnet members around the world 
>> we're so extensive that I never really thought it possible to filter based 
>> on sources, if so I'd like to see the list too.
> I emailed you. For years I ran it at home on a Cisco 1841, 100,000 BGP 
> prefixes is nothing these days. I am not surprised that Joe pushes that to 
> some CPEs.
>
>> Even so, this would not stop the attacks from hitting my front door, 
>> my side of my Internet uplink...when paying for a 30 gigs CIR and 
>> paying double for megabits per second over that, up to the ceiling of 100 
>> gig every bit that hits my front door over 30 gig would cost me extra, 
>> remotely triggering based on my victim IP address inside my network would be 
>> my solution to saving money.
> I agree. If you want to get a real use of source blacklisting, to save 
> bandwidth, you probably went to rent a U in a rack at your upstream(s) to 
> block it there.
> I never did it past 1GE, and I have never measured seriously the bandwidth it 
> would save, would be curious to know.
> I think the two approaches are complementary to each other though.
>
> Michel.
>
>
> On Aug 30, 2018, at 6:43 PM, Michel Py  wrote:
>
>>> Joe Maimon wrote :
>>> I use a bunch of scripts plus a supervisory sqlite3 database process 
>>> all injecting into quagga
>> I have the sqlite part planned, today I'm using a flat file :-( I 
>> know :-(
>>
>>> Also aimed at attacker sources. I feed it with honeypots and live servers, 
>>> hooked into fail2ban and using independent host scripts. Not very 
>>> sophisticated, the remotes use ssh executed commands to add/delete. I also 
>>> setup a promiscuous ebgp RR so I can extend my umbrella to CPE with diverse 
>>> connectivity.
>> I would like to have your feed. How many attacker prefixes do you currently 
>> have ?

Re: automatic rtbh trigger using flow data

[H I Baysal]

Most of the solutions mentioned are paid, or fastnetmon is partially 
paid. And the thing you want is paid i believe

Nice tool though, not saying anything against it. However

My personal view is, as long as you can store your flow info in a 
timeseries database (like influxdb and NOT SQL LIKE!!!) you can do 
whatever you want with the (raw) data. And create custom triggers for 
different calculations.


Flows are on the fly and are coming in constantly, you could have a 
calculation like group by srcip and whatever protocol you want or just 
srcip,
and make a calculation for every x seconds or minutes. As i mentioned 
the flow data is a constant stream, so you could have it triggered as 
fast as you want.


(and the nice thing is, with sflow, you also get as path, peer as, 
localpref,community (if enabled). You could group by anything.. :)


I admit it takes a bit more time to setup but the outcome is amazing ;) 
(especially if you graph it then with grafana)
And in your case it would be a script that does a influxdb command to 
make the calculations and if the outcome shows an IP meeting the 
thresholds you have set in the calculation, you trigger a script that 
adjusts the route to be announced to your upstream with the correct 
(rtbh) community.

( as i mentioned, as long as you have the "raw" flows, you can do anything )


Good luck, whatever you choose :)


On 31-08-18 02:14, Michel Py wrote:

Aaron Gould wrote :
I'm really surprised that you all are doing this based on source ip, simply 
because I thought the distribution of botnet members around
the world we're so extensive that I never really thought it possible to filter 
based on sources, if so I'd like to see the list too.

I emailed you. For years I ran it at home on a Cisco 1841, 100,000 BGP prefixes 
is nothing these days. I am not surprised that Joe pushes that to some CPEs.


Even so, this would not stop the attacks from hitting my front door, my side of 
my Internet uplink...when paying for a 30 gigs CIR
and paying double for megabits per second over that, up to the ceiling of 100 
gig every bit that hits my front door over 30 gig
would cost me extra, remotely triggering based on my victim IP address inside 
my network would be my solution to saving money.

I agree. If you want to get a real use of source blacklisting, to save 
bandwidth, you probably went to rent a U in a rack at your upstream(s) to block 
it there.
I never did it past 1GE, and I have never measured seriously the bandwidth it 
would save, would be curious to know.
I think the two approaches are complementary to each other though.

Michel.


On Aug 30, 2018, at 6:43 PM, Michel Py  wrote:


Joe Maimon wrote :
I use a bunch of scripts plus a supervisory sqlite3 database process all 
injecting into quagga

I have the sqlite part planned, today I'm using a flat file :-( I know :-(


Also aimed at attacker sources. I feed it with honeypots and live servers, 
hooked into fail2ban and using independent host scripts. Not very 
sophisticated, the remotes use ssh executed commands to add/delete. I also 
setup a promiscuous ebgp RR so I can extend my umbrella to CPE with diverse 
connectivity.

I would like to have your feed. How many attacker prefixes do you currently 
have ?


Using flow data, that sounds like an interesting direction to take this into, 
so thank you!

The one thing we can share here is the attacker prefixes. The victim prefixes 
are unique to each of us but I expect our attacker prefixes to be very close.

Michel.

TSI Disclaimer:  This message and any files or text attached to it are intended 
only for the recipients named above and contain information that may be 
confidential or privileged. If you are not the intended recipient, you must not 
forward, copy, use or otherwise disclose this communication or the information 
contained herein. In the event you have received this message in error, please 
notify the sender immediately by replying to this message, and then delete all 
copies of it from your system. Thank you!...

RE: automatic rtbh trigger using flow data

[Michel Py]

> Aaron Gould wrote :
> I'm really surprised that you all are doing this based on source ip, simply 
> because I thought the distribution of botnet members around
> the world we're so extensive that I never really thought it possible to 
> filter based on sources, if so I'd like to see the list too.

I emailed you. For years I ran it at home on a Cisco 1841, 100,000 BGP prefixes 
is nothing these days. I am not surprised that Joe pushes that to some CPEs.

> Even so, this would not stop the attacks from hitting my front door, my side 
> of my Internet uplink...when paying for a 30 gigs CIR
> and paying double for megabits per second over that, up to the ceiling of 100 
> gig every bit that hits my front door over 30 gig
> would cost me extra, remotely triggering based on my victim IP address inside 
> my network would be my solution to saving money.

I agree. If you want to get a real use of source blacklisting, to save 
bandwidth, you probably went to rent a U in a rack at your upstream(s) to block 
it there.
I never did it past 1GE, and I have never measured seriously the bandwidth it 
would save, would be curious to know.
I think the two approaches are complementary to each other though.

Michel.


On Aug 30, 2018, at 6:43 PM, Michel Py  wrote:

>> Joe Maimon wrote :
>> I use a bunch of scripts plus a supervisory sqlite3 database process all 
>> injecting into quagga
> 
> I have the sqlite part planned, today I'm using a flat file :-( I know :-(
> 
>> Also aimed at attacker sources. I feed it with honeypots and live servers, 
>> hooked into fail2ban and using independent host scripts. Not very 
>> sophisticated, the remotes use ssh executed commands to add/delete. I also 
>> setup a promiscuous ebgp RR so I can extend my umbrella to CPE with diverse 
>> connectivity.
> 
> I would like to have your feed. How many attacker prefixes do you currently 
> have ?
> 
>> Using flow data, that sounds like an interesting direction to take this 
>> into, so thank you!
> 
> The one thing we can share here is the attacker prefixes. The victim prefixes 
> are unique to each of us but I expect our attacker prefixes to be very close.
> 
> Michel.
> 
> TSI Disclaimer:  This message and any files or text attached to it are 
> intended only for the recipients named above and contain information that may 
> be confidential or privileged. If you are not the intended recipient, you 
> must not forward, copy, use or otherwise disclose this communication or the 
> information contained herein. In the event you have received this message in 
> error, please notify the sender immediately by replying to this message, and 
> then delete all copies of it from your system. Thank you!...

Re: automatic rtbh trigger using flow data

[Roland Dobbins]

On 31 Aug 2018, at 6:47, Aaron Gould wrote:

I'm really surprised that you all are doing this based on source ip, 
simply because I thought the distribution of botnet members around the 
world we're so extensive that I never really thought it possible to 
filter based on sources, i


Using S/RTBH to drop attack sources has been a valid and useful 
mitigation tactic for close to 20 years.  Any kind of modern router 
scales up to large numbers of sources; and note that S/RTBH isn't 
limited to /32s.


It's discussed in this .pdf preso:



---
Roland Dobbins 

Re: automatic rtbh trigger using flow data

[Aaron Gould]

I'm really surprised that you all are doing this based on source ip, simply 
because I thought the distribution of botnet members around the world we're so 
extensive that I never really thought it possible to filter based on sources, 
if so I'd like to see the list too

Even so, this would not stop the attacks from hitting my front door, my side of 
my Internet uplink...when paying for a 30 gigs CIR and paying double for 
megabits per second over that, up to the ceiling of 100 gig every bit that hits 
my front door over 30 gig would cost me extra, remotely triggering based on my 
victim IP address inside my network would be my solution to saving money

But stopping the attack even on my side of my Internet up like would at least 
stop it from proliferating throughout my internal network which is also costing 
me when it affects cell towers, etc.

Aaron

On Aug 30, 2018, at 6:43 PM, Michel Py  wrote:

>> Joe Maimon wrote :
>> I use a bunch of scripts plus a supervisory sqlite3 database process all 
>> injecting into quagga
> 
> I have the sqlite part planned, today I'm using a flat file :-( I know :-(
> 
>> Also aimed at attacker sources. I feed it with honeypots and live servers, 
>> hooked into fail2ban and using independent host scripts. Not very 
>> sophisticated, the remotes use ssh executed commands to add/delete. I also 
>> setup a promiscuous ebgp RR so I can extend my umbrella to CPE with diverse 
>> connectivity.
> 
> I would like to have your feed. How many attacker prefixes do you currently 
> have ?
> 
>> Using flow data, that sounds like an interesting direction to take this 
>> into, so thank you!
> 
> The one thing we can share here is the attacker prefixes. The victim prefixes 
> are unique to each of us but I expect our attacker prefixes to be very close.
> 
> Michel.
> 
> TSI Disclaimer:  This message and any files or text attached to it are 
> intended only for the recipients named above and contain information that may 
> be confidential or privileged. If you are not the intended recipient, you 
> must not forward, copy, use or otherwise disclose this communication or the 
> information contained herein. In the event you have received this message in 
> error, please notify the sender immediately by replying to this message, and 
> then delete all copies of it from your system. Thank you!...

RE: automatic rtbh trigger using flow data

[Michel Py]

> Joe Maimon wrote :
> I use a bunch of scripts plus a supervisory sqlite3 database process all 
> injecting into quagga

I have the sqlite part planned, today I'm using a flat file :-( I know :-(

> Also aimed at attacker sources. I feed it with honeypots and live servers, 
> hooked into fail2ban and using independent host scripts. Not very 
> sophisticated, the remotes use ssh executed commands to add/delete. I also 
> setup a promiscuous ebgp RR so I can extend my umbrella to CPE with diverse 
> connectivity.

I would like to have your feed. How many attacker prefixes do you currently 
have ?

> Using flow data, that sounds like an interesting direction to take this into, 
> so thank you!

The one thing we can share here is the attacker prefixes. The victim prefixes 
are unique to each of us but I expect our attacker prefixes to be very close.

Michel.

TSI Disclaimer:  This message and any files or text attached to it are intended 
only for the recipients named above and contain information that may be 
confidential or privileged. If you are not the intended recipient, you must not 
forward, copy, use or otherwise disclose this communication or the information 
contained herein. In the event you have received this message in error, please 
notify the sender immediately by replying to this message, and then delete all 
copies of it from your system. Thank you!...

Re: automatic rtbh trigger using flow data

[Joe Maimon]

Michel Py wrote:

Aaron Gould wrote :
Hi, does anyone know how to use flow data to trigger a rtbh (remotely triggered 
blackhole) route using bgp ?  ...I'm thinking we could use
quagga or a script of some sort to interact with a router to advertise to bgp 
the /32 host route of the victim under attack.

Look at Exabgp : https://github.com/Exa-Networks/exabgp
That's what I use in here : https://arneill-py.sacramento.ca.us/cbbc/ to inject 
the prefixes in BGP.
I block the attacker's addresses, not the victim but if you are willing to 
write your own scripts it does the job.

Michel.



I use a bunch of scripts plus a supervisory sqlite3 database process all 
injecting into quagga


Also aimed at attacker sources. I feed it with honeypots and live 
servers, hooked into fail2ban and using independent host scripts.


Not very sophisticated, the remotes use ssh executed commands to 
add/delete. I also setup a promiscuous ebgp RR so I can extend my 
umbrella to CPE with diverse connectivity.


Using flow data, that sounds like an interesting direction to take this 
into, so thank you!


Joe

RE: automatic rtbh trigger using flow data

[Michel Py]

> Aaron Gould wrote :
> Thanks, but what if the attacker is many... like thousands ?  ...isn't that 
> typically what we see, is tons and tons of sources (hence distributeddos) 
> ?

At this very moment I blacklist ~ 56,000 individual /32s and historically it 
has been up to 135,000 at times. It's not a problem for most routers, unless 
you're on one of these old clunkers with un-upgradable TCAM and a full feed (if 
you are, you don't have much time left anyway).

> Ryan Hamel wrote :
> Exactly Aaron. No provider will allow a customer to null route a source IP 
> address.

Yes, unless you have your own router on their side of the link and pay for it, 
or have your own VRF on their router which is not going to be cheap either.

> I could only assume that a null route on Michel's network is tanking the 
> packets at their edge to 192.0.2.1 (discard/null0).

Correct, and I clearly understand its limitations, paragraph below taken from 
https://arneill-py.sacramento.ca.us/cbbc/
There indeed is a value in blacklisting the IP address of the host being 
attacked and feed that with the appropriate community to the upstream that will 
accept it as it is part of your own space. You sacrifice one host to save the 
bandwidth to the rest.
That being said, if the DDOS targets your entire IP range, none of these will 
help.

I have to withstand DDOS attacks all the time, can the CBBC feed help ?
It depends on the type of attack; the CBBC feed is not designed as DDOS 
mitigation tool. There is no such thing as a free lunch : your ISP will not 
take the full CBBC feed for free when they can make you pay big bucks for their 
own one. The CBBC does not prevent the DDOS attack to get to you, it may help 
with attacks that are based on PPS, not raw bandwidth. What the CBBC does is to 
block the offending traffic at the router level, so it is blocked before it 
even reaches your server / firewall. However, the CBBC does not prevent the 
DDOS traffic from coming to you, so if you have a slow connection to the 
Internet and the DDOS sends more bandwidth than you have, you still are down. 
However, if the DDOS is based not on bandwidth but on a higher-level protocol 
such as DNS or HTTPS, it helps by taking the load off the server.

Michel.

-Aaron

-Original Message-
From: Michel Py [mailto:michel...@tsisemi.com] 
Sent: Thursday, August 30, 2018 3:17 PM
To: Aaron Gould; Nanog@nanog.org
Subject: RE: automatic rtbh trigger using flow data 

> Aaron Gould wrote :
> Hi, does anyone know how to use flow data to trigger a rtbh (remotely
triggered blackhole) route using bgp ?  ...I'm thinking we could use
> quagga or a script of some sort to interact with a router to advertise to
bgp the /32 host route of the victim under attack.

Look at Exabgp : https://github.com/Exa-Networks/exabgp
That's what I use in here : https://arneill-py.sacramento.ca.us/cbbc/ to
inject the prefixes in BGP.
I block the attacker's addresses, not the victim but if you are willing to
write your own scripts it does the job.

Michel.

TSI Disclaimer:  This message and any files or text attached to it are
intended only for the recipients named above and contain information that
may be confidential or privileged. If you are not the intended recipient,
you must not forward, copy, use or otherwise disclose this communication or
the information contained herein. In the event you have received this
message in error, please notify the sender immediately by replying to this
message, and then delete all copies of it from your system. Thank you!...

RE: automatic rtbh trigger using flow data

[Ryan Hamel]

Exactly Aaron. No provider will allow a customer to null route a source IP 
address. I could only assume that a null route on Michel's network is tanking 
the packets at their edge to 192.0.2.1 (discard/null0).

-- 
Ryan Hamel
Senior Support Engineer
ryan.ha...@quadranet.com | +1 (888) 578-2372
QuadraNet Enterprises, LLC. | Dedicated Servers, Colocation, Cloud

-Original Message-
From: NANOG  On Behalf Of Aaron Gould
Sent: Thursday, August 30, 2018 1:38 PM
To: 'Michel Py' ; Nanog@nanog.org
Subject: RE: automatic rtbh trigger using flow data

Thanks, but what if the attacker is many... like thousands ?  ...isn't that 
typically what we see, is tons and tons of sources (hence
distributeddos) ?

-Aaron

-Original Message-
From: Michel Py [mailto:michel...@tsisemi.com]
Sent: Thursday, August 30, 2018 3:17 PM
To: Aaron Gould; Nanog@nanog.org
Subject: RE: automatic rtbh trigger using flow data 

> Aaron Gould wrote :
> Hi, does anyone know how to use flow data to trigger a rtbh (remotely
triggered blackhole) route using bgp ?  ...I'm thinking we could use
> quagga or a script of some sort to interact with a router to advertise 
> to
bgp the /32 host route of the victim under attack.

Look at Exabgp : https://github.com/Exa-Networks/exabgp
That's what I use in here : https://arneill-py.sacramento.ca.us/cbbc/ to inject 
the prefixes in BGP.
I block the attacker's addresses, not the victim but if you are willing to 
write your own scripts it does the job.

Michel.

TSI Disclaimer:  This message and any files or text attached to it are intended 
only for the recipients named above and contain information that may be 
confidential or privileged. If you are not the intended recipient, you must not 
forward, copy, use or otherwise disclose this communication or the information 
contained herein. In the event you have received this message in error, please 
notify the sender immediately by replying to this message, and then delete all 
copies of it from your system. Thank you!...

RE: automatic rtbh trigger using flow data

[Aaron Gould]

Thanks, but what if the attacker is many... like thousands ?  ...isn't that
typically what we see, is tons and tons of sources (hence
distributeddos) ?

-Aaron

-Original Message-
From: Michel Py [mailto:michel...@tsisemi.com] 
Sent: Thursday, August 30, 2018 3:17 PM
To: Aaron Gould; Nanog@nanog.org
Subject: RE: automatic rtbh trigger using flow data 

> Aaron Gould wrote :
> Hi, does anyone know how to use flow data to trigger a rtbh (remotely
triggered blackhole) route using bgp ?  ...I'm thinking we could use
> quagga or a script of some sort to interact with a router to advertise to
bgp the /32 host route of the victim under attack.

Look at Exabgp : https://github.com/Exa-Networks/exabgp
That's what I use in here : https://arneill-py.sacramento.ca.us/cbbc/ to
inject the prefixes in BGP.
I block the attacker's addresses, not the victim but if you are willing to
write your own scripts it does the job.

Michel.

TSI Disclaimer:  This message and any files or text attached to it are
intended only for the recipients named above and contain information that
may be confidential or privileged. If you are not the intended recipient,
you must not forward, copy, use or otherwise disclose this communication or
the information contained herein. In the event you have received this
message in error, please notify the sender immediately by replying to this
message, and then delete all copies of it from your system. Thank you!...

RE: automatic rtbh trigger using flow data

[Michel Py]

> Aaron Gould wrote :
> Hi, does anyone know how to use flow data to trigger a rtbh (remotely 
> triggered blackhole) route using bgp ?  ...I'm thinking we could use
> quagga or a script of some sort to interact with a router to advertise to bgp 
> the /32 host route of the victim under attack.

Look at Exabgp : https://github.com/Exa-Networks/exabgp
That's what I use in here : https://arneill-py.sacramento.ca.us/cbbc/ to inject 
the prefixes in BGP.
I block the attacker's addresses, not the victim but if you are willing to 
write your own scripts it does the job.

Michel.

TSI Disclaimer:  This message and any files or text attached to it are intended 
only for the recipients named above and contain information that may be 
confidential or privileged. If you are not the intended recipient, you must not 
forward, copy, use or otherwise disclose this communication or the information 
contained herein. In the event you have received this message in error, please 
notify the sender immediately by replying to this message, and then delete all 
copies of it from your system. Thank you!...

RE: automatic rtbh trigger using flow data

[Aaron Gould]

Wow, 4 replies for fastnetmon, thanks Ryan, Vincente, Job and Kushal

 

I'll look into it

 

-Aaron

 

From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Aaron Gould
Sent: Thursday, August 30, 2018 2:53 PM
To: Nanog@nanog.org
Subject: automatic rtbh trigger using flow data 

 

Hi, does anyone know how to use flow data to trigger a rtbh (remotely
triggered blackhole) route using bgp ?  .I'm thinking we could use quagga or
a script of some sort to interact with a router to advertise to bgp the /32
host route of the victim under attack.

 

Btw, I already have nfsen running and we receive real-time alters of various
types of attacks, high volume, high ports, etc. and then we telnet into a
cisco trigger router and drop a few lines of code into it and then bgp does
the rest within seconds, the upstream providers learn of this route via
communities and they rtbh it in their cloud, BUT, I would like my alerts to
do this automatically. that would be very nice.  Any guidance would be
appreciated.

 

-Aaron

 

RE: automatic rtbh trigger using flow data

[Ryan Hamel]

There are software that combine your needs altogether. I'm sure there are 
others.

WANGuard from Andrisoft (https://www.andrisoft.com/software/wanguard)
Fastnetmon (https://fastnetmon.com/)

From: NANOG  On Behalf Of Aaron Gould
Sent: Thursday, August 30, 2018 12:53 PM
To: Nanog@nanog.org
Subject: automatic rtbh trigger using flow data

Hi, does anyone know how to use flow data to trigger a rtbh (remotely triggered 
blackhole) route using bgp ?  ...I'm thinking we could use quagga or a script 
of some sort to interact with a router to advertise to bgp the /32 host route 
of the victim under attack.

Btw, I already have nfsen running and we receive real-time alters of various 
types of attacks, high volume, high ports, etc... and then we telnet into a 
cisco trigger router and drop a few lines of code into it and then bgp does the 
rest within seconds, the upstream providers learn of this route via communities 
and they rtbh it in their cloud, BUT, I would like my alerts to do this 
automatically... that would be very nice.  Any guidance would be appreciated.

-Aaron

Re: automatic rtbh trigger using flow data

[Vicente De Luca]

fastnetmon does exactly what you’re looking for. https://fastnetmon.com/ 

there is also an open source version 
https://github.com/pavel-odintsov/fastnetmon 


my best

—vicente

> On Aug 30, 2018, at 12:52 PM, Aaron Gould  wrote:
> 
> Hi, does anyone know how to use flow data to trigger a rtbh (remotely 
> triggered blackhole) route using bgp ?  …I’m thinking we could use quagga or 
> a script of some sort to interact with a router to advertise to bgp the /32 
> host route of the victim under attack.
>  
> Btw, I already have nfsen running and we receive real-time alters of various 
> types of attacks, high volume, high ports, etc… and then we telnet into a 
> cisco trigger router and drop a few lines of code into it and then bgp does 
> the rest within seconds, the upstream providers learn of this route via 
> communities and they rtbh it in their cloud, BUT, I would like my alerts to 
> do this automatically… that would be very nice.  Any guidance would be 
> appreciated.
>  
> -Aaron