Re: Spoofing ASNs (Re: SNMP DDoS: the vulnerability you might not know you have)

[Heather Schiller]

This is the RFC that is being violated:

http://tools.ietf.org/html/rfc5625#section-6.2

6.2 .  Interface Binding

   Some gateways have been observed to have their DNS proxy listening on
   both internal (LAN) and external (WAN) interfaces.  In this
   configuration, it is possible for the proxy to be used to mount
   reflector attacks as described in [RFC5358
].

   The DNS proxy in a gateway SHOULD NOT, by default, be accessible from
   the WAN interfaces of the device.



Implemented correctly, CPE receives DNS request and proxies the request
internally. Usually they are only listening on the LAN side.  Sometimes
they take DNS requests from the WAN side.  When a local DNS cache is not
defined, these requests can leak out to whatever cache is defined.  In some
cases, the CPE retains the IP address that originally sourced the packet.
 So you end up with external caches responding to requests from 3rd
parties.  Add to that spoofing the original packets to the bad CPE and you
have that CPE proxying your DNS amplification attack to other caches.
 Don't blame the cache.  The attacker was intending that the CPE amp
directly.

Fix:
CPE should disable DNS proxying by default and require a restricted list of
sources (LAN, vpn, etc) when enabled.  Also/alternatively require cache to
be local.

--Heather


On Sun, Aug 11, 2013 at 5:45 AM, Florian Weimer  wrote:

> * Jared Mauch:
>
> > Number of unique IPs that spoofed a packet to me. (eg: I sent a
> > packet to 1.2.3.4 and 5.6.7.8 responded).
>
> That's not necessarily proof of spoofing, isn't it?  The system in
> question might legitimately own IP addresses from very different
> networks.  If the system is a router and the service you're pinging is
> not correctly implemented and it picks up the IP address of the
> outgoing interface instead of the source address of the request,
> that's totally expected.
>
> I'm not saying that BCP 38 is widely implement (it's not, unless
> operators have configured exceptions for ICMP traffic from private
> address, which I very much doubt).  I just think you aren't actually
> measuring spoofing capabilities.
>
>

Re: Spoofing ASNs (Re: SNMP DDoS: the vulnerability you might not know you have)

[Florian Weimer]

* Christopher Morrow:

> On Sun, Aug 11, 2013 at 11:40 AM, Florian Weimer  wrote:
>
>> Apparently, they're implementing DNS proxy by destination-NATting, and
>> because they listen also on the WAN interface, they get the source
>> address wrong.
>>
>> This is quite scary.
>
> which part? the fact that most NAT implementations on CPE are crap? or
> the spoofing bit?

The spoofing bit.  Among other things, it makes the impact of CPE
crappiness non-localized.

Re: Spoofing ASNs (Re: SNMP DDoS: the vulnerability you might not know you have)

[Christopher Morrow]

On Sun, Aug 11, 2013 at 11:40 AM, Florian Weimer  wrote:

> Apparently, they're implementing DNS proxy by destination-NATting, and
> because they listen also on the WAN interface, they get the source
> address wrong.
>
> This is quite scary.

which part? the fact that most NAT implementations on CPE are crap? or
the spoofing bit?

Re: Spoofing ASNs (Re: SNMP DDoS: the vulnerability you might not know you have)

[Florian Weimer]

* Jared Mauch:

> The incidence rate is too high for it to be multihomed hosts.
>
> Let me know if you want to look at the raw data. Very interesting stuff.
>
> Or just look for 8.8.8.8 in the openresolverproject page.

Indeed, I could verify that 5.61.0.0 can indeed spoof one of my IP
addresses to the 8.8.8.8 DNS resolver.  For a cache miss, I get a
query from a Google IP address and the 8.8.8.8 reply has a plausible
TTL, so I don't think it's spoofing the response.

Apparently, they're implementing DNS proxy by destination-NATting, and
because they listen also on the WAN interface, they get the source
address wrong.

This is quite scary.

Re: Spoofing ASNs (Re: SNMP DDoS: the vulnerability you might not know you have)

[Jared Mauch]

The incidence rate is too high for it to be multihomed hosts.

Let me know if you want to look at the raw data. Very interesting stuff.

Or just look for 8.8.8.8 in the openresolverproject page.

- Jared 

On Aug 11, 2013, at 8:45 AM, Florian Weimer  wrote:

> * Jared Mauch:
> 
>> Number of unique IPs that spoofed a packet to me. (eg: I sent a
>> packet to 1.2.3.4 and 5.6.7.8 responded).
> 
> That's not necessarily proof of spoofing, isn't it?  The system in
> question might legitimately own IP addresses from very different
> networks.  If the system is a router and the service you're pinging is
> not correctly implemented and it picks up the IP address of the
> outgoing interface instead of the source address of the request,
> that's totally expected.
> 
> I'm not saying that BCP 38 is widely implement (it's not, unless
> operators have configured exceptions for ICMP traffic from private
> address, which I very much doubt).  I just think you aren't actually
> measuring spoofing capabilities.

Re: Spoofing ASNs (Re: SNMP DDoS: the vulnerability you might not know you have)

[Jimmy Hess]

A strange thought occurs
Regarding, devices that unintentionally have SNMP open to the public.

They might also have write access open,  which could enable reconfiguring
the device to facilitate full  TCP spoofing, or opening up a tunnel;
 enabling 3 way handshake and everything,  permitting the possible DDoS
 conditions to go well beyond simple UDP reflection.

Those devices that just have SNMP read access;   might reveal enough
information in the exposed MIBs about the device,  timestamps,  connection
table status.  for an attacker to successfully  inject  false data into
a TCP stream.

For example...  spoofing a TCP message containing a false BGP route
advertisement;   if enough about the state of the router's  TCP connection
table and synchronization numbers,  timestamp,  and other hints about the
state of the random number generator, can be discovered directly or
indirectly through some piece of data in the SNMP MIB


--
-Jimmy



On Sun, Aug 11, 2013 at 7:45 AM, Florian Weimer  wrote:

> * Jared Mauch:
>
> > Number of unique IPs that spoofed a packet to me. (eg: I sent a
> > packet to 1.2.3.4 and 5.6.7.8 responded).
>
> That's not necessarily proof of spoofing, isn't it?  The system in
> question might legitimately own IP addresses from very different
> networks.  If the system is a router and the service you're pinging is
> not correctly implemented and it picks up the IP address of the
> outgoing interface instead of the source address of the request,
> that's totally expected.
>
> I'm not saying that BCP 38 is widely implement (it's not, unless
> operators have configured exceptions for ICMP traffic from private
> address, which I very much doubt).  I just think you aren't actually
> measuring spoofing capabilities.
>
>


-- 
-Mysid

Re: Spoofing ASNs (Re: SNMP DDoS: the vulnerability you might not know you have)

[Florian Weimer]

* Jared Mauch:

> Number of unique IPs that spoofed a packet to me. (eg: I sent a
> packet to 1.2.3.4 and 5.6.7.8 responded).

That's not necessarily proof of spoofing, isn't it?  The system in
question might legitimately own IP addresses from very different
networks.  If the system is a router and the service you're pinging is
not correctly implemented and it picks up the IP address of the
outgoing interface instead of the source address of the request,
that's totally expected.

I'm not saying that BCP 38 is widely implement (it's not, unless
operators have configured exceptions for ICMP traffic from private
address, which I very much doubt).  I just think you aren't actually
measuring spoofing capabilities.

Re: Spoofing ASNs (Re: SNMP DDoS: the vulnerability you might not know you have)

[Jared Mauch]

On Aug 8, 2013, at 9:13 PM, Blake Dunlap  wrote:

> Thanks, this is quite interesting. I never would have expected that kind of 
> behavior.

I've been having trouble getting in touch with the Netgear security group about 
this, if someone knows how to contact them, I'd appreciate a referral on this 
topic :)

- Jared

Re: Spoofing ASNs (Re: SNMP DDoS: the vulnerability you might not know you have)

[Blake Dunlap]

Thanks, this is quite interesting. I never would have expected that kind of
behavior.

-Blake


On Thu, Aug 8, 2013 at 3:37 PM, Jared Mauch  wrote:

>
> On Aug 8, 2013, at 2:07 PM, Blake Dunlap  wrote:
>
> > On a related note, how are you actually getting this data?
>
> Sure:
>
>
> https://www.nanog.org/sites/default/files/tue.lightning3.open_resolver.mauch_.pdf
>
> I would point you at the streaming archive, but I'm not sure where they
> went.  Perhaps they can post them to Youtube?
>
> Anyways, the alternate set of IPs responding is actually increasing over
> time:
>
> http://openresolverproject.org/breakdown-graph2.cgi
>
> > What you have said previously ( Number of unique IPs that spoofed a
> packet to me. (eg: I sent a packet to 1.2.3.4 and 5.6.7.8 responded). )
> doesn't even make sense.
>
> Many CPE devices will perform NAT on udp/53 packets received on their WAN
> interface and forward them to their configured DNS server.  Some will just
> take the source IP and copy it into the packet.  Because it comes in on
> their WAN interface, it will instead of copying the inside NAT address just
> copy my source IP from the weekly scan and use that.  Since it's on the
> outside, it doesn't copy it's outside IP and put that in, it copies mine.
>
> - Jared
>
> > On Thu, Aug 8, 2013 at 12:51 PM, Jared Mauch 
> wrote:
> > Oops, I pulled the wrong data (off by one column) out before a trip and
> didn't realize it until now.
> >
> > This is not the spoofer list, but the list of ASNs with open resolvers.
> >
> > Let me reprocess it.
> >
> > Apologies, corrected data being generated.
> >
> > - Jared
> >
> > On Aug 8, 2013, at 1:29 PM, Jared Mauch  wrote:
> >
> > > The following is a sorted list from worst to best of networks that
> allow spoofing: (cutoff here is 25k)
> > >
> > > (full list -
> http://openresolverproject.org/full-spoofer-asn-list-201307.txt )
> >
> >
> >
>
>

Re: Spoofing ASNs (Re: SNMP DDoS: the vulnerability you might not know you have)

[Jared Mauch]

On Aug 8, 2013, at 2:07 PM, Blake Dunlap  wrote:

> On a related note, how are you actually getting this data?

Sure:

https://www.nanog.org/sites/default/files/tue.lightning3.open_resolver.mauch_.pdf

I would point you at the streaming archive, but I'm not sure where they went.  
Perhaps they can post them to Youtube?

Anyways, the alternate set of IPs responding is actually increasing over time:

http://openresolverproject.org/breakdown-graph2.cgi

> What you have said previously ( Number of unique IPs that spoofed a packet to 
> me. (eg: I sent a packet to 1.2.3.4 and 5.6.7.8 responded). ) doesn't even 
> make sense.

Many CPE devices will perform NAT on udp/53 packets received on their WAN 
interface and forward them to their configured DNS server.  Some will just take 
the source IP and copy it into the packet.  Because it comes in on their WAN 
interface, it will instead of copying the inside NAT address just copy my 
source IP from the weekly scan and use that.  Since it's on the outside, it 
doesn't copy it's outside IP and put that in, it copies mine.

- Jared

> On Thu, Aug 8, 2013 at 12:51 PM, Jared Mauch  wrote:
> Oops, I pulled the wrong data (off by one column) out before a trip and 
> didn't realize it until now.
> 
> This is not the spoofer list, but the list of ASNs with open resolvers.
> 
> Let me reprocess it.
> 
> Apologies, corrected data being generated.
> 
> - Jared
> 
> On Aug 8, 2013, at 1:29 PM, Jared Mauch  wrote:
> 
> > The following is a sorted list from worst to best of networks that allow 
> > spoofing: (cutoff here is 25k)
> >
> > (full list - 
> > http://openresolverproject.org/full-spoofer-asn-list-201307.txt )
> 
> 
> 

Re: Spoofing ASNs (Re: SNMP DDoS: the vulnerability you might not know you have)

[Jared Mauch]

All, 

Here's the correct list, apologies for the confusion.

http://openresolverproject.org/spoofers-20130804-byasn-count.txt

Top ASN excerpt:

  Count ASN

  46024 5617 
  43729 9394 
  28358 17964 
  27923 3269 
  24323 12874 
  22726 4847 
  22690 286 1136 
  21541 6079 
  20380 20825 
  11538 17430 
  10657 7497 17430 
  10544 4766 
   9883 7497 
   9061 3462 
   8875 38208 
   8553 7385 
   8295 4812 
   7297 11830 
   7204 7029 
   7137 3215 
   6655 6854 
   6618 4788 
   6424 17621 
   5794 53173 
   5069 8452 
   4944 9808 
   4930 6830 
   4877 38511 
   4648 4134 
   4135 2856 
   3982 9340 
   3678 6805 
   3605 38235 
   3398 17816 
   3364 9299 
   3297 9812 
   3238 15003 
   3221 9116 
   3025 4565 




On Aug 8, 2013, at 1:51 PM, Jared Mauch  wrote:

> Oops, I pulled the wrong data (off by one column) out before a trip and 
> didn't realize it until now.
> 
> This is not the spoofer list, but the list of ASNs with open resolvers.
> 
> Let me reprocess it.
> 
> Apologies, corrected data being generated.
> 
> - Jared
> 
> On Aug 8, 2013, at 1:29 PM, Jared Mauch  wrote:
> 
>> The following is a sorted list from worst to best of networks that allow 
>> spoofing: (cutoff here is 25k)
>> 
>> (full list - http://openresolverproject.org/full-spoofer-asn-list-201307.txt 
>> )
> 

Re: Spoofing ASNs (Re: SNMP DDoS: the vulnerability you might not know you have)

[Blake Dunlap]

On a related note, how are you actually getting this data?

What you have said previously ( Number of unique IPs that spoofed a packet
to me. (eg: I sent a packet to 1.2.3.4 and 5.6.7.8 responded). ) doesn't
even make sense.

-Blake


On Thu, Aug 8, 2013 at 12:51 PM, Jared Mauch  wrote:

> Oops, I pulled the wrong data (off by one column) out before a trip and
> didn't realize it until now.
>
> This is not the spoofer list, but the list of ASNs with open resolvers.
>
> Let me reprocess it.
>
> Apologies, corrected data being generated.
>
> - Jared
>
> On Aug 8, 2013, at 1:29 PM, Jared Mauch  wrote:
>
> > The following is a sorted list from worst to best of networks that allow
> spoofing: (cutoff here is 25k)
> >
> > (full list -
> http://openresolverproject.org/full-spoofer-asn-list-201307.txt )
>
>
>

Re: Spoofing ASNs (Re: SNMP DDoS: the vulnerability you might not know you have)

[Valdis . Kletnieks]

On Thu, 08 Aug 2013 12:46:10 -0500, Blake Dunlap said:
> I noticed that two of my ASNs are on that list for example with low
> numbers. I can't fathom how as at least one of them has uRPF implemented on
> any actual interfaces and no downstreams/peers.

Most likely, you have places where one host in a /24 or /28 can spoof
a packet claiming to be another host in the same subnet, and have the
spoofed packet escape into the outside world.  There's really no way to
stop that unless you get *really* fascist with your edge-host facing
routers/switches.


pgpDDbPG5qyAq.pgp
Description: PGP signature

Re: Spoofing ASNs (Re: SNMP DDoS: the vulnerability you might not know you have)

[Jared Mauch]

Oops, I pulled the wrong data (off by one column) out before a trip and didn't 
realize it until now.

This is not the spoofer list, but the list of ASNs with open resolvers.

Let me reprocess it.

Apologies, corrected data being generated.

- Jared

On Aug 8, 2013, at 1:29 PM, Jared Mauch  wrote:

> The following is a sorted list from worst to best of networks that allow 
> spoofing: (cutoff here is 25k)
> 
> (full list - http://openresolverproject.org/full-spoofer-asn-list-201307.txt )

Re: Spoofing ASNs (Re: SNMP DDoS: the vulnerability you might not know you have)

[Blake Dunlap]

I noticed that two of my ASNs are on that list for example with low
numbers. I can't fathom how as at least one of them has uRPF implemented on
any actual interfaces and no downstreams/peers.

-Blake

On Thu, Aug 8, 2013 at 12:40 PM, Matthew Petach wrote:

> On Thu, Aug 8, 2013 at 10:29 AM, Jared Mauch 
> wrote:
>
> >
> > On Aug 1, 2013, at 2:31 AM, Saku Ytti  wrote:
> >
> > > On (2013-07-31 17:07 -0700), bottiger wrote:
> > >
> > >> But realistically those 2 problems are not going to be solved any time
> > >> in the next decade. I have tested 7 large hosting networks only one of
> > >> them had BCP38.
> > >
> > > I wonder if it's truly that unrealistic. If we target access networks,
> it
> > > seems impractical target.
> > >
> > > We have about 40k origin only ASNs and about 7k ASNs which offer
> transit,
> > > who could arguably trivially ACL those 40k peers.
> > >
> > > If we truly tried, as a community to make deploying these ACLs easy and
> > > actively reach out those 7k ASNs and offer help, would it be
> unrealistic
> > to
> > > have ACL deployed to sufficiently large portion of networks to make
> > > spoofing impractical/expensive?
> >
> > The following is a sorted list from worst to best of networks that allow
> > spoofing: (cutoff here is 25k)
> >
> > (full list -
> > http://openresolverproject.org/full-spoofer-asn-list-201307.txt )
> >
> >
>
> > Count   ASN#
> > 
> > 1323950 3462
> > 1300938 4134
> > 1270046 8151
> > 1213972 9737
>
> ...
>
> For the technically clueless among us...
>
> what does "count" refer to in this output?
> How many times you were able to spoof
> an address through them?  How many
> different addresses you could spoof through
> them?  How many spoofed packets made it
> through before being blocked?
>
> It's kinda hard to know what the list
> represents without a bit of explanation
> around it.  ^_^;
>
> Thanks!  :)
>
> Matt
>

Re: Spoofing ASNs (Re: SNMP DDoS: the vulnerability you might not know you have)

[Jared Mauch]

On Aug 8, 2013, at 1:40 PM, Matthew Petach  wrote:

> 
> 
> On Thu, Aug 8, 2013 at 10:29 AM, Jared Mauch  wrote:
> 
> On Aug 1, 2013, at 2:31 AM, Saku Ytti  wrote:
> 
> > On (2013-07-31 17:07 -0700), bottiger wrote:
> >
> >> But realistically those 2 problems are not going to be solved any time
> >> in the next decade. I have tested 7 large hosting networks only one of
> >> them had BCP38.
> >
> > I wonder if it's truly that unrealistic. If we target access networks, it
> > seems impractical target.
> >
> > We have about 40k origin only ASNs and about 7k ASNs which offer transit,
> > who could arguably trivially ACL those 40k peers.
> >
> > If we truly tried, as a community to make deploying these ACLs easy and
> > actively reach out those 7k ASNs and offer help, would it be unrealistic to
> > have ACL deployed to sufficiently large portion of networks to make
> > spoofing impractical/expensive?
> 
> The following is a sorted list from worst to best of networks that allow 
> spoofing: (cutoff here is 25k)
> 
> (full list - http://openresolverproject.org/full-spoofer-asn-list-201307.txt )
> 
>  
> Count   ASN#
> 
> 1323950 3462
> 1300938 4134
> 1270046 8151
> 1213972 9737
> ...
> 
> For the technically clueless among us...
> 
> what does "count" refer to in this output?
> How many times you were able to spoof
> an address through them?  How many
> different addresses you could spoof through
> them?  How many spoofed packets made it
> through before being blocked?
> 
> It's kinda hard to know what the list
> represents without a bit of explanation
> around it.  ^_^;

Number of unique IPs that spoofed a packet to me. (eg: I sent a packet to 
1.2.3.4 and 5.6.7.8 responded).

If those ASNs are downstream to you, or you are part of that ASN, you can ask 
for a list of the IPs involved.

Either way, if you have 1.2 million hosts, it may be a lot of BCP38 you need to 
apply.

- Jared

Re: Spoofing ASNs (Re: SNMP DDoS: the vulnerability you might not know you have)

[Matthew Petach]

On Thu, Aug 8, 2013 at 10:29 AM, Jared Mauch  wrote:

>
> On Aug 1, 2013, at 2:31 AM, Saku Ytti  wrote:
>
> > On (2013-07-31 17:07 -0700), bottiger wrote:
> >
> >> But realistically those 2 problems are not going to be solved any time
> >> in the next decade. I have tested 7 large hosting networks only one of
> >> them had BCP38.
> >
> > I wonder if it's truly that unrealistic. If we target access networks, it
> > seems impractical target.
> >
> > We have about 40k origin only ASNs and about 7k ASNs which offer transit,
> > who could arguably trivially ACL those 40k peers.
> >
> > If we truly tried, as a community to make deploying these ACLs easy and
> > actively reach out those 7k ASNs and offer help, would it be unrealistic
> to
> > have ACL deployed to sufficiently large portion of networks to make
> > spoofing impractical/expensive?
>
> The following is a sorted list from worst to best of networks that allow
> spoofing: (cutoff here is 25k)
>
> (full list -
> http://openresolverproject.org/full-spoofer-asn-list-201307.txt )
>
>

> Count   ASN#
> 
> 1323950 3462
> 1300938 4134
> 1270046 8151
> 1213972 9737

...

For the technically clueless among us...

what does "count" refer to in this output?
How many times you were able to spoof
an address through them?  How many
different addresses you could spoof through
them?  How many spoofed packets made it
through before being blocked?

It's kinda hard to know what the list
represents without a bit of explanation
around it.  ^_^;

Thanks!  :)

Matt

Spoofing ASNs (Re: SNMP DDoS: the vulnerability you might not know you have)

[Jared Mauch]

On Aug 1, 2013, at 2:31 AM, Saku Ytti  wrote:

> On (2013-07-31 17:07 -0700), bottiger wrote:
> 
>> But realistically those 2 problems are not going to be solved any time
>> in the next decade. I have tested 7 large hosting networks only one of
>> them had BCP38.
> 
> I wonder if it's truly that unrealistic. If we target access networks, it
> seems impractical target.
> 
> We have about 40k origin only ASNs and about 7k ASNs which offer transit,
> who could arguably trivially ACL those 40k peers.
> 
> If we truly tried, as a community to make deploying these ACLs easy and
> actively reach out those 7k ASNs and offer help, would it be unrealistic to
> have ACL deployed to sufficiently large portion of networks to make
> spoofing impractical/expensive?

The following is a sorted list from worst to best of networks that allow 
spoofing: (cutoff here is 25k)

(full list - http://openresolverproject.org/full-spoofer-asn-list-201307.txt )

Count   ASN#

1323950 3462 
1300938 4134 
1270046 8151 
1213972 9737 
 851124 22927 
 706434 45899 
 532546 3816 
 497303 1267 
 487965 17974 
 486882 4837 
 433170 9829 
 425991 18403 
 422356 19429 
 406870 24560 
 378440 4766 
 357974 6697 
 341044 6147 
 332602 18881 
 251074 7303 
 238461 9318 
 221201 4812 
 217794 7418 
 213049 17552 
 181995 7552 
 159078 13489 
 153877 9299 
 142740 7738 
 138730 209 
 120860 8452 
 118506 46606 
 117700 14420 
 107600 17813 
 101967 36947 
  98708 6400 
  93526 36351 
  92471 4788 
  89976 9198 
  88570 11556 
  81665 9050 
  81624 27695 
  80837 13354 
  80415 701 
  79032 6332 
  78164 4808 
  77937 55430 
  75800 2554 
  65618 9394 
  63992 4713 
  60380 9808 
  59274 6057 
  55177 8400 
  53862 9269 
  53266 13285 
  51620 9329 
  50822 22833 
  50320 16276 
  49847 23752 
  48998 4780 
  48278 31549 
  47195 8167 
  46484 10299 
  46270 21844 
  43439 26599 
  43211 32475 
  43048 36444 
  41688 27668 
  35448 24863 
  34160 27866 
  33068 26496 
  32166 14754 
  31656 2379 
  31450 32613 
  30641 27699 
  29225 45951 
  28804 6389 
  27836 56040 
  27406 5617 
  26758 39501 
  26454 24940 
  26175 13999 
  25736 7018 
  25482 131090 
  25478 1221 

Re: SNMP DDoS: the vulnerability you might not know you have

[Saku Ytti]

On (2013-07-31 17:07 -0700), bottiger wrote:

> But realistically those 2 problems are not going to be solved any time
> in the next decade. I have tested 7 large hosting networks only one of
> them had BCP38.

I wonder if it's truly that unrealistic. If we target access networks, it
seems impractical target.

We have about 40k origin only ASNs and about 7k ASNs which offer transit,
who could arguably trivially ACL those 40k peers.

If we truly tried, as a community to make deploying these ACLs easy and
actively reach out those 7k ASNs and offer help, would it be unrealistic to
have ACL deployed to sufficiently large portion of networks to make
spoofing impractical/expensive?


Do we have other approaches? Can we make this ACL dynamic to a degree? Can
we extract ACL information from BGP table?
If origin only ASN advertises prefix to global table anywhere, allow it at
matching 'remote-as' port. Does not look like difficult feature to build,
does not require magic HW support, essentially dynamically built ACL.
After this spoof would require injected trash BGP route, which would also
steal return traffic, making it useless for DoS.


-- 
  ++ytti

Re: SNMP DDoS: the vulnerability you might not know you have

[Mark Andrews]

In message 
, bottiger writes:
> I realize the root cause is security-oblivious designers and one level
> below that, lack of BCP38.
> 
> But realistically those 2 problems are not going to be solved any time
> in the next decade. I have tested 7 large hosting networks only one of
> them had BCP38.

Then name and shame.
 
> To my knowledge it is practically impossible for someone outside a
> multi-homed network to know if they allow spoofing which means it will
> be difficult to punish. It also cost time and money to maintain these
> ACLs, much more than blocking the occasional wide-spread protocol with
> 8000x amplification every couple of years.

Just define a EDNS option which contains the real ip address and
update who-am-i services to look for that when replying, use a
modified DNS cookies or similar to prevent this being used as
amplifier.  You can do this with a experimental EDNS option code
points today.

Send a non-spoofed request then send a spoofed request using the
cookie learnt from the first request.

Standardise it and every nameserver on the planet and be used to
detect if spoofing is possible.  Using DNS puts up to costs for
ISP's that want to block people checking.

The blackhats already check whether spoofing is possible so there
is little point in blocking whitehats checking.

As for time and money to maintain these acls it really shouldn't
now that SIDR is becoming a reality.  Present a properly signed
CERT to the other providers and automatically generate the acls
based on those.  There is a little setup cost to get the system
running and almost no additional recurrent costs to keep it up
to date.  The CERTs need to be generated for sBGP anyway.

> I am here to talk about solutions today. BCP38 has been repeated to
> death and people aren't going to start doing it because I said so. The
> fact that the amplification factor is so high means that you need to
> ensure near 100% conformity even if everyone started to become BCP38
> compliant today.
> 
> On Wed, Jul 31, 2013 at 4:42 PM, Jimmy Hess  wrote:
> > On 7/31/13, Blake Dunlap  wrote:
> >> I bet blocking all SYN packets and non related flow UDP packets to
> >> customers would be even more effective. Why don't we do that and be done
> >> with it instead of playing whack a mole every 3 months when someone finds
> >> some new service that was poorly designed so that it can be used to send a
> >> flood?
> >
> > Because it breaks applications that people are paying to be able to use.
> >
> > The way I see it;  more and more samples keep getting found about
> > protocols abused because networks have not implemented BCP38.
> >
> > The latest SNMP trend is just another uptick to the sample size,  and
> > proof that  Closing off   perfectly OK  recursive DNS services  is
> > totally inadequate and not a useful long-term fix  to the problem of
> > DDoS or IP/UDP reflection attacks.
> >
> > Asking folks to improve the security of access to their SNMP instances
> > is just chasing the latest exploit implementation,  with no attention
> > to the vulnerability or the root cause
> >
> > --
> > -JH
> >
> 
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742 INTERNET: ma...@isc.org

Re: SNMP DDoS: the vulnerability you might not know you have

[bottiger]

I realize the root cause is security-oblivious designers and one level
below that, lack of BCP38.

But realistically those 2 problems are not going to be solved any time
in the next decade. I have tested 7 large hosting networks only one of
them had BCP38.

To my knowledge it is practically impossible for someone outside a
multi-homed network to know if they allow spoofing which means it will
be difficult to punish. It also cost time and money to maintain these
ACLs, much more than blocking the occasional wide-spread protocol with
8000x amplification every couple of years.

I am here to talk about solutions today. BCP38 has been repeated to
death and people aren't going to start doing it because I said so. The
fact that the amplification factor is so high means that you need to
ensure near 100% conformity even if everyone started to become BCP38
compliant today.

On Wed, Jul 31, 2013 at 4:42 PM, Jimmy Hess  wrote:
> On 7/31/13, Blake Dunlap  wrote:
>> I bet blocking all SYN packets and non related flow UDP packets to
>> customers would be even more effective. Why don't we do that and be done
>> with it instead of playing whack a mole every 3 months when someone finds
>> some new service that was poorly designed so that it can be used to send a
>> flood?
>
> Because it breaks applications that people are paying to be able to use.
>
> The way I see it;  more and more samples keep getting found about
> protocols abused because networks have not implemented BCP38.
>
> The latest SNMP trend is just another uptick to the sample size,  and
> proof that  Closing off   perfectly OK  recursive DNS services  is
> totally inadequate and not a useful long-term fix  to the problem of
> DDoS or IP/UDP reflection attacks.
>
> Asking folks to improve the security of access to their SNMP instances
> is just chasing the latest exploit implementation,  with no attention
> to the vulnerability or the root cause
>
> --
> -JH
>

Re: SNMP DDoS: the vulnerability you might not know you have

[Jimmy Hess]

On 7/31/13, Blake Dunlap  wrote:
> I bet blocking all SYN packets and non related flow UDP packets to
> customers would be even more effective. Why don't we do that and be done
> with it instead of playing whack a mole every 3 months when someone finds
> some new service that was poorly designed so that it can be used to send a
> flood?

Because it breaks applications that people are paying to be able to use.

The way I see it;  more and more samples keep getting found about
protocols abused because networks have not implemented BCP38.

The latest SNMP trend is just another uptick to the sample size,  and
proof that  Closing off   perfectly OK  recursive DNS services  is
totally inadequate and not a useful long-term fix  to the problem of
DDoS or IP/UDP reflection attacks.

Asking folks to improve the security of access to their SNMP instances
is just chasing the latest exploit implementation,  with no attention
to the vulnerability or the root cause

--
-JH

Re: SNMP DDoS: the vulnerability you might not know you have

[Ricky Beam]

On Wed, 31 Jul 2013 18:50:18 -0400, Larry Sheldon   
wrote:
But after years of research I will tell you that there is no way to stop  
an avalanche once it has been released at the source.


http://youtu.be/60loeoblu0M

Anyone can make a device and connect it to the internet.

Re: SNMP DDoS: the vulnerability you might not know you have

[Larry Sheldon]

On 7/31/2013 4:29 PM, Blake Dunlap wrote:


It works better to fix the design issues than to play whack a mole
by blocking every imaginable service to your customers that responds
to the public with data larger than a FIN. Like getting their
providers to more proactively police their spew, manufactures to stop
making negligent devices, or implementing more intelligent filter
communication so the only option doesn't begin with calling your
provider and asking them over the phone to block X ip for you since
you're off the internet.

Maybe even look into liability laws for allowing said attacks to
originate from your customers and not doing anything about it, or
being manufacturer of said devices that harm others through their
lack of due diligence implementing proper security. It's still way
more effective than trying to fix the *last instance* of the problem,
instead of it's reasons for enduring as an issue at a global scale.


The first time I became a pariah on NANOG was for postulating precisely 
that view--that if the originators of problems would stop, we would not 
have to figure out how to clean up after them.  But I am past that now 
and out of work.


But it does occur to me for the first time that I can recall, that maybe 
the tremendous efforts to Get Control Of The Intertubes could be 
suckered into doing some good, say be establishing a certification 
authority to test and certify the safety of designs (is Scott B? 
still around) and devise a way to not permit traffic from uncertified 
devices or configurations.


But after years of research I will tell you that there is no way to stop 
an avalanche once it has been released at the source.

--
Requiescas in pace o email   Two identifying characteristics
of System Administrators:
Ex turpi causa non oritur actio  Infallibility, and the ability to
learn from their mistakes.
  (Adapted from Stephen Pinker)

Re: SNMP DDoS: the vulnerability you might not know you have

[bottiger]

This vulnerability has been present ever since SNMP v2 was announced
back in 1993.

There is a reason why the biggest attacks these days are from
protocols that are decades old like DNS and Chargen.

People making widely spread protocols these days are aware of the
problem and are usually able to get their software to auto-update.

Enforcing source IP filtering seems like a lost cause. Not much
progress has been made that I can tell and it is difficult to discover
if the network allows it without being inside it.

I don't see many uses for unsecured SNMP access so this is not like
blocking all syn and udp packets.

On Wed, Jul 31, 2013 at 2:29 PM, Blake Dunlap  wrote:
> I bet blocking all SYN packets and non related flow UDP packets to
> customers would be even more effective. Why don't we do that and be done
> with it instead of playing whack a mole every 3 months when someone finds
> some new service that was poorly designed so that it can be used to send a
> flood?
>
> Yes, I'm being sarcastic above.
>
> This is hardly the first finger of death amplification attack, and I
> strongly doubt it'll be the last. Years ago it was smurf, then Quake
> servers, then DNS, then Battlefield boxes, etc etc. Now it seems to be snmp
> and recently chargen, and tomorrow it'll be some peer 2 peer service, the
> next day it'll be a voice app. It will never end, and breaking the internet
> port by port doesn't do anything to make it better.
>
> I've been the victim of week long DDoS attacks that took down our
> upstreams, not to mention us, and I still maintain the above.
>
> It works better to fix the design issues than to play whack a mole by
> blocking every imaginable service to your customers that responds to the
> public with data larger than a FIN. Like getting their providers to more
> proactively police their spew, manufactures to stop making negligent
> devices, or implementing more intelligent filter communication so the only
> option doesn't begin with calling your provider and asking them over the
> phone to block X ip for you since you're off the internet.
>
> Maybe even look into liability laws for allowing said attacks to originate
> from your customers and not doing anything about it, or being manufacturer
> of said devices that harm others through their lack of due diligence
> implementing proper security. It's still way more effective than trying to
> fix the *last instance* of the problem, instead of it's reasons for
> enduring as an issue at a global scale.
>
> -Blake
>
>
> On Wed, Jul 31, 2013 at 3:46 PM, Dobbins, Roland  wrote:
>
>>
>> On Aug 1, 2013, at 3:11 AM, bottiger wrote:
>>
>> > The most disturbing part is the lack of logging.
>>
>> Flow telemetry can be of use in this instance.
>>
>> ---
>> Roland Dobbins  // 
>>
>>   Luck is the residue of opportunity and design.
>>
>>-- John Milton
>>
>>
>>

Re: SNMP DDoS: the vulnerability you might not know you have

[Blake Dunlap]

I bet blocking all SYN packets and non related flow UDP packets to
customers would be even more effective. Why don't we do that and be done
with it instead of playing whack a mole every 3 months when someone finds
some new service that was poorly designed so that it can be used to send a
flood?

Yes, I'm being sarcastic above.

This is hardly the first finger of death amplification attack, and I
strongly doubt it'll be the last. Years ago it was smurf, then Quake
servers, then DNS, then Battlefield boxes, etc etc. Now it seems to be snmp
and recently chargen, and tomorrow it'll be some peer 2 peer service, the
next day it'll be a voice app. It will never end, and breaking the internet
port by port doesn't do anything to make it better.

I've been the victim of week long DDoS attacks that took down our
upstreams, not to mention us, and I still maintain the above.

It works better to fix the design issues than to play whack a mole by
blocking every imaginable service to your customers that responds to the
public with data larger than a FIN. Like getting their providers to more
proactively police their spew, manufactures to stop making negligent
devices, or implementing more intelligent filter communication so the only
option doesn't begin with calling your provider and asking them over the
phone to block X ip for you since you're off the internet.

Maybe even look into liability laws for allowing said attacks to originate
from your customers and not doing anything about it, or being manufacturer
of said devices that harm others through their lack of due diligence
implementing proper security. It's still way more effective than trying to
fix the *last instance* of the problem, instead of it's reasons for
enduring as an issue at a global scale.

-Blake


On Wed, Jul 31, 2013 at 3:46 PM, Dobbins, Roland  wrote:

>
> On Aug 1, 2013, at 3:11 AM, bottiger wrote:
>
> > The most disturbing part is the lack of logging.
>
> Flow telemetry can be of use in this instance.
>
> ---
> Roland Dobbins  // 
>
>   Luck is the residue of opportunity and design.
>
>-- John Milton
>
>
>

Re: SNMP DDoS: the vulnerability you might not know you have

[Dobbins, Roland]

On Aug 1, 2013, at 3:11 AM, bottiger wrote:

> The most disturbing part is the lack of logging. 

Flow telemetry can be of use in this instance.

---
Roland Dobbins  // 

  Luck is the residue of opportunity and design.

   -- John Milton

Re: SNMP DDoS: the vulnerability you might not know you have

[Warren Bailey]

Would it be possible to add SNMP to your (collective cable labs buddies) 
shapers and it would be taken care of prior to it leaving your network but 
after the cmts?


Sent from my Mobile Device.


 Original message 
From: "Livingood, Jason" 
Date: 07/31/2013 10:07 AM (GMT-08:00)
To: bottiger ,nanog@nanog.org
Subject: Re: SNMP DDoS: the vulnerability you might not know you have


A relevant paper was released by the BITAG, see 
http://www.bitag.org/report-snmp-ddos-attacks.php Section 7 includes 
recommendations.

See also this blog post I wrote one day short of a year ago that may be of 
interest: 
http://corporate.comcast.com/comcast-voices/taking-steps-to-prevent-unintentional-network-abuse

A remaining issue out there for the community is taking action to reduce 
spoofing. A related project is the Open Resolver Project at 
http://openresolverproject.org/.

- Jason



On 7/31/13 6:25 AM, "bottiger" 
mailto:bottige...@gmail.com>> wrote:

Before you skim past this email because you already read the Prolexic
report on it or some other article on the internet, there are 2
disturbing properties that I haven't found anywhere else online.

1) After sending abuse emails to many networks, we received many angry
replies that they monitored their traffic for days without seeing
anything (even as we were being attacked) and that their IPs were
spoofed and would block us for spamming them.

What we discovered was that their firewalls/routers/gateways coming
from vendors like Cisco and SonicWall apparently didn't record SNMP
traffic going in or out of themselves. We confirmed this multiple
times by running a query to an IP that was claimed to be clean and
watching the response come 10-60 seconds later because the device was
being so heavily abused.

2) SNMP reflection offers the largest amplification factor by far,
even surpassing DNS, Chargen, or NTP by a wide margin. I have tested a
68 byte query and received responses of up to 30,000 to 60,000 bytes.
The trick is to use GetBulkRequest to start enumerating from the first
OID and setting max repetitions to a large number. This is contrary to
the other articles online which suggest a much smaller amplification
factor with other queries.

This protocol is also prevalent in many devices ranging from routers
to printers.

To solve this problem you should block SNMP traffic coming from
outside your network and whitelist outside IPs that require it.

Re: SNMP DDoS: the vulnerability you might not know you have

[Warren Bailey]

Write into your TOS a block for SNMP. Deal with the whiners on a case by case 
basis. Problem solved.


Sent from my Mobile Device.


 Original message 
From: bottiger 
Date: 07/31/2013 1:13 PM (GMT-08:00)
To: Blake Dunlap 
Cc: nanog@nanog.org
Subject: Re: SNMP DDoS: the vulnerability you might not know you have


Public SNMP being exploited for 8000x amplification is a very serious
issue. It is
arguably worse than open email relays.

Not only does it expose critical information from your users
but it offers the largest possible amplified DDoS by far, likely
bigger than DNS when you take into account the amplification size and
ubiquity. It will also cause your user's device to lag.

The most disturbing part is the lack of logging. We have tried
reporting these exploited servers for many weeks and because of the
logging problem most of them never get shut down because they just
assume they were being spoofed. We even had replies threatening to
block us because they thought were because they couldn't see they
were sending anything. When we were reported chargen attacks we
had much more positive responses.

Maybe you could refine the block by denying SNMP requests with the
public string. As network operators some compromises must be
made for a problem of this magnitude instead of just saying that you
should only be the best dumb pipe you can be.

We have seen attacks large enough to disturb 10G uplinks so as network
administrators you should not ignore this issue because you think it
is a small problem affecting only end users. This will affect you once
more people figure out how to get 8000x amplification from it.

It is great news that Comcast is trying to proactively solve this
problem on their network and hope that more networks would follow
their example.

On Wed, Jul 31, 2013 at 8:24 AM, Blake Dunlap  wrote:
> Agreed, but progressively breaking every service on the internet at the edge
> because you think there might possibly be an issue just leads to bad places.
>
> Get better defaults sure, but don't slowly turn the internet into a cable
> distribution system because "they're just users". It's bad enough already,
> don't make it worse trying to solve every issue with the nuclear option
> before trying anything else.
>
> -Blake
>
>
> On Wed, Jul 31, 2013 at 10:17 AM, Thomas St-Pierre 
> wrote:
>>
>> The problem isn't the people on this list leaving the public snmp
>> community on their devices, it's the vendors of home routers leaving it
>> there in their devices. Normal end users don't know or even care what snmp
>> is. (nor can we expect them too)
>>
>> A simple scan of a large cable/dsl ISP's address space will likely net you
>> tens of thousands of devices which respond to the "public" snmp community.
>>
>> Thomas
>>
>>
>>
>> On 13-07-31 10:57 AM, "Blake Dunlap"  wrote:
>>
>> >This looks like more a security issue with the devices, not border
>> >security
>> >issues.
>> >
>> >If you're seeing replies of that size, it means the devices themselves
>> > are
>> >set up to allow public queries of their information (not secured by even
>> >keys), which no one should be comfortable with. People should never be
>> >leaving the public access snmp strings on devices even if they are
>> >internal. Edge blocking just masks the real issue.
>> >
>> >
>> >-Blake
>> >
>> >
>> >On Tue, Jul 30, 2013 at 11:25 PM, bottiger  wrote:
>> >
>> >> Before you skim past this email because you already read the Prolexic
>> >> report on it or some other article on the internet, there are 2
>> >> disturbing properties that I haven't found anywhere else online.
>> >>
>> >> 1) After sending abuse emails to many networks, we received many angry
>> >> replies that they monitored their traffic for days without seeing
>> >> anything (even as we were being attacked) and that their IPs were
>> >> spoofed and would block us for spamming them.
>> >>
>> >> What we discovered was that their firewalls/routers/gateways coming
>> >> from vendors like Cisco and SonicWall apparently didn't record SNMP
>> >> traffic going in or out of themselves. We confirmed this multiple
>> >> times by running a query to an IP that was claimed to be clean and
>> >> watching the response come 10-60 seconds later because the device was
>> >> being so heavily abused.
>> >>
>> >> 2) SNMP reflection offers the largest amplification factor by far,
>> >> even surpa

Re: SNMP DDoS: the vulnerability you might not know you have

[bottiger]

Public SNMP being exploited for 8000x amplification is a very serious
issue. It is
arguably worse than open email relays.

Not only does it expose critical information from your users
but it offers the largest possible amplified DDoS by far, likely
bigger than DNS when you take into account the amplification size and
ubiquity. It will also cause your user's device to lag.

The most disturbing part is the lack of logging. We have tried
reporting these exploited servers for many weeks and because of the
logging problem most of them never get shut down because they just
assume they were being spoofed. We even had replies threatening to
block us because they thought were because they couldn't see they
were sending anything. When we were reported chargen attacks we
had much more positive responses.

Maybe you could refine the block by denying SNMP requests with the
public string. As network operators some compromises must be
made for a problem of this magnitude instead of just saying that you
should only be the best dumb pipe you can be.

We have seen attacks large enough to disturb 10G uplinks so as network
administrators you should not ignore this issue because you think it
is a small problem affecting only end users. This will affect you once
more people figure out how to get 8000x amplification from it.

It is great news that Comcast is trying to proactively solve this
problem on their network and hope that more networks would follow
their example.

On Wed, Jul 31, 2013 at 8:24 AM, Blake Dunlap  wrote:
> Agreed, but progressively breaking every service on the internet at the edge
> because you think there might possibly be an issue just leads to bad places.
>
> Get better defaults sure, but don't slowly turn the internet into a cable
> distribution system because "they're just users". It's bad enough already,
> don't make it worse trying to solve every issue with the nuclear option
> before trying anything else.
>
> -Blake
>
>
> On Wed, Jul 31, 2013 at 10:17 AM, Thomas St-Pierre 
> wrote:
>>
>> The problem isn't the people on this list leaving the public snmp
>> community on their devices, it's the vendors of home routers leaving it
>> there in their devices. Normal end users don't know or even care what snmp
>> is. (nor can we expect them too)
>>
>> A simple scan of a large cable/dsl ISP's address space will likely net you
>> tens of thousands of devices which respond to the "public" snmp community.
>>
>> Thomas
>>
>>
>>
>> On 13-07-31 10:57 AM, "Blake Dunlap"  wrote:
>>
>> >This looks like more a security issue with the devices, not border
>> >security
>> >issues.
>> >
>> >If you're seeing replies of that size, it means the devices themselves
>> > are
>> >set up to allow public queries of their information (not secured by even
>> >keys), which no one should be comfortable with. People should never be
>> >leaving the public access snmp strings on devices even if they are
>> >internal. Edge blocking just masks the real issue.
>> >
>> >
>> >-Blake
>> >
>> >
>> >On Tue, Jul 30, 2013 at 11:25 PM, bottiger  wrote:
>> >
>> >> Before you skim past this email because you already read the Prolexic
>> >> report on it or some other article on the internet, there are 2
>> >> disturbing properties that I haven't found anywhere else online.
>> >>
>> >> 1) After sending abuse emails to many networks, we received many angry
>> >> replies that they monitored their traffic for days without seeing
>> >> anything (even as we were being attacked) and that their IPs were
>> >> spoofed and would block us for spamming them.
>> >>
>> >> What we discovered was that their firewalls/routers/gateways coming
>> >> from vendors like Cisco and SonicWall apparently didn't record SNMP
>> >> traffic going in or out of themselves. We confirmed this multiple
>> >> times by running a query to an IP that was claimed to be clean and
>> >> watching the response come 10-60 seconds later because the device was
>> >> being so heavily abused.
>> >>
>> >> 2) SNMP reflection offers the largest amplification factor by far,
>> >> even surpassing DNS, Chargen, or NTP by a wide margin. I have tested a
>> >> 68 byte query and received responses of up to 30,000 to 60,000 bytes.
>> >> The trick is to use GetBulkRequest to start enumerating from the first
>> >> OID and setting max repetitions to a large number. This is contrary to
>> >> the other articles online which suggest a much smaller amplification
>> >> factor with other queries.
>> >>
>> >> This protocol is also prevalent in many devices ranging from routers
>> >> to printers.
>> >>
>> >> To solve this problem you should block SNMP traffic coming from
>> >> outside your network and whitelist outside IPs that require it.
>> >>
>> >>
>>
>

Re: SNMP DDoS: the vulnerability you might not know you have

[Livingood, Jason]

A relevant paper was released by the BITAG, see 
http://www.bitag.org/report-snmp-ddos-attacks.php Section 7 includes 
recommendations.

See also this blog post I wrote one day short of a year ago that may be of 
interest: 
http://corporate.comcast.com/comcast-voices/taking-steps-to-prevent-unintentional-network-abuse

A remaining issue out there for the community is taking action to reduce 
spoofing. A related project is the Open Resolver Project at 
http://openresolverproject.org/.

- Jason



On 7/31/13 6:25 AM, "bottiger" 
mailto:bottige...@gmail.com>> wrote:

Before you skim past this email because you already read the Prolexic
report on it or some other article on the internet, there are 2
disturbing properties that I haven't found anywhere else online.

1) After sending abuse emails to many networks, we received many angry
replies that they monitored their traffic for days without seeing
anything (even as we were being attacked) and that their IPs were
spoofed and would block us for spamming them.

What we discovered was that their firewalls/routers/gateways coming
from vendors like Cisco and SonicWall apparently didn't record SNMP
traffic going in or out of themselves. We confirmed this multiple
times by running a query to an IP that was claimed to be clean and
watching the response come 10-60 seconds later because the device was
being so heavily abused.

2) SNMP reflection offers the largest amplification factor by far,
even surpassing DNS, Chargen, or NTP by a wide margin. I have tested a
68 byte query and received responses of up to 30,000 to 60,000 bytes.
The trick is to use GetBulkRequest to start enumerating from the first
OID and setting max repetitions to a large number. This is contrary to
the other articles online which suggest a much smaller amplification
factor with other queries.

This protocol is also prevalent in many devices ranging from routers
to printers.

To solve this problem you should block SNMP traffic coming from
outside your network and whitelist outside IPs that require it.

Re: SNMP DDoS: the vulnerability you might not know you have

[Enno Rey]

Hi,

On Wed, Jul 31, 2013 at 03:17:37PM +, Thomas St-Pierre wrote:
> The problem isn't the people on this list leaving the public snmp
> community on their devices, it's the vendors of home routers leaving it
> there in their devices. Normal end users don't know or even care what snmp
> is. (nor can we expect them too)
> 
> A simple scan of a large cable/dsl ISP's address space will likely net you
> tens of thousands of devices which respond to the "public" snmp community.

I can confirm this.
we did some enumeration (and discussed the said amplification attack) here:
http://conference.hitb.org/hitbsecconf2007dubai/materials/D1%20-%20Enno%20Rey%20-%20Digging%20into%20SNMP%202007%20-%20An%20Excercise%20on%20Breaking%20Networks.pdf

at the time once you scanned "typical broadband segments" of major European 
carriers, pretty much every address responding to a ping had SNMP "public" 
also. 

we gave the talk several times and demoed the amplification attack (with a 
slightly modified version of this tool: 
https://www.ernw.de/download/snmpattack.pl) against some of our systems, 
abusing $SOME_RANDOM_SEGMENT as amplifiers (we asked to stop [camera] recording 
in those cases where the talks were recorded) and it worked pretty much all the 
time (~20:1 ratio, initiated from the respective conferences' hotel wifi).

thanks

Enno




> 
> Thomas
> 
> 
> 
> On 13-07-31 10:57 AM, "Blake Dunlap"  wrote:
> 
> >This looks like more a security issue with the devices, not border
> >security
> >issues.
> >
> >If you're seeing replies of that size, it means the devices themselves are
> >set up to allow public queries of their information (not secured by even
> >keys), which no one should be comfortable with. People should never be
> >leaving the public access snmp strings on devices even if they are
> >internal. Edge blocking just masks the real issue.
> >
> >
> >-Blake
> >
> >
> >On Tue, Jul 30, 2013 at 11:25 PM, bottiger  wrote:
> >
> >> Before you skim past this email because you already read the Prolexic
> >> report on it or some other article on the internet, there are 2
> >> disturbing properties that I haven't found anywhere else online.
> >>
> >> 1) After sending abuse emails to many networks, we received many angry
> >> replies that they monitored their traffic for days without seeing
> >> anything (even as we were being attacked) and that their IPs were
> >> spoofed and would block us for spamming them.
> >>
> >> What we discovered was that their firewalls/routers/gateways coming
> >> from vendors like Cisco and SonicWall apparently didn't record SNMP
> >> traffic going in or out of themselves. We confirmed this multiple
> >> times by running a query to an IP that was claimed to be clean and
> >> watching the response come 10-60 seconds later because the device was
> >> being so heavily abused.
> >>
> >> 2) SNMP reflection offers the largest amplification factor by far,
> >> even surpassing DNS, Chargen, or NTP by a wide margin. I have tested a
> >> 68 byte query and received responses of up to 30,000 to 60,000 bytes.
> >> The trick is to use GetBulkRequest to start enumerating from the first
> >> OID and setting max repetitions to a large number. This is contrary to
> >> the other articles online which suggest a much smaller amplification
> >> factor with other queries.
> >>
> >> This protocol is also prevalent in many devices ranging from routers
> >> to printers.
> >>
> >> To solve this problem you should block SNMP traffic coming from
> >> outside your network and whitelist outside IPs that require it.
> >>
> >>
> 
> 

-- 
Enno Rey

ERNW GmbH - Carl-Bosch-Str. 4 - 69115 Heidelberg - www.ernw.de
Tel. +49 6221 480390 - Fax 6221 419008 - Cell +49 174 3082474

Handelsregister Mannheim: HRB 337135
Geschaeftsfuehrer: Enno Rey

Troopers 2013 Videos online: 
http://www.youtube.com/user/TROOPERScon?feature=watch

===
Blog: www.insinuator.net || Conference: www.troopers.de
===

RE: SNMP DDoS: the vulnerability you might not know you have

[James Braunegg]

These attacks are known as SAD attacks ;( ... yes very SAD ;(

SNMP Amplification DDoS


Kindest Regards 

James Braunegg
P:  1300 769 972  |  M:  0488 997 207 |  D:  (03) 9751 7616
E:   james.braun...@micron21.com  |  ABN:  12 109 977 666   
W:  www.micron21.com/tv-hosting  T: @micron21



This message is intended for the addressee named above. It may contain 
privileged or confidential information. If you are not the intended recipient 
of this message you must not use, copy, distribute or disclose it to anyone 
other than the addressee. If you have received this message in error please 
return the message to the sender by replying to it and then delete the message 
from your computer.


-Original Message-
From: Blake Dunlap [mailto:iki...@gmail.com] 
Sent: Thursday, August 01, 2013 1:25 AM
To: Thomas St-Pierre
Cc: nanog@nanog.org
Subject: Re: SNMP DDoS: the vulnerability you might not know you have

Agreed, but progressively breaking every service on the internet at the
edge because you think there might possibly be an issue just leads to bad
places.

Get better defaults sure, but don't slowly turn the internet into a cable
distribution system because "they're just users". It's bad enough already,
don't make it worse trying to solve every issue with the nuclear option
before trying anything else.

-Blake


On Wed, Jul 31, 2013 at 10:17 AM, Thomas St-Pierre wrote:

> The problem isn't the people on this list leaving the public snmp
> community on their devices, it's the vendors of home routers leaving it
> there in their devices. Normal end users don't know or even care what snmp
> is. (nor can we expect them too)
>
> A simple scan of a large cable/dsl ISP's address space will likely net you
> tens of thousands of devices which respond to the "public" snmp community.
>
> Thomas
>
>
>
> On 13-07-31 10:57 AM, "Blake Dunlap"  wrote:
>
> >This looks like more a security issue with the devices, not border
> >security
> >issues.
> >
> >If you're seeing replies of that size, it means the devices themselves are
> >set up to allow public queries of their information (not secured by even
> >keys), which no one should be comfortable with. People should never be
> >leaving the public access snmp strings on devices even if they are
> >internal. Edge blocking just masks the real issue.
> >
> >
> >-Blake
> >
> >
> >On Tue, Jul 30, 2013 at 11:25 PM, bottiger  wrote:
> >
> >> Before you skim past this email because you already read the Prolexic
> >> report on it or some other article on the internet, there are 2
> >> disturbing properties that I haven't found anywhere else online.
> >>
> >> 1) After sending abuse emails to many networks, we received many angry
> >> replies that they monitored their traffic for days without seeing
> >> anything (even as we were being attacked) and that their IPs were
> >> spoofed and would block us for spamming them.
> >>
> >> What we discovered was that their firewalls/routers/gateways coming
> >> from vendors like Cisco and SonicWall apparently didn't record SNMP
> >> traffic going in or out of themselves. We confirmed this multiple
> >> times by running a query to an IP that was claimed to be clean and
> >> watching the response come 10-60 seconds later because the device was
> >> being so heavily abused.
> >>
> >> 2) SNMP reflection offers the largest amplification factor by far,
> >> even surpassing DNS, Chargen, or NTP by a wide margin. I have tested a
> >> 68 byte query and received responses of up to 30,000 to 60,000 bytes.
> >> The trick is to use GetBulkRequest to start enumerating from the first
> >> OID and setting max repetitions to a large number. This is contrary to
> >> the other articles online which suggest a much smaller amplification
> >> factor with other queries.
> >>
> >> This protocol is also prevalent in many devices ranging from routers
> >> to printers.
> >>
> >> To solve this problem you should block SNMP traffic coming from
> >> outside your network and whitelist outside IPs that require it.
> >>
> >>
>
>

Re: SNMP DDoS: the vulnerability you might not know you have

[Blake Dunlap]

Agreed, but progressively breaking every service on the internet at the
edge because you think there might possibly be an issue just leads to bad
places.

Get better defaults sure, but don't slowly turn the internet into a cable
distribution system because "they're just users". It's bad enough already,
don't make it worse trying to solve every issue with the nuclear option
before trying anything else.

-Blake


On Wed, Jul 31, 2013 at 10:17 AM, Thomas St-Pierre wrote:

> The problem isn't the people on this list leaving the public snmp
> community on their devices, it's the vendors of home routers leaving it
> there in their devices. Normal end users don't know or even care what snmp
> is. (nor can we expect them too)
>
> A simple scan of a large cable/dsl ISP's address space will likely net you
> tens of thousands of devices which respond to the "public" snmp community.
>
> Thomas
>
>
>
> On 13-07-31 10:57 AM, "Blake Dunlap"  wrote:
>
> >This looks like more a security issue with the devices, not border
> >security
> >issues.
> >
> >If you're seeing replies of that size, it means the devices themselves are
> >set up to allow public queries of their information (not secured by even
> >keys), which no one should be comfortable with. People should never be
> >leaving the public access snmp strings on devices even if they are
> >internal. Edge blocking just masks the real issue.
> >
> >
> >-Blake
> >
> >
> >On Tue, Jul 30, 2013 at 11:25 PM, bottiger  wrote:
> >
> >> Before you skim past this email because you already read the Prolexic
> >> report on it or some other article on the internet, there are 2
> >> disturbing properties that I haven't found anywhere else online.
> >>
> >> 1) After sending abuse emails to many networks, we received many angry
> >> replies that they monitored their traffic for days without seeing
> >> anything (even as we were being attacked) and that their IPs were
> >> spoofed and would block us for spamming them.
> >>
> >> What we discovered was that their firewalls/routers/gateways coming
> >> from vendors like Cisco and SonicWall apparently didn't record SNMP
> >> traffic going in or out of themselves. We confirmed this multiple
> >> times by running a query to an IP that was claimed to be clean and
> >> watching the response come 10-60 seconds later because the device was
> >> being so heavily abused.
> >>
> >> 2) SNMP reflection offers the largest amplification factor by far,
> >> even surpassing DNS, Chargen, or NTP by a wide margin. I have tested a
> >> 68 byte query and received responses of up to 30,000 to 60,000 bytes.
> >> The trick is to use GetBulkRequest to start enumerating from the first
> >> OID and setting max repetitions to a large number. This is contrary to
> >> the other articles online which suggest a much smaller amplification
> >> factor with other queries.
> >>
> >> This protocol is also prevalent in many devices ranging from routers
> >> to printers.
> >>
> >> To solve this problem you should block SNMP traffic coming from
> >> outside your network and whitelist outside IPs that require it.
> >>
> >>
>
>

Re: SNMP DDoS: the vulnerability you might not know you have

[Thomas St-Pierre]

The problem isn't the people on this list leaving the public snmp
community on their devices, it's the vendors of home routers leaving it
there in their devices. Normal end users don't know or even care what snmp
is. (nor can we expect them too)

A simple scan of a large cable/dsl ISP's address space will likely net you
tens of thousands of devices which respond to the "public" snmp community.

Thomas



On 13-07-31 10:57 AM, "Blake Dunlap"  wrote:

>This looks like more a security issue with the devices, not border
>security
>issues.
>
>If you're seeing replies of that size, it means the devices themselves are
>set up to allow public queries of their information (not secured by even
>keys), which no one should be comfortable with. People should never be
>leaving the public access snmp strings on devices even if they are
>internal. Edge blocking just masks the real issue.
>
>
>-Blake
>
>
>On Tue, Jul 30, 2013 at 11:25 PM, bottiger  wrote:
>
>> Before you skim past this email because you already read the Prolexic
>> report on it or some other article on the internet, there are 2
>> disturbing properties that I haven't found anywhere else online.
>>
>> 1) After sending abuse emails to many networks, we received many angry
>> replies that they monitored their traffic for days without seeing
>> anything (even as we were being attacked) and that their IPs were
>> spoofed and would block us for spamming them.
>>
>> What we discovered was that their firewalls/routers/gateways coming
>> from vendors like Cisco and SonicWall apparently didn't record SNMP
>> traffic going in or out of themselves. We confirmed this multiple
>> times by running a query to an IP that was claimed to be clean and
>> watching the response come 10-60 seconds later because the device was
>> being so heavily abused.
>>
>> 2) SNMP reflection offers the largest amplification factor by far,
>> even surpassing DNS, Chargen, or NTP by a wide margin. I have tested a
>> 68 byte query and received responses of up to 30,000 to 60,000 bytes.
>> The trick is to use GetBulkRequest to start enumerating from the first
>> OID and setting max repetitions to a large number. This is contrary to
>> the other articles online which suggest a much smaller amplification
>> factor with other queries.
>>
>> This protocol is also prevalent in many devices ranging from routers
>> to printers.
>>
>> To solve this problem you should block SNMP traffic coming from
>> outside your network and whitelist outside IPs that require it.
>>
>>

Re: SNMP DDoS: the vulnerability you might not know you have

[Blake Dunlap]

This looks like more a security issue with the devices, not border security
issues.

If you're seeing replies of that size, it means the devices themselves are
set up to allow public queries of their information (not secured by even
keys), which no one should be comfortable with. People should never be
leaving the public access snmp strings on devices even if they are
internal. Edge blocking just masks the real issue.


-Blake


On Tue, Jul 30, 2013 at 11:25 PM, bottiger  wrote:

> Before you skim past this email because you already read the Prolexic
> report on it or some other article on the internet, there are 2
> disturbing properties that I haven't found anywhere else online.
>
> 1) After sending abuse emails to many networks, we received many angry
> replies that they monitored their traffic for days without seeing
> anything (even as we were being attacked) and that their IPs were
> spoofed and would block us for spamming them.
>
> What we discovered was that their firewalls/routers/gateways coming
> from vendors like Cisco and SonicWall apparently didn't record SNMP
> traffic going in or out of themselves. We confirmed this multiple
> times by running a query to an IP that was claimed to be clean and
> watching the response come 10-60 seconds later because the device was
> being so heavily abused.
>
> 2) SNMP reflection offers the largest amplification factor by far,
> even surpassing DNS, Chargen, or NTP by a wide margin. I have tested a
> 68 byte query and received responses of up to 30,000 to 60,000 bytes.
> The trick is to use GetBulkRequest to start enumerating from the first
> OID and setting max repetitions to a large number. This is contrary to
> the other articles online which suggest a much smaller amplification
> factor with other queries.
>
> This protocol is also prevalent in many devices ranging from routers
> to printers.
>
> To solve this problem you should block SNMP traffic coming from
> outside your network and whitelist outside IPs that require it.
>
>

SNMP DDoS: the vulnerability you might not know you have

[bottiger]

Before you skim past this email because you already read the Prolexic
report on it or some other article on the internet, there are 2
disturbing properties that I haven't found anywhere else online.

1) After sending abuse emails to many networks, we received many angry
replies that they monitored their traffic for days without seeing
anything (even as we were being attacked) and that their IPs were
spoofed and would block us for spamming them.

What we discovered was that their firewalls/routers/gateways coming
from vendors like Cisco and SonicWall apparently didn't record SNMP
traffic going in or out of themselves. We confirmed this multiple
times by running a query to an IP that was claimed to be clean and
watching the response come 10-60 seconds later because the device was
being so heavily abused.

2) SNMP reflection offers the largest amplification factor by far,
even surpassing DNS, Chargen, or NTP by a wide margin. I have tested a
68 byte query and received responses of up to 30,000 to 60,000 bytes.
The trick is to use GetBulkRequest to start enumerating from the first
OID and setting max repetitions to a large number. This is contrary to
the other articles online which suggest a much smaller amplification
factor with other queries.

This protocol is also prevalent in many devices ranging from routers
to printers.

To solve this problem you should block SNMP traffic coming from
outside your network and whitelist outside IPs that require it.