Re: [secdir] secdir review of draft-ietf-dnsop-reflectors-are-evil-04.txt
On Monday, October 01, 2007 10:34:37 AM -0600 Danny McPherson [EMAIL PROTECTED] wrote: Note that in real deployments just this behavior has broken things on occasion, as many firewall and other such policy application points assume things like DNS resolution will only be UDP/53 transactions. Yeah; I'm getting a little tired of having our protocols redefined based on the incorrect assumptions of people who don't understand them. The DNS sometimes uses TCP, UDP flows can last more than one round trip, and ICMP unreachable messages are an essential part of IP; vendors and operators who assume otherwise should be made to fix their assumptions, instead of everyone else having to cripple their applications and networks to make the assumptions true. -- Jeff ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: [secdir] secdir review of draft-ietf-dnsop-reflectors-are-evil-04.txt
From: [EMAIL PROTECTED] Second, you're talking about potentially orders of magnitude more data: for each destination, there are worldwide likely hundreds (or more) of ISP's which are likely to be viable backbone transits. ... With ISP's which are directly connected with a customer, one can assume that there's an implicit authorization, but what about steps past that? If one further assumes that by p connecting to q, p is transitively authorizing q (with respect to [destination] A), then that can be done with straightforward (albeit complex and expensive) security on routing. However, if you go that way, then you lose the ability for A to make assertions about which q's are not acceptable. However, if you go that way, then you lose the ability for A to assertions about which q's are not acceptable. If you add that back in as a policy knob ... We have the technology to deal with orders of magnitude more data, assuming that the task is delegated to servers, not to routers. I guess I wasn't explicit enough; I made a reference to straightforward (albeit complex and expensive) security on routing, referring to automatic mechanisms, but didn't specifically describe them as automatic; nor did I specifically refer to the policy knob as being something that was manual. I'm talking about *configuration* data; i.e. something humans have to enter, and maintain. The issue is not the mechanical tasks of storing and distributing the data, but the human effort required to set up it and update it as needed, plus the likelihood (as always, when people are involved) of errors being introduced. Noel PS: The potential size of the problem is actually worse than I described above, because there's an additional axis of freedom. Not only do you have to multiply the number of destinations by the number of backbone transits, you also have to multiply by the number of destinations *again*, because destination A may think backbone transit q is fine, with respect to communicating with B, but not when it's communicating with C. Fundamentally, destination-vector systems (of which path-vector, such as BGP, are a subset) are just not suited to doing this kind of stuff. ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: [secdir] secdir review of draft-ietf-dnsop-reflectors-are-evil-04.txt
Danny McPherson writes: Really? How many ISPs are you aware of that have *decommissioned* every piece of routing gear in their network in the past 7 years? I think we're an ISP (AS559), and we don't have any equipment that would be unable to filter against spoofed source addresses. In fact I think that even seven years ago all our equipment could do this. Yes, I know about certain Engine-N generations of certain linecards for certain high-end routers, but my sympathy to those ISPs who still use them is limited. (Especially with those ISPs that keep a few in their network just as an excuse for not deploying BCP39 anywhere :-) -- Simon. ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: [secdir] secdir review of draft-ietf-dnsop-reflectors-are-evil-04.txt
On Oct 1, 2007, at 9:24 PM, Mark Andrews wrote: Note that in real deployments just this behavior has broken things on occasion, as many firewall and other such policy application points assume things like DNS resolution will only be UDP/53 transactions. That assumption has always been wrong. Not in my experience. Actually, there are two separate things here. One, is implementation/ product, the other is configuration and device administration. I'm not sure how your average user would separate the two from a practical standpoint, and it really doesn't matter. I'm aware of at two products in the last few months that, in production deployment forced TCP switch-over, only to find that this broke name resolution completely for a large pool of subscribers. In addition, in my own experience, more often than not when folks clamp down firewall policies, in particular in enterprise or restricted space, they often deny all TCP/53 to address spaces (in one case the culprit for the brokenness above). Another common place to see policies that block TCP/53 is roaming access points captive user environments. E.g., SSH tunneling over DNS was easy enough over UDP. To further support my statement, just google for +firewall policy +TCP/53 +DNS, here are a few examples: http://www.whitehats.ca/downloads/cerberus/Rick_Wanner_GCFW.pdf Service: The enabled service is DNS (domain-udp, port 53/udp). Firewall-1’s DNS service by default contains both domain-udp (53/udp) and domain-tcp (53/tcp). We have removed domain- tcp from the object definition, on the grounds that we will not be permitting zone transfers. It will be necessary to watch carefully since removing domain-tcp also means that long dns-queries will not be supported. It is important to note that this will not work unless “Accept UDP replies” is enabled on the Firewall-1 Security Properties screen. Without “Accept UDP replies” enabled, the queries will still be allowed through the firewall, but the replies will be dropped on the firewall. http://security.ucdavis.edu/basic_firewall_rules.pdf: Allow DNS (UDP 53) to internal DNS server – If the unit runs internal DNS servers this rule is recommended. The rule is needed if a Windows Active Directory server is hosted on the internal network. You must permit TCP 53 for zone transfer capability, however this permission should not be applied by default. Right or wrong, it's quite common. I would also dispute the many above. Most firewalls actually handle the transition to TCP perfectly fine. There are the odd few that are misconfigured. When that happens people complain because nameservers resolution fails. Either the dataset is fixed or the firewall is fixed. I'd be quite interested in any pointers you might have to empirical evidence supporting this position. I don't believe it's an odd few that are misconfigured, I believe it's often done as a conscious effort. -danny ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: [secdir] secdir review of draft-ietf-dnsop-reflectors-are-evil-04.txt
On Oct 1, 2007, at 8:55 PM, Mark Andrews wrote: I'm not blaming the victim, I'm pointing out the contributory negligence on behalf of the ISP that is supplying the attacker bandwidth. BCP 38 is over 7 years old now. The time has past where you can blame the hardware vendors for lack of support. The blame now has to be squarely brought down on the ISP's that have failed to deploy BCP 38. Really? How many ISPs are you aware of that have *decommissioned* every piece of routing gear in their network in the past 7 years? The ugly bit here is that routers usually are pushed further and further to the edge of the network, until they finally fall off. The closer to the edge they get to the edge, the less feature capability they usually have, while all the more you need them. Furthermore, it's pretty much impossible to explicitly filter based on sources from large peers with lots of routes and lots of churn, or ever large customers, for that matter. Dynamically generated feasible path RPF models are the best solution for this, but there's little (no?) support. And even those models are derived based on RIB data, which means route policy essentially dictates what you'll accept for both data plane and control plane. But wait, where's the authoritative source for who owns what prefixes, and who's permitted to originate/transit what prefixes? BTW: Even NIST Guidelines on Firewalls and Firewall Policy recommends blocking TCP/53, except for external secondaries. -danny ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: [secdir] secdir review of draft-ietf-dnsop-reflectors-are-evil-04.txt
On Oct 1, 2007, at 9:24 PM, Mark Andrews wrote: Note that in real deployments just this behavior has broken things on occasion, as many firewall and other such policy application =20 points assume things like DNS resolution will only be UDP/53 transactions. That assumption has always been wrong. Not in my experience. I repeat. The ASSUMPTION as ALWAYS been wrong. From day one the DNS specified fall back to TCP for QUERY. Actually, there are two separate things here. One, is implementation/ product, the other is configuration and device administration. I'm not sure how your average user would separate the two from a practical standpoint, and it really doesn't matter. I'm aware of at two products in the last few months that, in production deployment forced TCP switch-over, only to find that this broke name resolution completely for a large pool of subscribers. It still doesn't mean that many are badly configured only that some are badly configured. In addition, in my own experience, more often than not when folks clamp down firewall policies, in particular in enterprise or =20 restricted space, they often deny all TCP/53 to address spaces (in one case the culprit for the brokenness above). And I presume they adjusted the policy once they discovered how idiotic it is. Another common place to see policies that block TCP/53 is roaming access points captive user environments. E.g., SSH tunneling over DNS was easy enough over UDP. I didn't say that it didn't occur. Some people are stupid. To further support my statement, just google for +firewall policy +TCP/53 +DNS, here are a few examples: http://www.whitehats.ca/downloads/cerberus/Rick_Wanner_GCFW.pdf Service: The enabled service is DNS (domain-udp, port 53/udp). =20 Firewall-1=92s DNS service by default contains both domain-udp (53/udp) and domain-tcp (53/tcp). =20 We have removed domain- tcp from the object definition, on the grounds that we will not be =20 permitting zone transfers. It will be necessary to watch carefully since removing domain-tcp also means =20 that long dns-queries will not be supported. It is important to note that this will not work =20 unless =93Accept UDP replies=94 is enabled on the Firewall-1 Security Properties screen. Without =20 =93Accept UDP replies=94 enabled, the queries will still be allowed through the firewall, but the replies =20 will be dropped on the firewall. So. It just means they intend to live dangerously. This policy will bite them at some point. http://security.ucdavis.edu/basic_firewall_rules.pdf: Allow DNS (UDP 53) to internal DNS server =96 If the unit runs internal =20= DNS servers this rule is recommended. The rule is needed if a Windows Active Directory =20= server is hosted on the internal network. You must permit TCP 53 for zone transfer =20 capability, however this permission should not be applied by default. Someone should talk to ucdavis.edu and get this idiocy pulled. Right or wrong, it's quite common. I would also dispute the many above. Most firewalls actually handle the transition to TCP perfectly fine. There are the odd few that are misconfigured. When that happens people complain because nameservers resolution fails. Either the dataset is fixed or the firewall is fixed. I'd be quite interested in any pointers you might have to empirical evidence supporting this position. I don't believe it's an odd few that are misconfigured, I believe it's often done as a conscious effort. Because there are lots of recursive and authoritative nameservers out there behind firewalls that get it right. I've seen many more complaints about UDP packets 512 bytes being blocked than complaints about fallback to TCP failing. Most people actually do the right thing without thinking about it. The allow TCP out to anything this includes DNS servers. Most allow both UDP and TCP in to their nameservers. This is the silent majority. Mark -danny= -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: [EMAIL PROTECTED] ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: [secdir] secdir review of draft-ietf-dnsop-reflectors-are-evil-04.txt
On Oct 2, 2007, at 1:41 AM, Mark Andrews wrote: Someone should talk to ucdavis.edu and get this idiocy pulled. And NIST, and many many others.. Because there are lots of recursive and authoritative nameservers out there behind firewalls that get it right. I've seen many more complaints about UDP packets 512 bytes being blocked than complaints about fallback to TCP failing. Most people actually do the right thing without thinking about it. The allow TCP out to anything this includes DNS servers. Most allow both UDP and TCP in to their nameservers. This is the silent majority. Again, any pointers empirical data along these lines would be appreciated. -danny ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: [secdir] secdir review of draft-ietf-dnsop-reflectors-are-evil-04.txt
On Oct 1, 2007, at 8:55 PM, Mark Andrews wrote: I'm not blaming the victim, I'm pointing out the contributory negligence on behalf of the ISP that is supplying the attacker bandwidth. BCP 38 is over 7 years old now. The time has past where you can blame the hardware vendors for lack of support. The blame now has to be squarely brought down on the ISP's that have failed to deploy BCP 38. Really? How many ISPs are you aware of that have *decommissioned* every piece of routing gear in their network in the past 7 years? The ugly bit here is that routers usually are pushed further and further to the edge of the network, until they finally fall off. The closer to the edge they get to the edge, the less feature capability they usually have, while all the more you need them. Again, its the ISP's *choice* to use such equipment. At some point someone that has been attacked will sue the connecting ISP's from which the packets originated and I wouldn't be suprised to see the ISP being fined or otherwise penalised. Furthermore, it's pretty much impossible to explicitly filter based on sources from large peers with lots of routes and lots of churn, or ever large customers, for that matter. Dynamically generated feasible path RPF models are the best solution for this, but there's little (no?) support. BCP 38 is about *everyone* filtering as close to the source as possible. You do your part and your peer does their part. And even those models are derived based on RIB data, which means route policy essentially dictates what you'll accept for both data plane and control plane. But wait, where's the authoritative source for who owns what prefixes, and who's permitted to originate/transit what prefixes? BTW: Even NIST Guidelines on Firewalls and Firewall Policy recommends blocking TCP/53, except for external secondaries. Even NIST can get it wrong. -danny -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: [EMAIL PROTECTED] ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: [secdir] secdir review of draft-ietf-dnsop-reflectors-are-evil-04.txt
On Tue, 2 Oct 2007 01:48:36 -0600 Danny McPherson [EMAIL PROTECTED] wrote: Again, any pointers empirical data along these lines would be appreciated. I said this awhile back: http://www.merit.edu/mail.archives/nanog/msg02196.html As a datapoint I ran some tests against a reasonably diverse and sizeable TLD zone I work with in another forum. I queried the name servers listed in the parent to see if I could successfuly query them for their corresponding domain name they are configured for using TCP. Out of about 9,300 unique name servers I failed to receive any answer from about 1700 of them. That is a bit more than an 18% failure rate. I think I overcompensated as I later found what looked like BIND 8 servers being unresponsive for multiple TCP queries in queue. I let the numbers stay, since the percentage of those servers were fairly small and, well, they were BIND 8 and probably have other problems anyway. :) John ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: [secdir] secdir review of draft-ietf-dnsop-reflectors-are-evil-04.txt
From: Danny McPherson [EMAIL PROTECTED] where's the authoritative source for who owns what prefixes This, one could imagining putting together. and who's permitted to originate/transit what prefixes? This is a whole different kettle of bits. For one thing, there's no guaranteed-uniquely-definitive answer: entity A might have one idea of how it wants packets to flow from it to a given other entity B (say, 'don't use provider X because we despise them'), but entity B might disagree. Who should win that dispute? So the whole notion of 'allowed to transit' is a potential tussle space; mind, I'm not saying there always will be this sort of issue, but it's *possible*. Second, you're talking about potentially orders of magnitude more data: for each destination, there are worldwide likely hundreds (or more) of ISP's which are likely to be viable backbone transits. (By 'backbone transit', I mean ISP's/AS's which are not even directly connected to the organization which is the source or destination of the packets; e.g. customer A is connected to ISP p which is connected to ISP q which is connected to ISP r which is connected to customer B; q is a 'backbone transit'.) With ISP's which are directly connected with a customer, one can assume that there's an implicit authorization, but what about steps past that? If one further assumes that by p connecting to q, p is transitively authorizing q (with respect to A), then that can be done with straightforward (albeit complex and expensive) security on routing. However, if you go that way, then you lose the ability for A to make assertions about which q's are not acceptable. If you add that back in as a policy knob, then you're back to potentially orders of magnitude more data. Noel ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: [secdir] secdir review of draft-ietf-dnsop-reflectors-are-evil-04.txt
On Tuesday, September 25, 2007 09:36:23 AM +1000 Mark Andrews [EMAIL PROTECTED] wrote: The Introduction seems a bit defensive in stating that the DOS attacks are not due to any flaw in the design of DNS or its implementations. While the blame for the attacks lies with the attackers, some aspects of nameserver configuration, behavior, and even protocol design make the systems vulnerable to these attacks. I suggest that the defensive language be removed. No, the blame lies with the parties not implementing BCP 38. A rogue end site should not be able to inject spoofed packets. No; the blame for an attack _always_ lies with the attacker, not the victim. While I certainly wish more network providers would implement BCP 38, those who fail to do so are not to blame for the bad acts of others. FWIW, I believe the defensive language in question is neither necessary nor particularly problematic, so I take no position on whether it should be removed. Finally, I wonder whether other more fundamental techniques for addressing the problem have been explored. For instance, if DNS clients were required to perform a simple handshake before a DNS server sent a long response, fake requests would provide little amplification. For example, requests that elicit long responses could prompt a shift to TCP. The DNS already does this. The DNS is optimised so that the normal responses go via UDP and the exceptional responses via TCP. It does, but normally only responses which are too long for UDP require the use of TCP. A recursive nameserver could mitigate this type of attack by lowering the maximum response size it is willing to send via UDP, forcing the use of TCP and thus a three-way handshake for larger responses. The tricky part is that setting the threshold too low can have serious performance impact. ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: [secdir] secdir review of draft-ietf-dnsop-reflectors-are-evil-04.txt
On Oct 1, 2007, at 10:10 AM, Jeffrey Hutzelman wrote: No; the blame for an attack _always_ lies with the attacker, not the victim. While I certainly wish more network providers would implement BCP 38, those who fail to do so are not to blame for the bad acts of others. Given the reality with bots et al. today, most of the attacking systems are actually victims themselves. It does, but normally only responses which are too long for UDP require the use of TCP. A recursive nameserver could mitigate this type of attack by lowering the maximum response size it is willing to send via UDP, forcing the use of TCP and thus a three-way handshake for larger responses. The tricky part is that setting the threshold too low can have serious performance impact. Note that in real deployments just this behavior has broken things on occasion, as many firewall and other such policy application points assume things like DNS resolution will only be UDP/53 transactions. YMMV. -danny ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: [secdir] secdir review of draft-ietf-dnsop-reflectors-are-evil-04.txt
On Tuesday, September 25, 2007 09:36:23 AM +1000 Mark Andrews [EMAIL PROTECTED] wrote: The Introduction seems a bit defensive in stating that the DOS attacks are not due to any flaw in the design of DNS or its implementations. While the blame for the attacks lies with the attackers, some aspects of nameserver configuration, behavior, and even protocol design make the systems vulnerable to these attacks. I suggest that the defensive language be removed. No, the blame lies with the parties not implementing BCP 38. A rogue end site should not be able to inject spoofed packets. No; the blame for an attack _always_ lies with the attacker, not the victim. I'm not blaming the victim, I'm pointing out the contributory negligence on behalf of the ISP that is supplying the attacker bandwidth. BCP 38 is over 7 years old now. The time has past where you can blame the hardware vendors for lack of support. The blame now has to be squarely brought down on the ISP's that have failed to deploy BCP 38. A attacker should not be able to send spoofed packet and have them reach the destination. I don't care if the destination is the other side of the world or the neighbor. ISP's should be doing this anyway to protect their customers from accidental traffic. e.g. DHCP lease changes where not all of the sockets with the old address have shutdown. RFC 1918 sourced traffic. While I certainly wish more network providers would implement BCP 38, those who fail to do so are not to blame for the bad acts of others. FWIW, I believe the defensive language in question is neither necessary nor particularly problematic, so I take no position on whether it should be removed. Finally, I wonder whether other more fundamental techniques for addressing the problem have been explored. For instance, if DNS clients were required to perform a simple handshake before a DNS server sent a long response, fake requests would provide little amplification. For example, requests that elicit long responses could prompt a shift to TCP. The DNS already does this. The DNS is optimised so that the normal responses go via UDP and the exceptional responses via TCP. It does, but normally only responses which are too long for UDP require the use of TCP. A recursive nameserver could mitigate this type of attack by lowering the maximum response size it is willing to send via UDP, forcing the use of TCP and thus a three-way handshake for larger responses. The tricky part is that setting the threshold too low can have serious performance impact. ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: [EMAIL PROTECTED] ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: [secdir] secdir review of draft-ietf-dnsop-reflectors-are-evil-04.txt
It does, but normally only responses which are too long for UDP require the use of TCP. A recursive nameserver could mitigate this type of attack by lowering the maximum response size it is willing to send via UDP, forcing the use of TCP and thus a three-way handshake for larger responses. The tricky part is that setting the threshold too low can have serious performance impact. Note that in real deployments just this behavior has broken things on occasion, as many firewall and other such policy application points assume things like DNS resolution will only be UDP/53 transactions. That assumption has always been wrong. I would also dispute the many above. Most firewalls actually handle the transition to TCP perfectly fine. There are the odd few that are misconfigured. When that happens people complain because nameservers resolution fails. Either the dataset is fixed or the firewall is fixed. Mark -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: [EMAIL PROTECTED] ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: [secdir] secdir review of draft-ietf-dnsop-reflectors-are-evil-04.txt
On Monday, October 01, 2007 10:34:37 AM -0600 Danny McPherson [EMAIL PROTECTED] wrote: Note that in real deployments just this behavior has broken things on occasion, as many firewall and other such policy application points assume things like DNS resolution will only be UDP/53 transactions. Yeah; I'm getting a little tired of having our protocols redefined based on the incorrect assumptions of people who don't understand them. The DNS sometimes uses TCP, UDP flows can last more than one round trip, and ICMP unreachable messages are an essential part of IP; vendors and operators who assume otherwise should be made to fix their assumptions, instead of everyone else having to cripple their applications and networks to make the assumptions true. -- Jeff And IP fragnments exist and are useful. -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: [EMAIL PROTECTED] ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
