Re: [DNSOP] Fwd: New Version Notification for draft-livingood-dnsop-negative-trust-anchors-01.txt
Over on the BIND-Users list there is currently a discussion of fema.net (one the Federal Emergency Management Agency domains) being DNSSEC borked (https://lists.isc.org/pipermail/bind-users/2014-October/094142.html) This is an example of the sort of issues that an NTA could address -- I'd like to note that currently neither Google Public DNS (8.8.8.8) nor Comcast (75.75.75.75) have put in an NTA for it, but if it were fema.gov, and this were during some sort of national disaster in the US, things might be different... W On Mon, Oct 27, 2014 at 10:04 AM, Dan York y...@isoc.org wrote: Many others have made the points I would have made about the operational value of NTAs so I won't repeat those... but I want to just say that I think Paul Ebersman nails it here: On Oct 26, 2014, at 12:09 PM, Paul Ebersman list-dn...@dragon.net wrote: I see NTA as a tool that we should try to never use but which is invaluable when we do need it. Exactly! Hopefully everything just works 99% of the time... but in the event something doesn't work right the operators have a narrow scalpel tool in their toolbox that they can pull out rather than resorting to more drastic measures such as, for example, disabling all DNSSEC validation. Ideally NTAs never get used and as DNSSEC deployment moves along and DNS operators get increasingly familiar with the operational practices required of DNSSEC then the need for NTAs will eventually fade away. My agenda in pushing this draft is not to advocate wide spread use but to guarantee that all of my vendors have an RFC to code against so that I have consistent behavior and plenty of server choices for server diversity. Yes! If we have an operational need to have a way to generate DNSSEC validation exceptions, let's please have *one* way that we can collectively agree upon rather than having every different operator come up with some custom mechanism that works only for them. This will make the overall deployment that much easier if the one method is spread across multiple software vendors and systems. My 2 cents, Dan P.S. Nice quote, Warren! -- Dan York Senior Content Strategist, Internet Society y...@isoc.org +1-802-735-1624 Jabber: y...@jabber.isoc.org Skype: danyork http://twitter.com/danyork http://www.internetsociety.org/deploy360/ ___ DNSOP mailing list DNSOP@ietf.org https://www.ietf.org/mailman/listinfo/dnsop -- I don't think the execution is relevant when it was obviously a bad idea in the first place. This is like putting rabid weasels in your pants, and later expressing regret at having chosen those particular rabid weasels and that pair of pants. ---maf ___ DNSOP mailing list DNSOP@ietf.org https://www.ietf.org/mailman/listinfo/dnsop
Re: [DNSOP] Fwd: New Version Notification for draft-livingood-dnsop-negative-trust-anchors-01.txt
Warren Kumari wrote: Over on the BIND-Users list there is currently a discussion of fema.net (one the Federal Emergency Management Agency domains) being DNSSEC borked (https://lists.isc.org/pipermail/bind-users/2014-October/094142.html) This is an example of the sort of issues that an NTA could address -- I'd like to note that currently neither Google Public DNS (8.8.8.8) nor Comcast (75.75.75.75) have put in an NTA for it, but if it were fema.gov, and this were during some sort of national disaster in the US, things might be different... If an authoritative domain (e.g. irs.gov) screwed up its delegation NS records so it effectively went dark or made some similar sort of authoritative DNS or nameserver error, we wouldn't expect the recursive, caching side to resolve those sorts of errors. The domain's DNS would simply be unavailable until they resolved their problem. I'm not sure I understand why DNSSEC is somehow different. If a domain owner chooses to sign its authoritative zones and at some point screws up either their signing or their chain of trust, they should reasonably expect their DNS to go dark to a certain percentage of the world. (I believe in the United States currently, that's around a quarter of the population, at least according to APNIC Labs numbers. That tends to be the part of the world I watch most closely.) I do understand the need ISPs had to manage customer perceptions, especially for the earliest adopters like Comcast. Support calls cost money and in some instances, an irate customer may choose to switch providers. That likely persists to some extent today, but with Google on board the pressure is, at least, less than it was before. And as those implementing DNSSEC validation continue to increase, that pressure will continue to drop. Outside of the ISP early adopter use case, though, I'm not sure I understand the need for NTAs. We've had DNSSEC validation of Internet queries enabled for our enterprise since 2011. On the enterprise side, we simply explain the problem and that it's on the domain provider's end and that it's their responsibility to fix it. Until they do we won't be able to resolve their domain. We've never viewed it as our responsibility to try to fix problems on the authoritative side of DNS for domains we don't own or manage. Truthfully, we don't really encounter as many issues as we once did. Given the limited nature of the use case, I'm not convinced it matters if there's a single specification for implementing it or not. I'm not really opposed to the idea either, nor do I have any issues with the draft. But after several years of experience without NTAs from a non-ISP perspective, I do know it hasn't been a burden or major issue. Scott ___ DNSOP mailing list DNSOP@ietf.org https://www.ietf.org/mailman/listinfo/dnsop
Re: [DNSOP] Fwd: New Version Notification for draft-livingood-dnsop-negative-trust-anchors-01.txt
In message cahw9_ik5yvsc07co_cjz2go2oquyqbjjverw6ceexbuj2cv...@mail.gmail.com, Warren Kumari writes: Over on the BIND-Users list there is currently a discussion of fema.net (one the Federal Emergency Management Agency domains) being DNSSEC borked (https://lists.isc.org/pipermail/bind-users/2014-October/094142.html) This is an example of the sort of issues that an NTA could address -- I'd like to note that currently neither Google Public DNS (8.8.8.8) nor Comcast (75.75.75.75) have put in an NTA for it, but if it were fema.gov, and this were during some sort of national disaster in the US, things might be different... W It is also a example of why valid whois data is required. whois fema.net - contact details whois fema.gov - a farce Mark On Mon, Oct 27, 2014 at 10:04 AM, Dan York y...@isoc.org wrote: Many others have made the points I would have made about the operational value of NTAs so I won't repeat those... but I want to just say that I think Paul Ebersman nails it here: On Oct 26, 2014, at 12:09 PM, Paul Ebersman list-dn...@dragon.net wrote: I see NTA as a tool that we should try to never use but which is invaluable when we do need it. Exactly! Hopefully everything just works 99% of the time... but in the event something doesn't work right the operators have a narrow scalpel tool in their toolbox that they can pull out rather than resorting to more drastic measures such as, for example, disabling all DNSSEC validation. Ideally NTAs never get used and as DNSSEC deployment moves along and DNS operators get increasingly familiar with the operational practices required of DNSSEC then the need for NTAs will eventually fade away. My agenda in pushing this draft is not to advocate wide spread use but to guarantee that all of my vendors have an RFC to code against so that I have consistent behavior and plenty of server choices for server diversity. Yes! If we have an operational need to have a way to generate DNSSEC validation exceptions, let's please have *one* way that we can collectively agree upon rather than having every different operator come up with some custom mechanism that works only for them. This will make the overall deployment that much easier if the one method is spread across multiple software vendors and systems. My 2 cents, Dan P.S. Nice quote, Warren! -- Dan York Senior Content Strategist, Internet Society y...@isoc.org +1-802-735-1624 Jabber: y...@jabber.isoc.org Skype: danyork http://twitter.com/danyork http://www.internetsociety.org/deploy360/ ___ DNSOP mailing list DNSOP@ietf.org https://www.ietf.org/mailman/listinfo/dnsop -- I don't think the execution is relevant when it was obviously a bad idea in the first place. This is like putting rabid weasels in your pants, and later expressing regret at having chosen those particular rabid weasels and that pair of pants. ---maf ___ DNSOP mailing list DNSOP@ietf.org https://www.ietf.org/mailman/listinfo/dnsop -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: ma...@isc.org ___ DNSOP mailing list DNSOP@ietf.org https://www.ietf.org/mailman/listinfo/dnsop
Re: [DNSOP] Fwd: New Version Notification for draft-livingood-dnsop-negative-trust-anchors-01.txt
Many others have made the points I would have made about the operational value of NTAs so I won't repeat those... but I want to just say that I think Paul Ebersman nails it here: On Oct 26, 2014, at 12:09 PM, Paul Ebersman list-dn...@dragon.netmailto:list-dn...@dragon.net wrote: I see NTA as a tool that we should try to never use but which is invaluable when we do need it. Exactly! Hopefully everything just works 99% of the time... but in the event something doesn't work right the operators have a narrow scalpel tool in their toolbox that they can pull out rather than resorting to more drastic measures such as, for example, disabling all DNSSEC validation. Ideally NTAs never get used and as DNSSEC deployment moves along and DNS operators get increasingly familiar with the operational practices required of DNSSEC then the need for NTAs will eventually fade away. My agenda in pushing this draft is not to advocate wide spread use but to guarantee that all of my vendors have an RFC to code against so that I have consistent behavior and plenty of server choices for server diversity. Yes! If we have an operational need to have a way to generate DNSSEC validation exceptions, let's please have *one* way that we can collectively agree upon rather than having every different operator come up with some custom mechanism that works only for them. This will make the overall deployment that much easier if the one method is spread across multiple software vendors and systems. My 2 cents, Dan P.S. Nice quote, Warren! -- Dan York Senior Content Strategist, Internet Society y...@isoc.orgmailto:y...@isoc.org +1-802-735-1624 Jabber: y...@jabber.isoc.orgmailto:y...@jabber.isoc.org Skype: danyork http://twitter.com/danyork http://www.internetsociety.org/deploy360/ ___ DNSOP mailing list DNSOP@ietf.org https://www.ietf.org/mailman/listinfo/dnsop
Re: [DNSOP] Fwd: New Version Notification for draft-livingood-dnsop-negative-trust-anchors-01.txt
pwouters So I'm confused. What is the goal of this document? How does pwouters it help us? The other side of this document is that there 3 of the most popular vendors of DNS server software have this out. We as the IETF should be explaining the tradeoffs and risks of using an NTA. I certainly don't want enterprises, small service providers and govt agencies that may not have senior DNS folks just putting these into their configs with no idea of the security implications of doing so. ___ DNSOP mailing list DNSOP@ietf.org https://www.ietf.org/mailman/listinfo/dnsop
Re: [DNSOP] Fwd: New Version Notification for draft-livingood-dnsop-negative-trust-anchors-01.txt
On 10/24/14, 6:06 PM, Doug Barton do...@dougbarton.us wrote: But worse yet, in the operator error case you make such errors painless. Instead, if they are painful (as in, screw up DNSSEC and you go off line) then it leads to more people taking DNSSEC seriously, and doing it right. Of course I realize that the counter-argument is that if DNSSEC fails are too painful then people will simply not do it. But to the extent that people use that as a line of reasoning it's simply one more in a long string of excuses. I think we¹re seeing a collision of how we¹d like the world to work and how it actually works. ;-) Operators are on the front lines here; trust us on the realities of getting stuff like this deployed. While I totally get 110% agree that when someone screws up their auth records they should feel the pain - and that this pain is a signal to do it better/right the next time - the reality is that the real pain is felt by the DNS operator. Of course I am going to pains to point out who is really to blame here: http://tools.ietf.org/html/draft-livingood-dnsop-auth-dnssec-mistakes-00 Anyway, if a popular domain like google.com or facebook.com or netflix.com was to sign - and then made a mistake - the call centers of an validating operator would pretty much melt down. It wouldn¹t matter that we were not to blame, we¹d feel the pain and people would be angry at us. There¹d be pressure to Œmake the pain stop¹ (and associated costs). Oh - and these days in the U.S. we¹d probably be blamed for ³blocking access² to the site - violating net neutrality (that the facts would say otherwise wouldn¹t matter**). So - two choices - turn off DNSSEC validation for ALL domains or just the one affected (assuming you can prove out it is a mistake and not an attack). Which one is worse? The market has already answered that: Negative Trust Anchors - turn it off in as narrow a case as possible. The question for us here is whether we wish to acknowledge this reality or pretend it does not exist. ;-) The other problem is that this feature is only really useful in the DNSSEC ramp-up period. Sure, mistakes are more common now, software is immature, etc. etc. But if DNSSEC is successful, the software will get better (it already is a lot better than even a few years ago), and mistakes will be less common (both on an absolute, and on a percentage basis). But once you introduce a feature like this, you cannot remove it. We address the timeframe issue in the draft. Certainly you are correct that the feature is still there afterwards, but of course the feature already exists today so the genie is out of the bottle as they say. So can we please let the functionality of DNSSEC stand as designed, and not give people a giant trapdoor that they can trigger at the slightest sign of trouble? Otherwise, why bother? I wouldn¹t say that it is at the Œslightest¹ sign of trouble. This is intended for use when there are major problems. Turning this around: DNSSEC deployment only got as far as it did to date because of NTAs. Without NTAs, I don¹t believe it¹ll go much further. So do we want to help the world understand what it takes to push ahead with DNSSEC? ... and of course, I should point out that adding this as a knob is utterly pointless, since any reasonably competent validating resolver operator can engineer their own trapdoor. Yes - we all decided the way to do it was with a Negative Trust Anchor. That¹s what the draft documents. ;-) - Jason Livingood ** From the discussion above (scroll back up to refresh your memory). A good recent example is a claim by some anonymous poster in a Reddit thread that Comcast was blocking Tor. Within hours it went viral and was posted to major news sites, never mind that it wasn¹t true. I posted http://corporate.comcast.com/comcast-voices/setting-the-record-straight-on- tor and yet still to this day there are tweets and reposts about how we block Tor, pointing back to the original news articles. I can¹t wait for what will happen when a major domain breaks if a NTA did not exist. ___ DNSOP mailing list DNSOP@ietf.org https://www.ietf.org/mailman/listinfo/dnsop
Re: [DNSOP] Fwd: New Version Notification for draft-livingood-dnsop-negative-trust-anchors-01.txt
On Sat, Oct 25, 2014 at 3:30 PM, Paul Ebersman list-dn...@dragon.net wrote: dougb It's not just a philosophical objection, it's an operational dougb one. When DNSSEC fails for a domain there are 2 main dougb reasons. Operator error, and an actual MITM or similar attack. If dougb the operators of validating resolvers simply turn off validation dougb for domains that should be validating but are not,* it kicks the dougb door open for the exact problem that DNSSEC was designed to dougb solve. Have you actually read through the new draft? It specifically prohibits automatic installation of NTAs and says that you should have folks familiar with operating DNS servers making any decisions. That isn't painless. It means that this skips past all 1st tier and gets to senior engineers. Don't know about you but I hate getting on-call problems caused by someone else that I have no direct way to fix but that my customers beat me for. ... or that your customers move away because of... http://forums.comcast.com/t5/Basic-Internet-Connectivity-And/Re-Weather-Gov-DNS/td-p/1175625 http://www.dslreports.com/forum/remark,25180845?hilite=dns http://forums.comcast.com/t5/Basic-Internet-Connectivity-And/Comcast-DNS-can-t-won-t-find-state-gov-sites-How-do-I-report/td-p/1824503 And that's way more expensive than standard help desk... But it is a good way to make sure that this isn't a constant and knee jerk reaction but an operationally sane one. dougb The other problem is that this feature is only really useful in dougb the DNSSEC ramp-up period. Sure, mistakes are more common now, dougb software is immature, etc. etc. But if DNSSEC is successful, the dougb software will get better (it already is a lot better than even a dougb few years ago), and mistakes will be less common (both on an dougb absolute, and on a percentage basis). But once you introduce a dougb feature like this, you cannot remove it. ... but you don't have to remove it. If DNSSEC is successful, the software will get better (it already is a lot better than even a few years ago), and mistakes will be less common then there will be no need to install NTAs. Sure. It will only be temporary/fast. Like rolling out IPv6. Or getting all the broken CPEs using old dnsmasq and being open resolvers. Or getting folks to roll out BCP38. How long have we been trying to get folks to use DNSSEC? ... a long time... How far are we? not far enough... And why do we keep insisting that only by painful and expensive suffering will we do it correctly? Not a clue. This is like saying that airbags and selt-belts are a bad idea in cars -- if we didn't have them, people would learn to drive more carefully[0]. This is potentially true, but... seriously... W [0]: This inserted somewhat so that I can trot out my current favorite quote: Analogies are like bicycles: It doesn’t matter how many marbles it takes to fill the outhouse, telco execs will always be making silly metaphors — Patrik Gilmore. ___ DNSOP mailing list DNSOP@ietf.org https://www.ietf.org/mailman/listinfo/dnsop -- I don't think the execution is relevant when it was obviously a bad idea in the first place. This is like putting rabid weasels in your pants, and later expressing regret at having chosen those particular rabid weasels and that pair of pants. ---maf ___ DNSOP mailing list DNSOP@ietf.org https://www.ietf.org/mailman/listinfo/dnsop
[DNSOP] Fwd: New Version Notification for draft-livingood-dnsop-negative-trust-anchors-01.txt
Hi all, This draft has risen from the deep... It describes a technique that a number of DNS operators use to surgically / tactically deal with DNSSEC validation failures, for large-scale outages. We believe that this is needed -- simply telling customers This doesn't work though us, but does work through $non-validating-competitor because we are better simply leads to customers changing to $non-validating-competitor, or operator turning off DNSSEC for everybody. I know that there will be some philosophical objections / discussions on this... W W -- Forwarded message -- From: internet-dra...@ietf.org Date: Thu, Oct 23, 2014 at 1:06 PM Subject: New Version Notification for draft-livingood-dnsop-negative-trust-anchors-01.txt To: Ralf Weber ralf.we...@nominum.com, Jason Livingood jason_living...@cable.comcast.com, Warren Kumari war...@kumari.net, Chris Griffiths cgriffi...@gmail.com, Paul Ebersman ebersman-i...@dragon.net A new version of I-D, draft-livingood-dnsop-negative-trust-anchors-01.txt has been successfully submitted by Warren Kumari and posted to the IETF repository. Name: draft-livingood-dnsop-negative-trust-anchors Revision: 01 Title: Definition and Use of DNSSEC Negative Trust Anchors Document date: 2014-10-23 Group: Individual Submission Pages: 17 URL: http://www.ietf.org/internet-drafts/draft-livingood-dnsop-negative-trust-anchors-01.txt Status: https://datatracker.ietf.org/doc/draft-livingood-dnsop-negative-trust-anchors/ Htmlized: http://tools.ietf.org/html/draft-livingood-dnsop-negative-trust-anchors-01 Diff: http://www.ietf.org/rfcdiff?url2=draft-livingood-dnsop-negative-trust-anchors-01 Abstract: DNS Security Extensions (DNSSEC) is now entering widespread deployment. However, domain signing tools and processes are not yet as mature and reliable as those for non-DNSSEC-related domain administration tools and processes. Negative Trust Anchors (described in this document) can be used to mitigate DNSSEC validation failures. [ Editor note: This document was originally draft-livingood-negative- trust-anchors-07 - renamved at the request of the DNSOP chairs ] Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. The IETF Secretariat -- I don't think the execution is relevant when it was obviously a bad idea in the first place. This is like putting rabid weasels in your pants, and later expressing regret at having chosen those particular rabid weasels and that pair of pants. ---maf ___ DNSOP mailing list DNSOP@ietf.org https://www.ietf.org/mailman/listinfo/dnsop
Re: [DNSOP] Fwd: New Version Notification for draft-livingood-dnsop-negative-trust-anchors-01.txt
On Oct 24, 2014, at 9:01 AM, Warren Kumari war...@kumari.net wrote: This draft has risen from the deep... Like http://www.sfweekly.com/foodie/2008/03/31/dread-cherry-chip-cthulhu-rises-from-the-deep? We believe that this is needed -- simply telling customers This doesn't work though us, but does work through $non-validating-competitor because we are better simply leads to customers changing to $non-validating-competitor, or operator turning off DNSSEC for everybody. +1 Regards, -drc signature.asc Description: Message signed with OpenPGP using GPGMail ___ DNSOP mailing list DNSOP@ietf.org https://www.ietf.org/mailman/listinfo/dnsop
Re: [DNSOP] Fwd: New Version Notification for draft-livingood-dnsop-negative-trust-anchors-01.txt
On 10/24/14 12:06 PM, David Conrad wrote: On Oct 24, 2014, at 9:01 AM, Warren Kumari war...@kumari.net wrote: This draft has risen from the deep... Like http://www.sfweekly.com/foodie/2008/03/31/dread-cherry-chip-cthulhu-rises-from-the-deep? We believe that this is needed -- simply telling customers This doesn't work though us, but does work through $non-validating-competitor because we are better simply leads to customers changing to $non-validating-competitor, or operator turning off DNSSEC for everybody. +1 And since Mr. Conrad has shown fine experience using Internet Search Tools, I'm volunteering him for editing and review of this document. tim ___ DNSOP mailing list DNSOP@ietf.org https://www.ietf.org/mailman/listinfo/dnsop
Re: [DNSOP] Fwd: New Version Notification for draft-livingood-dnsop-negative-trust-anchors-01.txt
On 10/24/14 9:01 AM, Warren Kumari wrote: Hi all, This draft has risen from the deep... It describes a technique that a number of DNS operators use to surgically / tactically deal with DNSSEC validation failures, for large-scale outages. We believe that this is needed -- simply telling customers This doesn't work through us, but does work through $non-validating-competitor because we are better simply leads to customers changing to $non-validating-competitor, or operator turning off DNSSEC for everybody. Well that would be an incredibly stupid thing to tell a customer. :) Currently the operators of example.com have introduced an error into their DNS information which is causing the domain to be unresolvable. Once they have corrected that error our customers will be able to reach their site again. You can go to the following web page to view detailed information about the problem ... I'm aware of the problems that providers face when something doesn't work and it's not their fault; been there, done that. But ... I know that there will be some philosophical objections / discussions on this... It's not just a philosophical objection, it's an operational one. When DNSSEC fails for a domain there are 2 main reasons. Operator error, and an actual MITM or similar attack. If the operators of validating resolvers simply turn off validation for domains that should be validating but are not,* it kicks the door open for the exact problem that DNSSEC was designed to solve. But worse yet, in the operator error case you make such errors painless. Instead, if they are painful (as in, screw up DNSSEC and you go off line) then it leads to more people taking DNSSEC seriously, and doing it right. Of course I realize that the counter-argument is that if DNSSEC fails are too painful then people will simply not do it. But to the extent that people use that as a line of reasoning it's simply one more in a long string of excuses. The other problem is that this feature is only really useful in the DNSSEC ramp-up period. Sure, mistakes are more common now, software is immature, etc. etc. But if DNSSEC is successful, the software will get better (it already is a lot better than even a few years ago), and mistakes will be less common (both on an absolute, and on a percentage basis). But once you introduce a feature like this, you cannot remove it. So can we please let the functionality of DNSSEC stand as designed, and not give people a giant trapdoor that they can trigger at the slightest sign of trouble? Otherwise, why bother? ... and of course, I should point out that adding this as a knob is utterly pointless, since any reasonably competent validating resolver operator can engineer their own trapdoor. IMO this further cements the argument that adding this as a knob in software will only facilitate its use by people who should not be using it. Doug * I realize that the proponents of this mechanism believe that everyone who uses it will use it properly, by validating a trusted source for information about why it's failing, only leaving it enabled for a short time, etc. Can we please be honest about the fact that in the majority of cases that simply won't be true? ___ DNSOP mailing list DNSOP@ietf.org https://www.ietf.org/mailman/listinfo/dnsop
Re: [DNSOP] Fwd: New Version Notification for draft-livingood-dnsop-negative-trust-anchors-01.txt
Moin! On 25 Oct 2014, at 01:06, Doug Barton do...@dougbarton.us wrote: Currently the operators of example.com have introduced an error into their DNS information which is causing the domain to be unresolvable. Once they have corrected that error our customers will be able to reach their site again. You can go to the following web page to view detailed information about the problem ... How are you going to tell that to your customers? Who will pay for the calls to the ISPs call center? The other problem is that this feature is only really useful in the DNSSEC ramp-up period. Sure, mistakes are more common now, software is immature, etc. etc. But if DNSSEC is successful, the software will get better (it already is a lot better than even a few years ago), and mistakes will be less common (both on an absolute, and on a percentage basis). But once you introduce a feature like this, you cannot remove it. The feature already exists in the most commonly used validating resolvers (see Appendix A). Beside my tiny contribution to the draft you may notice that the other authors of the draft come from the two biggest currently operational validating resolver farms (Comcast and Google). So I would rather say it is operational reality and so far all the ISPs I talked to who are thinking about doing validation see this as a critical feature also. This draft describes that and tries to tell them to be super cautious with that feature. So long Ralf Sent from my iPhone ___ DNSOP mailing list DNSOP@ietf.org https://www.ietf.org/mailman/listinfo/dnsop
