Re: FW: DNS TTL adherence
:: >So, if you, or the original poster, is going to move
:: ${important_resource}
:: >around ip-wise keep in mind that your ${important_thing} may have to
:: >answer to more than 1 ip address for a period much longer than your
:: tuned
:: >TTL :(
::
:: Thanks all for the responses. I do understand we may need to support the
:: old IP addresses for sometime. I was hoping someone had performed a
:: study out there to determine what a ratio maybe for us supporting an old
:: IP address (I know our traffic profile will be unique for us thus it
:: would only give us a general idea).
::
:: For example if we change ip addresses will we need to plan on 20%
:: traffic at old site on day1, 10% day2, 5%, day3, and so on...? There are
:: also issues related to proxy servers and browser caching that are
:: independent of DNS we will need to quantify to understand full risk. The
:: more data we have will drive some of our decisions.
In my not-so-scientific "studies" with changind IPs for a fairly large
volume site, I found that 90% of the people will use the new ip within an
hour of TTL expiration, 99.999% of the people within 3 days, and that
remaining .001% may take years
As someone said earlier, some parts of the 'net are just broken beyond
your control...
-igor
Re: DNS TTL adherence
On Thursday 16 Mar 2006 04:23, you wrote: > > You might consider the following paper from IMC 2003: "On the > Responsiveness of DNS-based Network Control" by Jeffrey Pang, Aditya > Akella, Anees Shaikh, Balachander Krishnamurthy, Srinivasan Seshan, > http://www.imconf.net/imc-2004/papers/p21-pang.pdf The results are greatly at odds with my experience. As they imply the problem may be specifically misconfigured ISPs DNS server, which might explain why we see less violations, if our sites aren't popular with those ISPs users. However I wouldn't trust any report where the control of the authoritative DNS itself wasn't explicitly monitored and reported. They may think they have updated the authoritative answers (and TTL), but in my experience when you find violators you often find that the authoritative DNS servers didn't all update as, or when, expected, or that earlier records were returned with a longer TTL from those servers. Certainly that was the experience of moving many sites last week. Where you can in real time check the logs and find which domains we messed up on by the traffic still arriving. Looking at the 4 long term violators for one site Hits Source IP 8 198.78.130.68 <--- ?? 1 212.95.252.16 <--- lager.netcraft.com 15 66.147.154.3 <--- IBM Almaden Research Center 5 70.42.51.10 <--- Fast Search & Transfer During this period (starting 3 days after moving a 10 minute TTL) we saw 27234 hits (okay not exactly a busy site) for that site on the correct server. So roughly 1 in a 1000 hits during days 3 to 6 went to the old web server, and this domain had the most lost hits, most of the moved domains don't show in the old server's log at all. Given I think we can exclude at least 21 out of 29 safely as being "non-human" (sorry IBM Research if you were deeply interested in proof reading), and I'm guessing have made a deliberate effort to cache stale data for their own reasons. So I can put an upper estimate on our sites of 1 in 1000 hits of interest going to the wrong site during days 3 to 6. The most popular site moved, had only two DNS violators days 3 to 6, the most notable being the same "Fast Search & Transfer" IP above. It may be that popular sites have a far worse problem by dint of exercising more caching code, but this site is far from being our most popular. And these sites were moved by reducing the TTL to a low value (10 minutes) and keeping it there for a long period of time, before we actually performed the move.
RE: DNS TTL adherence
(re-sending because I wasn't on nanog-post) > For example if we change ip addresses will we need to plan on > 20% traffic at old site on day1, 10% day2, 5%, day3, and so > on...? There are also issues related to proxy servers and > browser caching that are independent of DNS we will need to > quantify to understand full risk. The more data we have will > drive some of our decisions. You might consider the following paper from IMC 2003: "On the Responsiveness of DNS-based Network Control" by Jeffrey Pang, Aditya Akella, Anees Shaikh, Balachander Krishnamurthy, Srinivasan Seshan, http://www.imconf.net/imc-2004/papers/p21-pang.pdf It sheds some light on how widely DNS TTLs are adhered to. The CDF graphs on the 4th page suggest that you should be fairly safe after a day, though I don't see if the paper specifically states what the largest recorded violation was. Sharad.
FW: DNS TTL adherence
On Wed, 15 Mar 2006, Simon Waters wrote:
>
> > This behavior is unfortunately not unique.
>
> Alas what others peoples servers do, shouldn't be an issue for you.
Your
> problem is they can be coerced into a DoS attack, not that the data is
stale.
>actually, dos-attack-aside, the interesting thing is that lots of
people
>(original poster perhaps included) believe that TTL's are adhered to
>except in some marginal cases. I think Rodney's point is that they are
not
>adhered to anywhere near as much as we would all like to believe :(
>So, if you, or the original poster, is going to move
${important_resource}
>around ip-wise keep in mind that your ${important_thing} may have to
>answer to more than 1 ip address for a period much longer than your
tuned
>TTL :(
Thanks all for the responses. I do understand we may need to support the
old IP addresses for sometime. I was hoping someone had performed a
study out there to determine what a ratio maybe for us supporting an old
IP address (I know our traffic profile will be unique for us thus it
would only give us a general idea).
For example if we change ip addresses will we need to plan on 20%
traffic at old site on day1, 10% day2, 5%, day3, and so on...? There are
also issues related to proxy servers and browser caching that are
independent of DNS we will need to quantify to understand full risk. The
more data we have will drive some of our decisions.
Thanks again,
Steve
Re: DNS TTL adherence
On Wed, 15 Mar 2006, Simon Waters wrote:
>
> > This behavior is unfortunately not unique.
>
> Alas what others peoples servers do, shouldn't be an issue for you. Your
> problem is they can be coerced into a DoS attack, not that the data is stale.
actually, dos-attack-aside, the interesting thing is that lots of people
(original poster perhaps included) believe that TTL's are adhered to
except in some marginal cases. I think Rodney's point is that they are not
adhered to anywhere near as much as we would all like to believe :(
So, if you, or the original poster, is going to move ${important_resource}
around ip-wise keep in mind that your ${important_thing} may have to
answer to more than 1 ip address for a period much longer than your tuned
TTL :(
Re: DNS TTL adherence
On Wed, 15 Mar 2006, Simon Waters wrote: This behavior is unfortunately not unique. Alas what others peoples servers do, shouldn't be an issue for you. Your problem is they can be coerced into a DoS attack, not that the data is stale. Tell that to a customer when you've moved some resource of theirs to a different IP...having been careful to decrease the TTL well in advance. Some parts of the internet are simply broken and beyond your control. -- Jon Lewis | I route Senior Network Engineer | therefore you are Atlantic Net| _ http://www.lewis.org/~jlewis/pgp for PGP public key_
Re: DNS TTL adherence
On Wednesday 15 Mar 2006 14:16, you wrote: > > Let me help you become aware, then... :) > Some people don't believe it is a bug, and therefor don't see that > anything needs "fixing". Oh the one shown is a bug, and needs fixing. > Feel free to, for example, send 2 consecutive queries for a record > that has a short (<10,000 second TTL) to 212.23.11.206. Safecom http response, busybox on telnet, some sort of embedded Linux device. Safecom sell routers... Of course can't tell if the broken DNS behaviour is the device, or possibly it is proxying upstream DNS servers. > This behavior is unfortunately not unique. Alas what others peoples servers do, shouldn't be an issue for you. Your problem is they can be coerced into a DoS attack, not that the data is stale.
Re: DNS TTL adherence
On Mar 15, 2006, at 1:56 AM, Simon Waters wrote: In answer to the original question, I'm not aware of any DNS servers that don't expire data at the end of the TTL period correctly. Failing to expire such data would be a good way of breaking things, and people would just not use such broken software. Let me help you become aware, then... I'm not sure why the OP thinks someone would research such a bug in detail, my experience is they would just fix it. Some people don't believe it is a bug, and therefor don't see that anything needs "fixing". Feel free to, for example, send 2 consecutive queries for a record that has a short (<10,000 second TTL) to 212.23.11.206. This is one of the over 100,000 random open recursive servers that have been party to some of the recursive DNS server amplification DDoS attacks over the last few weeks... and this behavior exists in a number of them. If you can't think of a record to query for that has a short enough TTL, I've created a wildcard entry of: *.example.centergate.com so that you can test this repeatedly without having to wait for the overridden TTL to expire. Just use a different random wildcard record each time (remembering to send 2 consecutive identical queries to see the misbehavior). $ dig @212.23.11.206 jhgfd.example.centergate.com a This behavior is unfortunately not unique. /rlj
Re: DNS TTL adherence
On Wednesday 15 Mar 2006 02:32, Joe Maimon wrote: > > And the dnscache resolver cache service in win2k and up. > > http://support.microsoft.com/kb/318803/en-us > http://support.microsoft.com/kb/245437/EN-US/ Both these article say the DNS TTL is honoured by the cache. Microsoft may have done some horrid things with DNS over the years, but returning stale data just breaks things. > If you are expecting hot cutovers to anything by utilizing DNS, sure > seems that you need to expect to support traffic to the values of the > old records for some time. Nope. You can bin traffic as soon as the TTL for it expires. Everything else is broken, and experience here is that it is either spam/zombie generated or googlebots . > And if you are expecting very long TTL's to give you extra insurance for > outages and what-nots, expect spotty effectiveness. Agreed, there is no requirement to cache records for the whole of the advertised TTL (or -ve TTL). In answer to the original question, I'm not aware of any DNS servers that don't expire data at the end of the TTL period correctly. Failing to expire such data would be a good way of breaking things, and people would just not use such broken software. I'm not sure why the OP thinks someone would research such a bug in detail, my experience is they would just fix it.
Re: DNS TTL adherence
[EMAIL PROTECTED] wrote: Although you asked for DNS servers - it helps to remember that no matter what the servers and resolvers do - IE will bring that behaviour to naught in many cases http://support.microsoft.com/default.aspx?scid=KB;en-us;263558 */"Thurman, Steven" <[EMAIL PROTECTED]>/* wrote: Does anyone know if there is a research paper or statistics related to what percentage of DNS servers do not adhere to advertised TTL ’ s? I am looking for some verifiable research on this topic if it is available. Thanks, Steve And the dnscache resolver cache service in win2k and up. http://support.microsoft.com/kb/318803/en-us http://support.microsoft.com/kb/245437/EN-US/ If you are expecting hot cutovers to anything by utilizing DNS, sure seems that you need to expect to support traffic to the values of the old records for some time. And if you are expecting very long TTL's to give you extra insurance for outages and what-nots, expect spotty effectiveness.
Re: DNS TTL adherence
Title: DNS TTL adherence Although you asked for DNS servers - it helps to remember that no matter what the servers and resolvers do - IE will bring that behaviour to naught in many caseshttp://support.microsoft.com/default.aspx?scid=KB;en-us;263558"Thurman, Steven" <[EMAIL PROTECTED]> wrote: Does anyone know if there is a research paper or statistics related to what percentage of DNS servers do not adhere to advertised TTLâs? I am looking for some verifiable research on this topic if it is available. Thanks, Steve
DNS TTL adherence
Title: DNS TTL adherence Does anyone know if there is a research paper or statistics related to what percentage of DNS servers do not adhere to advertised TTL’s? I am looking for some verifiable research on this topic if it is available. Thanks, Steve
