Re: redundancy via DNS
Maybe the Linux Open Router project could be helpfull. It is in a very early stage but AFAIK it aims at similar tasks. http://www.open-router.org/about.html Cheers, Marcel On 17 Jun 2001, at 14:50, :yegon wrote: we have several servers colocated with several ISP's i am trying to sort out some configuration that would ensure good uptime for customers i want to place the html documents of every customer on two separate servers connected to separate ISP's the dns servers will point to one server and the second one will be just a backup, in case the main server goes down we just change the DNS and point the affected domains to the backup server. when the main server is back up the dns changes back to normal and now my questions: 1. what should the times in zone files be set to to enable the dns change to be propagated very quickly, say 5 minutes max. is it possible/wise to use TTL=0 2. if a domain has 2 name servers set during registration, are both of these servers used for lookups? Or is it so that just the primary is querried if it works, and the secondary is querried only if the primary is not responding? 3. is this whole idea worth consideration anyway or should I forget it? thanks for answers Martin Dragun -- __ .´ `. : :' ! Enjoy `. `´ Debian/GNU Linux `- -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: redundancy via DNS
On Wed, Jun 20, 2001 at 10:49:24AM +0200, Marcel Hicking wrote: Maybe the Linux Open Router project could be helpfull. It is in a very early stage but AFAIK it aims at similar tasks. http://www.open-router.org/about.html linux virtual server (LVS) is a good linux-based load-balancer (layer 4 switch, not DNS round-robin). i've used LVS for large squid proxy-cache arrays and am about to use it for a large array of web servers. it's remarkably easy to set up, and does an excellent job. highly recommended. more details on LVS can be found at: http://www.linuxvirtualserver.org/ LVS supports load-balancing of servers at different physical locations, but it's extremely difficult to eliminate all single-points-of-failure - most of the work is going to be in *designing* your redundant network, rather than implementing it. e.g. as a starting point, think about DNS round-robin with A records pointing to two or more LVS load-balancer boxes at different locations, using Direct or Tunnel methods to get to the individual real-servers. you'd need several secondary NS machines, low TTLs on the DNS records (which won't defeat all caching - some admins ignore TTL), and tools to change the DNS records if one of the network paths went down...and then push the changes out to the secondary NS machines. this is, of course, a single-point-of-failure in itselfif your primary NS goes down or is unreachable for any reason then everything is affected. so you'd need a second one located somewhere else which can take over if necessary...complicating the setup even further. don't be discouraged by these comments, load-balancing (whether with LVS or another tool) is still very useful but it's not a magic solution to all outages. craig -- craig sanders [EMAIL PROTECTED] Fabricati Diem, PVNC. -- motto of the Ankh-Morpork City Watch -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: redundancy via DNS
Maybe the Linux Open Router project could be helpfull. It is in a very early stage but AFAIK it aims at similar tasks. http://www.open-router.org/about.html Cheers, Marcel On 17 Jun 2001, at 14:50, :yegon wrote: we have several servers colocated with several ISP's i am trying to sort out some configuration that would ensure good uptime for customers i want to place the html documents of every customer on two separate servers connected to separate ISP's the dns servers will point to one server and the second one will be just a backup, in case the main server goes down we just change the DNS and point the affected domains to the backup server. when the main server is back up the dns changes back to normal and now my questions: 1. what should the times in zone files be set to to enable the dns change to be propagated very quickly, say 5 minutes max. is it possible/wise to use TTL=0 2. if a domain has 2 name servers set during registration, are both of these servers used for lookups? Or is it so that just the primary is querried if it works, and the secondary is querried only if the primary is not responding? 3. is this whole idea worth consideration anyway or should I forget it? thanks for answers Martin Dragun -- __ .´ `. : :' ! Enjoy `. `´ Debian/GNU Linux `-
Re: redundancy via DNS
Won't work unless your TTL is set extremely low, and even then other DNS admins can override that on their servers and cache the down site. Remember, not everyone comes back to you for an authoritative answer for every lookup. Using DNS for load-balancing is NOT a good idea, ever. Or RARELY is is useful, and it's NOT good for distaster-recovery/redundancy switching. On Tue, Jun 19, 2001 at 10:29:33AM -0400, Fraser Campbell wrote: :yegon [EMAIL PROTECTED] writes: we have several servers colocated with several ISP's i am trying to sort out some configuration that would ensure good uptime for customers We're helping a customer with a similar situation. They have multiple incoming Internet connections. What we plan to do: - Have a DNS server for each Internet connection - Servers are replicated/available via every connection - Each DNS server gives out IPs only within it's subnet This way if one of the connections go down, that DNS server becomes available and those IPs stop being handed out ... effectively removing those IPs from your DNS rotation and automatically failing over to the remaining connections. This also provides a load balancing effect. Fraser -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED] -- Nate Duehr [EMAIL PROTECTED] GPG Key fingerprint = DCAF 2B9D CC9B 96FA 7A6D AAF4 2D61 77C5 7ECE C1D2 Public Key available upon request, or at wwwkeys.pgp.net and others.
Re: redundancy via DNS
On Wed, Jun 20, 2001 at 10:49:24AM +0200, Marcel Hicking wrote: Maybe the Linux Open Router project could be helpfull. It is in a very early stage but AFAIK it aims at similar tasks. http://www.open-router.org/about.html linux virtual server (LVS) is a good linux-based load-balancer (layer 4 switch, not DNS round-robin). i've used LVS for large squid proxy-cache arrays and am about to use it for a large array of web servers. it's remarkably easy to set up, and does an excellent job. highly recommended. more details on LVS can be found at: http://www.linuxvirtualserver.org/ LVS supports load-balancing of servers at different physical locations, but it's extremely difficult to eliminate all single-points-of-failure - most of the work is going to be in *designing* your redundant network, rather than implementing it. e.g. as a starting point, think about DNS round-robin with A records pointing to two or more LVS load-balancer boxes at different locations, using Direct or Tunnel methods to get to the individual real-servers. you'd need several secondary NS machines, low TTLs on the DNS records (which won't defeat all caching - some admins ignore TTL), and tools to change the DNS records if one of the network paths went down...and then push the changes out to the secondary NS machines. this is, of course, a single-point-of-failure in itselfif your primary NS goes down or is unreachable for any reason then everything is affected. so you'd need a second one located somewhere else which can take over if necessary...complicating the setup even further. don't be discouraged by these comments, load-balancing (whether with LVS or another tool) is still very useful but it's not a magic solution to all outages. craig -- craig sanders [EMAIL PROTECTED] Fabricati Diem, PVNC. -- motto of the Ankh-Morpork City Watch
Re: redundancy via DNS
:yegon [EMAIL PROTECTED] writes: we have several servers colocated with several ISP's i am trying to sort out some configuration that would ensure good uptime for customers We're helping a customer with a similar situation. They have multiple incoming Internet connections. What we plan to do: - Have a DNS server for each Internet connection - Servers are replicated/available via every connection - Each DNS server gives out IPs only within it's subnet This way if one of the connections go down, that DNS server becomes available and those IPs stop being handed out ... effectively removing those IPs from your DNS rotation and automatically failing over to the remaining connections. This also provides a load balancing effect. Fraser -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: redundancy via DNS
Fraser Campbell wrote: :yegon [EMAIL PROTECTED] writes: we have several servers colocated with several ISP's i am trying to sort out some configuration that would ensure good uptime for customers We're helping a customer with a similar situation. They have multiple incoming Internet connections. What we plan to do: - Have a DNS server for each Internet connection - Servers are replicated/available via every connection - Each DNS server gives out IPs only within it's subnet This way if one of the connections go down, that DNS server becomes available and those IPs stop being handed out ... effectively removing those IPs from your DNS rotation and automatically failing over to the remaining connections. This also provides a load balancing effect. Fraser -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED] Do you then have the root servers pointing to all of the DNS servers? Have you modified the cache/ttl times to compensate for outages and rollovers? My understanding of the way DNS works, is that when a query hits the root servers, they throw back any authoritative nameserver(s). The requesting client then queries one nameserver. If it is unavailable, it doesn't then query the other one(s). It times out. There is, to my knowledge, no way to specify which DNS server the requesting client will query. which would lead to clients attempting to query the server on the downed link, and timing out. Or am I wrong? Isn't the kind of situation that bgp or other dynamic routing protocols were designed for? ~duane -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: redundancy via DNS
Hi, I don't quite understand one bit of your statement... This way if one of the connections go down, that DNS server becomes available and those IPs stop being handed out ... effectively removing those IPs from your DNS rotation and automatically failing over to the remaining connections. This also provides a load balancing effect. I can understand how DNS rotation provides rudimentary load balancing, but how does it fail over? The downed DNS server's IPs (because the ISP's link has servered to it) cannot be transferred over to the other links. Fail over would mean that somehow the dead DNS server's job is taken over. To do that with your configuration, you'd need to change the domain name's DNS entries to either remove the dead DNS server, or to change it's IP. If people do a DNS lookup, and you have 4 connections, then there is a 1 in 4 chance the DNS lookup may fail. Not all clients will try all the other DNS servers before declaring the domain name unresolvable. I'm not picking holes in the system, I'm also trying to come up with a good solution for this. The solutions we use now are similar (main difference is we have the servers physically located in different places, and use some dedicated hardware solutions) so if there was some way to overcome the above problems, that would be great. Maybe someone else on the list has already found a way to solve them? Sincerely, Jason - Original Message - From: Fraser Campbell [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Tuesday, June 19, 2001 10:29 PM Subject: Re: redundancy via DNS :yegon [EMAIL PROTECTED] writes: we have several servers colocated with several ISP's i am trying to sort out some configuration that would ensure good uptime for customers We're helping a customer with a similar situation. They have multiple incoming Internet connections. What we plan to do: - Have a DNS server for each Internet connection - Servers are replicated/available via every connection - Each DNS server gives out IPs only within it's subnet This way if one of the connections go down, that DNS server becomes available and those IPs stop being handed out ... effectively removing those IPs from your DNS rotation and automatically failing over to the remaining connections. This also provides a load balancing effect. Fraser -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED] http://www.zentek-international.com/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: redundancy via DNS
:yegon [EMAIL PROTECTED] writes: we have several servers colocated with several ISP's i am trying to sort out some configuration that would ensure good uptime for customers We're helping a customer with a similar situation. They have multiple incoming Internet connections. What we plan to do: - Have a DNS server for each Internet connection - Servers are replicated/available via every connection - Each DNS server gives out IPs only within it's subnet This way if one of the connections go down, that DNS server becomes available and those IPs stop being handed out ... effectively removing those IPs from your DNS rotation and automatically failing over to the remaining connections. This also provides a load balancing effect. Fraser
Re: redundancy via DNS
Fraser Campbell wrote: :yegon [EMAIL PROTECTED] writes: we have several servers colocated with several ISP's i am trying to sort out some configuration that would ensure good uptime for customers We're helping a customer with a similar situation. They have multiple incoming Internet connections. What we plan to do: - Have a DNS server for each Internet connection - Servers are replicated/available via every connection - Each DNS server gives out IPs only within it's subnet This way if one of the connections go down, that DNS server becomes available and those IPs stop being handed out ... effectively removing those IPs from your DNS rotation and automatically failing over to the remaining connections. This also provides a load balancing effect. Fraser -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED] Do you then have the root servers pointing to all of the DNS servers? Have you modified the cache/ttl times to compensate for outages and rollovers? My understanding of the way DNS works, is that when a query hits the root servers, they throw back any authoritative nameserver(s). The requesting client then queries one nameserver. If it is unavailable, it doesn't then query the other one(s). It times out. There is, to my knowledge, no way to specify which DNS server the requesting client will query. which would lead to clients attempting to query the server on the downed link, and timing out. Or am I wrong? Isn't the kind of situation that bgp or other dynamic routing protocols were designed for? ~duane
Re: redundancy via DNS
Hi, I don't quite understand one bit of your statement... This way if one of the connections go down, that DNS server becomes available and those IPs stop being handed out ... effectively removing those IPs from your DNS rotation and automatically failing over to the remaining connections. This also provides a load balancing effect. I can understand how DNS rotation provides rudimentary load balancing, but how does it fail over? The downed DNS server's IPs (because the ISP's link has servered to it) cannot be transferred over to the other links. Fail over would mean that somehow the dead DNS server's job is taken over. To do that with your configuration, you'd need to change the domain name's DNS entries to either remove the dead DNS server, or to change it's IP. If people do a DNS lookup, and you have 4 connections, then there is a 1 in 4 chance the DNS lookup may fail. Not all clients will try all the other DNS servers before declaring the domain name unresolvable. I'm not picking holes in the system, I'm also trying to come up with a good solution for this. The solutions we use now are similar (main difference is we have the servers physically located in different places, and use some dedicated hardware solutions) so if there was some way to overcome the above problems, that would be great. Maybe someone else on the list has already found a way to solve them? Sincerely, Jason - Original Message - From: Fraser Campbell [EMAIL PROTECTED] To: debian-isp@lists.debian.org Sent: Tuesday, June 19, 2001 10:29 PM Subject: Re: redundancy via DNS :yegon [EMAIL PROTECTED] writes: we have several servers colocated with several ISP's i am trying to sort out some configuration that would ensure good uptime for customers We're helping a customer with a similar situation. They have multiple incoming Internet connections. What we plan to do: - Have a DNS server for each Internet connection - Servers are replicated/available via every connection - Each DNS server gives out IPs only within it's subnet This way if one of the connections go down, that DNS server becomes available and those IPs stop being handed out ... effectively removing those IPs from your DNS rotation and automatically failing over to the remaining connections. This also provides a load balancing effect. Fraser -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED] http://www.zentek-international.com/
redundancy via DNS
we have several servers colocated with several ISP's i am trying to sort out some configuration that would ensure good uptime for customers i want to place the html documents of every customer on two separate servers connected to separate ISP's the dns servers will point to one server and the second one will be just a backup, in case the main server goes down we just change the DNS and point the affected domains to the backup server. when the main server is back up the dns changes back to normal and now my questions: 1. what should the times in zone files be set to to enable the dns change to be propagated very quickly, say 5 minutes max. is it possible/wise to use TTL=0 2. if a domain has 2 name servers set during registration, are both of these servers used for lookups? Or is it so that just the primary is querried if it works, and the secondary is querried only if the primary is not responding? 3. is this whole idea worth consideration anyway or should I forget it? thanks for answers Martin Dragun -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
redundancy via DNS
we have several servers colocated with several ISP's i am trying to sort out some configuration that would ensure good uptime for customers i want to place the html documents of every customer on two separate servers connected to separate ISP's the dns servers will point to one server and the second one will be just a backup, in case the main server goes down we just change the DNS and point the affected domains to the backup server. when the main server is back up the dns changes back to normal and now my questions: 1. what should the times in zone files be set to to enable the dns change to be propagated very quickly, say 5 minutes max. is it possible/wise to use TTL=0 2. if a domain has 2 name servers set during registration, are both of these servers used for lookups? Or is it so that just the primary is querried if it works, and the secondary is querried only if the primary is not responding? 3. is this whole idea worth consideration anyway or should I forget it? thanks for answers Martin Dragun
Re: redundancy via DNS
It would depend on how popular the sites hosted on the servers were. If you set a the times to be too low, say 1 minute, then every time someone looks up the DNS records, then BLAM... your dns servers are hit because things aren't cached anywhere. So I would use something like an hour (we use this). An hour is reasonable unless you need total 100% uptime. If you needed 100% uptime, you wouldn't just rely on DNS for this anyway. You'd need something more reliable like IP takeover, dedicated hardware solutions, etc. Depends greatly on what your budget is. The dns servers are queried randomly, so say you have 4 DNS servers listed, then each 4, in theory, should get approximately the same amount of traffic. If one of them goes down, then the client SHOULD try the next available dns server. You'd also want to colocate somewhere WAY out of the same network neighbourhood. Interestingly a few of our clients from the USA do this. Since we are located in Hong Kong, our networks are totally seperate from anything you use in the USA. So when these california blackouts (is that the right term?) hit them, they were fine. If you really want to keep everything in the USA, try and find totally seperate networks... and i mean totally (if you want to be real safe). UUnet and the big boys in the USA tend to have a few core NOCs (even if they tell you everything is distributed and safe, blah blah blah), and if any one of them is hit with a blackout, earthquake, etc. then the whole network is affected. This happened to UUnet in one of the countries in Asia (won't mention which country it just in case UUnet is watching this) once... something happened to one of their core international-link routers, and many countries were affected, including the one our client was in. UUnet may deny it but we... the people who actually use them... know the true story ;-) Anyway, if you're really into reliability, you might want to colocate in hong kong. Can't get much more diversified network-wise than that. Email me back if you're interested in working something out. Otherwise, consider the above carefully about the US networks. Sincerely, Jason - Original Message - From: :yegon [EMAIL PROTECTED] To: debian-isp@lists.debian.org Sent: Sunday, June 17, 2001 8:50 PM Subject: redundancy via DNS we have several servers colocated with several ISP's i am trying to sort out some configuration that would ensure good uptime for customers i want to place the html documents of every customer on two separate servers connected to separate ISP's the dns servers will point to one server and the second one will be just a backup, in case the main server goes down we just change the DNS and point the affected domains to the backup server. when the main server is back up the dns changes back to normal and now my questions: 1. what should the times in zone files be set to to enable the dns change to be propagated very quickly, say 5 minutes max. is it possible/wise to use TTL=0 2. if a domain has 2 name servers set during registration, are both of these servers used for lookups? Or is it so that just the primary is querried if it works, and the secondary is querried only if the primary is not responding? 3. is this whole idea worth consideration anyway or should I forget it? thanks for answers Martin Dragun -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: redundancy via DNS
There are a number of very effect appliance style solutions to doing this. Please have a look at RadWare (WSD) and F5 Networks (3DNS); I have had great success with both companies. The bonus is that these solutions can automaticly determine if a server is up. Ken Seefried, CISSP :yegon writes: we have several servers colocated with several ISP's i am trying to sort out some configuration that would ensure good uptime for customers i want to place the html documents of every customer on two separate servers connected to separate ISP's the dns servers will point to one server and the second one will be just a backup, in case the main server goes down we just change the DNS and point the affected domains to the backup server. when the main server is back up the dns changes back to normal and now my questions: 1. what should the times in zone files be set to to enable the dns change to be propagated very quickly, say 5 minutes max. is it possible/wise to use TTL=0 2. if a domain has 2 name servers set during registration, are both of these servers used for lookups? Or is it so that just the primary is querried if it works, and the secondary is querried only if the primary is not responding? 3. is this whole idea worth consideration anyway or should I forget it? thanks for answers Martin Dragun -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: redundancy via DNS
I mentioned hardware solutions in my email... however, the cost of these hardware appliances is pretty high. In theory, you can do the same thing with a properly configured linux server at less than half the price. Of course... the money is in the configuration ;-) Sincerely, Jason - Original Message - From: Ken Seefried [EMAIL PROTECTED] To: :yegon [EMAIL PROTECTED] Cc: debian-isp@lists.debian.org Sent: Sunday, June 17, 2001 10:33 PM Subject: Re: redundancy via DNS There are a number of very effect appliance style solutions to doing this. Please have a look at RadWare (WSD) and F5 Networks (3DNS); I have had great success with both companies. The bonus is that these solutions can automaticly determine if a server is up. Ken Seefried, CISSP :yegon writes: we have several servers colocated with several ISP's i am trying to sort out some configuration that would ensure good uptime for customers i want to place the html documents of every customer on two separate servers connected to separate ISP's the dns servers will point to one server and the second one will be just a backup, in case the main server goes down we just change the DNS and point the affected domains to the backup server. when the main server is back up the dns changes back to normal and now my questions: 1. what should the times in zone files be set to to enable the dns change to be propagated very quickly, say 5 minutes max. is it possible/wise to use TTL=0 2. if a domain has 2 name servers set during registration, are both of these servers used for lookups? Or is it so that just the primary is querried if it works, and the secondary is querried only if the primary is not responding? 3. is this whole idea worth consideration anyway or should I forget it? thanks for answers Martin Dragun -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
