[swinog] datacenter failover?
hey guys, we have some servers in datacenter#1: ns1, ns2, web1(mail/sql). we also have ns3 and web2 outside this web. how can we make this working? ok, we can copy the data by cron, no problem. but can i give ns3 another ip for an a record? what`s your solution? greets, sebastian ___ swinog mailing list swinog@lists.swinog.ch http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog
Re: [swinog] datacenter failover?
Hi, > we have some servers in datacenter#1: ns1, ns2, web1(mail/sql). > we also have ns3 and web2 outside this web. > > how can we make this working? > > ok, we can copy the data by cron, no problem. Not a problem for static content, but in case of webmail and a database, its not that easy anymore. Assuming, I read my email during failover to the other datacenter, the cronjob has to know, that the master of the storage resides in the backup datacenter. > > but can i give ns3 another ip for an a record? I would personally define ns1 as master and ns2/ns3 as slaves. ns2/ns3 are the A records for the various domains, but you only edit your zonefiles on ns1. kind regards Robert -- Robert Meyer r.me...@net-wizard.org ___ swinog mailing list swinog@lists.swinog.ch http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog
Re: [swinog] datacenter failover?
hi sebastian ns1 = master ns2/3 = slaves (which are public to the customers) sql stuff are solved by master-master replication setups (quite easy for mysql), web can be solved by using rsync or other stuff (dont use NFS ,-)) -steven > -Original Message- > From: swinog-boun...@lists.swinog.ch > [mailto:swinog-boun...@lists.swinog.ch] On Behalf Of . . > Sent: Tuesday, August 04, 2009 4:12 AM > To: swi...@swinog.ch > Subject: [swinog] datacenter failover? > > hey guys, > we have some servers in datacenter#1: ns1, ns2, web1(mail/sql). > we also have ns3 and web2 outside this web. > > how can we make this working? > > ok, we can copy the data by cron, no problem. > > but can i give ns3 another ip for an a record? > > what`s your solution? > > greets, > sebastian > > ___ > swinog mailing list > swinog@lists.swinog.ch > http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog > ___ swinog mailing list swinog@lists.swinog.ch http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog
Re: [swinog] datacenter failover?
two redundant Layer4 switches with VRRP would help. You can deploy some expensive boxes like Alteon, or try building your own from open source. Actually two Cisco boxes with some tricky NAT would help too. Basically you need to redirect the requests for a single public IP address into several private IP addresses in your datacenter. - Original Message > From: . . > To: swi...@swinog.ch > Sent: Tuesday, August 4, 2009 4:11:38 AM > Subject: [swinog] datacenter failover? > > hey guys, > we have some servers in datacenter#1: ns1, ns2, web1(mail/sql). > we also have ns3 and web2 outside this web. > > how can we make this working? > > ok, we can copy the data by cron, no problem. > > but can i give ns3 another ip for an a record? > > what`s your solution? > > greets, > sebastian > > ___ > swinog mailing list > swinog@lists.swinog.ch > http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog ___ swinog mailing list swinog@lists.swinog.ch http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog
Re: [swinog] datacenter failover?
I perfer to do the "Unison" way. I have some identical sites, that people are uploading web stuff to it. when one server get's updated, they will just keep the files updated http://www.cis.upenn.edu/~bcpierce/unison/ is deprecated, and will be replaced by harmony, but right now serves it's purpose. for other stuff I sometimes use either iSCSI (Target=Solaris, love it) with oracleFS on it or I just do DRBD and read and write to a small shared network drive by just "tar" into it ;) this could be a idea for a talk at the next Swinog meeting, is there an interest? I could present a bit about it... lg silvan Am 04.08.2009 um 09:18 schrieb : > hi sebastian > > ns1 = master > ns2/3 = slaves (which are public to the customers) > > sql stuff are solved by master-master replication setups (quite easy > for mysql), web can be solved by using rsync or other stuff (dont > use NFS ,-)) > > -steven ___ swinog mailing list swinog@lists.swinog.ch http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog
Re: [swinog] datacenter failover?
can be done at nearly no cost with two ALIX boards (depends on the load but often is enough) that are kept redundant with CARP on an interface, and then just do failover/loadbalancing with PFsense installed. an alixboard has 3 interfaces, fits perfect: 1) Carp Sync 2) inside 3) outside lg silvan Am 04.08.2009 um 09:24 schrieb Stanislav Sinyagin: > > two redundant Layer4 switches with VRRP would help. You can deploy > some expensive boxes > like Alteon, or try building your own from open source. > Actually two Cisco boxes with some tricky NAT would help too. > > Basically you need to redirect the requests for a single public IP > address into > several private IP addresses in your datacenter. > > > > > - Original Message >> From: . . >> To: swi...@swinog.ch >> Sent: Tuesday, August 4, 2009 4:11:38 AM >> Subject: [swinog] datacenter failover? >> >> hey guys, >> we have some servers in datacenter#1: ns1, ns2, web1(mail/sql). >> we also have ns3 and web2 outside this web. >> >> how can we make this working? >> >> ok, we can copy the data by cron, no problem. >> >> but can i give ns3 another ip for an a record? >> >> what`s your solution? >> >> greets, >> sebastian >> >> ___ >> swinog mailing list >> swinog@lists.swinog.ch >> http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog > > > ___ > swinog mailing list > swinog@lists.swinog.ch > http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog ___ swinog mailing list swinog@lists.swinog.ch http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog
Re: [swinog] datacenter failover?
Stanislav Sinyagin wrote: > Basically you need to redirect the requests for a single public IP > address into several private IP addresses in your datacenter. One acronym: LVS. /Per -- Per Jessen, Zürich (13.4°C) ___ swinog mailing list swinog@lists.swinog.ch http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog
Re: [swinog] datacenter failover?
- Original Message > From: Per Jessen > Stanislav Sinyagin wrote: > > > Basically you need to redirect the requests for a single public IP > > address into several private IP addresses in your datacenter. > > One acronym: LVS. yep, that is. I'm just not familiar with all those acronyms :) ___ swinog mailing list swinog@lists.swinog.ch http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog
Re: [swinog] datacenter failover?
Hi, TLA are your friends :) When it comes to DNS having a hidden master and then public slave to serve your zone is common, or you can have your zone in a DB an push the update on change. You may want to have a look at which DNS you want to use : Bind, NSD, PowerDNS, MaraDNS, djbdns .. one may be better than the other for your setup. There is as well plenty of tools to keep you DNS data in SQL. I really liked NameSurfer[1] when I used it (back around 2000) but it was _horribly_ expensive back then. I have used happily Sauron[2] , the interface is not really pretty but it works well. If the reason for having two DC is not resilience then LVS, or HAProxy[3] (never looked at PFsense) are good, I would be tempted to use HAProxy with apache mod_rpaf. It detects dead backend and I think it is simpler to configure, for failover you can as well have a look at spread/wackamole [4] I would love to see browser use SRV records for web but lost hope so unless you can unicast your network (very unlikely) the second DC is more a pain than anything for web. For webmail, as long as it uses imap as a backend it should not be an issue. Roundcube[4] for example use a DB for storing attachment and can be load balanced easily. I will stop here as if we start looking at DB replication or Mail clusters, the mail risk to be very long :D Thomas [1] http://www.nixusoftware.com/products_nss.html [2] http://sauron.jyu.fi/ [3] http://haproxy.1wt.eu/ [4] http://www.backhand.org/wackamole/ http://www.google.com/search?q=spread+wackamole ___ swinog mailing list swinog@lists.swinog.ch http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog
Re: [swinog] datacenter failover?
Hi, I can publish a small piece of software, named ABcg (for Azylog Bind config generator) which is able to take records from a SQL database and generate the zone files for Bind. This is intended for internal use but coded with an OSS mind. We're using it to generate zones for our DNS servers around the Europe with mysql master/master replication. It handles reverse zones, second level domain names (like something.asso.fr) and SPF records but come with no interface, you'll have to develop your own (ours is far too specific). At your disposal for a tarball of the whole thing (and a few advices for deployment) but also for feedback and contributions. * regards from France and frnog * Julien Escario Silvan M. Gebhardt a écrit : > I perfer to do the "Unison" way. I have some identical sites, that > people are uploading web stuff to it. when one server get's updated, > they will just keep the files updated > > http://www.cis.upenn.edu/~bcpierce/unison/ > > > is deprecated, and will be replaced by harmony, but right now serves > it's purpose. > > > for other stuff I sometimes use either iSCSI (Target=Solaris, love it) > with oracleFS on it or I just do DRBD and read and write to a small > shared network drive by just "tar" into it ;) > > > > this could be a idea for a talk at the next Swinog meeting, is there > an interest? I could present a bit about it... > > > lg > silvan > > Am 04.08.2009 um 09:18 schrieb : > >> hi sebastian >> >> ns1 = master >> ns2/3 = slaves (which are public to the customers) >> >> sql stuff are solved by master-master replication setups (quite easy >> for mysql), web can be solved by using rsync or other stuff (dont >> use NFS ,-)) >> >> -steven > > > ___ > swinog mailing list > swinog@lists.swinog.ch > http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog ___ swinog mailing list swinog@lists.swinog.ch http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog
Re: [swinog] datacenter failover?
Hi,
Here is attached the tinydns export script for sauron following an off-
list request, with many domains (100's/1000's) this is _MUCH_ faster
(never looked exactly) than the built in sauron tinydns export.
Thomas
#!/usr/bin/python
# Generate one big configuration file and does not care about the servers
defined in the DB
# XXX: To check: Take care of treating the numeric in the DB correctly and not
string (convert if needed)
# XXX: Sauron check as well the modification date to only write modified files,
we do not as those query are really fast
# XXX: we do not set the record as "exported" neither (so the interface still
show them as not-exported)
import sys
import os
import pgdb as db
# XXX: This is IPV4 only ...
# quick hack as we have lib somewhere
def iptoarpa (data):
return '.'.join(data.split('.')[::-1]) + ".in-addr.arpa."
option = {}
option['export-tinydns'] = True
option['export-bind'] = False
option['all-in-one'] = True
option['output-stdout'] = False
option['tmp-directory'] = './export'
option['one-zone-only'] = False
BIND_LINE = "%-32s %6s %2s %-6s %s\n"
domain_template = """
select
zones.name,
servers.hostname as ns,
coalesce(zones.hostmaster,servers.hostmaster) as hostmaster,
zones.serial,
coalesce(zones.refresh,servers.refresh) as refresh,
coalesce(zones.retry,servers.retry) as retry,
coalesce(zones.expire,servers.expire) as expire,
coalesce(zones.minimum,servers.minimum) as minimum,
coalesce(zones.ttl,servers.ttl),
zones.type,
zones.cuser,
zones.cdate,
zones.mdate,
zones.muser,
zones.active,
zones.serial_date,
zones.comment
from servers
join zones
on servers.id = zones.server
where
zones.active = true
%s
order by zones.name;
"""
domain_query = domain_template % ("and zones.dummy = 'f'\n\tand zones.reverse =
'f'\n\t%s")
domain_reverse = domain_template % ("and zones.reverse = 't'\n\t%s")
ns_query = """
select
hosts.domain as host,
zones.name as zone,
ns_entries.ns,
coalesce(hosts.ttl,zones.ttl) as ttl,
coalesce(zones.ttl,servers.ttl) as default_ttl
from servers
join zones
on servers.id = zones.server
join hosts
on zones.id = hosts.zone
join ns_entries
on ns_entries.ref = hosts.id
where
(hosts.type = 10 or hosts.type = 2)
and (zones.active = true)
%s
order by zones.name, hosts.domain;
"""
mx_query = """
select
hosts.domain as host,
zones.name as zone,
mx_entries.mx as mx_entries,
mx_entries.pri as mx_pri,
coalesce(hosts.ttl,zones.ttl) as ttl,
coalesce(zones.ttl,servers.ttl) as default_ttl
from servers
join zones
on servers.id = zones.server
join hosts
on zones.id = hosts.zone
join mx_entries
on hosts.type in (3,10)
and mx_entries.type = 2
and mx_entries.ref = hosts.id
where
zones.active = true
%s
order by zones.name, mx_entries.pri, hosts.domain;
"""
a_query = """
select
hosts.domain as host,
zones.name as zone,
a_entries.ip as a_ip,
a_entries.forward,
coalesce(hosts.ttl,zones.ttl) as ttl,
coalesce(zones.ttl,servers.ttl) as default_ttl,
hosts.type
from servers
join zones
on servers.id = zones.server
join hosts
on zones.id = hosts.zone
join a_entries
on hosts.type in (1,10)
and a_entries.host = hosts.id
where
zones.active = true
%s
order by zones.name, hosts.domain;
"""
a_reverse = a_query % "and a_entries.reverse = 't' %s"
a_forward = a_query % "and a_entries.forward = 't' %s"
txt_query = """
select
hosts.domain as host,
zones.name as zone,
txt_entries.txt as txt,
coalesce(hosts.ttl,zones.ttl) as ttl,
coalesce(zones.ttl,servers.ttl) as default_ttl
from servers
join zones
on servers.id = zones.server
join hosts
on zones.id = hosts.zone
join txt_entries
on txt_entries.ref = hosts.id
and txt_entries.txt != ''
where
zones.active = true
%s
order by zones.name, hosts.domain;
"""
# This query takes AGES ... :(
internal_cname_query = """
select
hosts.domain as host,
zones.name as zone,
int_hosts.domain as cname_host,
int_zones.name as cname_zone,
coalesce(hosts.ttl,zones.ttl) as ttl,
coalesce(zones.ttl,servers.ttl) as default_ttl
from servers
join zones
on servers.id = zones.server
join hosts
on zones.id = hosts.zone
join hosts as int_hosts
on hosts.alias
Re: [swinog] datacenter failover?
Yeah, thank you. Pretty big script as I'm really not a python user (Perl is better ... ok, i'm feeding the trolls). Julien Thomas Mangin a écrit : > Hi, > > Here is attached the tinydns export script for sauron following an > off-list request, with many domains (100's/1000's) this is _MUCH_ faster > (never looked exactly) than the built in sauron tinydns export. > > Thomas ___ swinog mailing list swinog@lists.swinog.ch http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog
Re: [swinog] datacenter failover?
Salut, On Tue, Aug 04, 2009 at 04:11:38AM +0200, . . wrote: > ok, we can copy the data by cron, no problem. Synchronize data as it is written. (In whatever way is appropriate; DNS NOTIFY, whatever works for the mails, etc.) > but can i give ns3 another ip for an a record? > > what`s your solution? For services other than DNS, which is totally distributed, use a separate network with an IP for every publically reachable IP. Map this network to the actual IPs of hosts in every datacenter. Use OSPF to direct this network to either datacenter. Tonnerre pgpmffiFF9I5j.pgp Description: PGP signature ___ swinog mailing list swinog@lists.swinog.ch http://lists.swinog.ch/cgi-bin/mailman/listinfo/swinog
