Re: [Dovecot] Clustering dovecot?
On Fri, 2009-05-29 at 09:41 +0200, list...@gmx.de wrote: > (Take care not to use NFS for storage if you take another setup than the here If you use MailDir you will never have NFS problems, we have multiple SMTP's, multiple POP3's and 2 x webmail (IMAP) servers, all to a netapp NAS for mail storage, it works incredibly well.
Re: [Dovecot] Clustering dovecot?
Hi Rick, at the moment I´m building the same setup than you. I have no further experience with it, but I made a setup in our testing lab and under testing conditions it seems to run quite nice. I took 2 servers with heartbeat1 in active/passive node. Each server has its own IP, and they have a cluster IP that´s managed by heartbeat only. This cluster IP is provided in our DNS for accessing the mailstorage cluster, and only the active node has it at the time. Then I have a DRBD shared storage on the two nodes. On the DRBD storage I only put the dovecot maildir and mysql databases. The dovecot and mysql binaries are not shared and the configuration also not. DRBD, dovecot and Mysql are managed by heartbeart. There is always a danger that the connection between the 2 nodes is failing and you will have a "split brain" then with a big data mess. So it´s important to provide redundancy in the connections. For heartbeat, I have one dedicated LAN connection and a serial connection. For DRBD, I use 2 bonded NICs on different PCI cards. Take a look at DOPD for DRBD. This marks the passive DRBD partition "outdated" if the DRBD connection fails, and because heartbeat can only takeover if it can start all resources of a resource group, a failover is not possible anymore if the DRBD connection is broken, so you can´t mess up your DRBD so easy any more. If both heartbeat connections fail, you will have lots of trouble, and that´s easy to achieve with some wrong iptables if you take only LAN connections. So the serial cable is a nice thing because it´s not affected! We use heartbeat1 because we had some trouble bringing heartbeat2 to run. Heartbeat1 is not able to monitor it´s resources, so we thought about using MON for this. And to take some STONITH devices like telnet accessible power outlets to switch off the power of a failing node automatically. But this setup seems to be rather complex, which is the enemy of reliability, and we heard about people having problems with accidently automatic failovers or reboots. So in the end we decided against an automatic failover in the case a service dies. We use only the failover of heartbeat1, e.g. if the active node dies completely, there will be a failover to the passive node. And we use connection redundancy to hopefully not have a split brain. And make a good backup ;-) (Take care not to use NFS for storage if you take another setup than the here described because you can have trouble with file locking!) Our cluster is protecting against hardware problems, and against some kind of software problems. Because of DRBD, if you do a "rm -rf" on the maildir, you loose all data on _both_ nodes in the same second, so the protection against administration faults is not very good! Backups are really important. But if we have some trouble with the active node, and we can´t fix it in some minutes, we can try a failover to the passive node and there is a big chance that the service is running on the other node quite well. A nice things for software updates. For MTA we use Postfix. Because it´s not a good idea to put the postfix mailqueue on a DRBD (bad experiences), you will have some mails (temporarily) lost if you do a failover. So it´s a good idea to minimize the time mails are held in the queue. Because of this and because we need a longtime stable mailstorage but an always up-to-date brand new SPAM and virus filter, we decided to put 2 Postfix/Amavis/Spamassassin/Antivirus relays in front of the IMAP cluster. They´re identical, with the same MX priority in DNS, so if one of the relays fails, the other one takes the load. As I said, this solution is working only in the lab now and not yet in production, but there the failover seems to be no problem at all for the clients. So I hope I could give you some ideas. regards, Andreas -- Nur bis 31.05.: GMX FreeDSL Komplettanschluss mit DSL 6.000 Flatrate und Telefonanschluss nur 17,95 Euro/mtl.!* http://portal.gmx.net/de/go/dsl02
Re: [Dovecot] Clustering dovecot?
dove...@corwyn.net schrieb: > At 01:17 PM 5/27/2009, Seth Mattinen wrote: >> You're probably looking for Dovecot replication, which as far as I'm >> aware, has not made itself known in a functional form yet. I really want >> to see master/master replication. > > I at least don't think I am - I think I'd rather run dovecot with two > front-end servers clustered to a backend SAN. If I"m running two > separate servers then all of my clients (which i can point to only 1 IP) > I think woudl go down if "their" server went down. > > But what it sounds like you're saying is that there's no good way to run > dovecot so that if the single server it's on fails I can keep service > availability? > > Rick > > > Hi, you need i.e. a ha loadbalancing setup, and a master master replication sql server setup this do the job with 2 or 4 servers ( with 2 loadbalancers ), as you can use DRBD as written before -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria
Re: [Dovecot] Clustering dovecot?
dove...@corwyn.net wrote: > At 02:00 PM 5/27/2009, Seth Mattinen wrote: >> You're going to need something in front of the two servers to abstract >> the connections (another poster recommended ultramonkey) if you want >> them both to be serviced by the same IP address. My suggestion was to >> use active/standby, where the standby doesn't "up" its mail IP unless it >> detects the primary is dead, typically through some kind of heartbeat; >> not separate clients across the two servers. > > I'm not sure what you mean by "client"; do you mean instances of dovecot? > > wouldn't that be two server instances, each running it's own copy of > dovecot? > Yes, two totally separate servers, either running shared storage (still a common failure point, unless you have redundancy there too) or one that waits in the wings in standby mode until the primary fails using DBRD or something else to keep the mail stores in sync. Their website explains it all better than I can: http://www.drbd.org/ ~Seth
Re: [Dovecot] Clustering dovecot?
On Wed, 2009-05-27 at 21:47 -0400, dove...@corwyn.net wrote: > At 04:21 PM 5/27/2009, you wrote: > >On Wed, 2009-05-27 at 14:02 -0400, dove...@corwyn.net wrote: > >yes, and use NAS not SAN > > Why NAS and not SAN? > Because the two technologies differ and it permits you to run non clustering FS's, remember, SAN is block device, so it can only do one single write operation at a time, NAS, as many as you can throw at it, each has its own pros and cons, for mail NAS is perfect, for databases, SAN is better. > >you could also use DNS load balancing in a away that has 0 cache TTL on > >MX addresses, thats better than a software based LB if you cant afford a > >hardware LB > I'm not sure I understand that. The MX records have little to do with Your correct (too early in the morning when I sent that) but the same applies, if its for redundancy set 0 in DNS record for the hosts so if one fails, retrying has a better chance of getting the alive one, if you are that worried about it, get yourself a hardware solution, there are plenty around and cheap enough these days.
Re: [Dovecot] Clustering dovecot?
At 06:39 PM 5/27/2009, Timo Sirainen wrote: If you don't necessarily need load-balancing to multiple servers, it should be faster and more reliable to use active/passive and some (automated) failover between them. If you use active/active servers, you either need some clustered filesystem (not that efficient) or NFS (kind of sucks too). Is active/passive a dovecot configuration option? Rick
Re: [Dovecot] Clustering dovecot?
At 04:21 PM 5/27/2009, you wrote: On Wed, 2009-05-27 at 14:02 -0400, dove...@corwyn.net wrote: yes, and use NAS not SAN Why NAS and not SAN? you could also use DNS load balancing in a away that has 0 cache TTL on MX addresses, thats better than a software based LB if you cant afford a hardware LB I'm not sure I understand that. The MX records have little to do with Dovecot richt (as that would be where things look for the SMTP record), which would be inbound mail which is handled by the SMTP server (in my case postfix). I can see load balancing DNS for what the IMAP server is, but why the MX records? Rick
Re: [Dovecot] Clustering dovecot?
At 02:00 PM 5/27/2009, Seth Mattinen wrote: You're going to need something in front of the two servers to abstract the connections (another poster recommended ultramonkey) if you want them both to be serviced by the same IP address. My suggestion was to use active/standby, where the standby doesn't "up" its mail IP unless it detects the primary is dead, typically through some kind of heartbeat; not separate clients across the two servers. I'm not sure what you mean by "client"; do you mean instances of dovecot? wouldn't that be two server instances, each running it's own copy of dovecot? Rick ~Seth
Re: [Dovecot] Clustering dovecot?
On Wed, 2009-05-27 at 14:02 -0400, dove...@corwyn.net wrote: > At 01:49 PM 5/27/2009, Rick Romero wrote: > > >>But what it sounds like you're saying is that there's no good way to > >>run dovecot so that if the single server it's on fails I can keep > >>service availability? > > > >If you have 2 dovecot servers with your mailboxes stored on a SAN > >(over NFS), you can front your 2 dovecot machines with a load balancer > >like UltraMonkey (LHA/Heartbeat). > > Would this mean I would configure two identical dovecot systems, and > point them both at the same SAN space? (so instead of having a > "clustered" environment both instances of dovecot are just using > shared filespace?) Isn't there a risk there of both dovecot > instances writing a file/email with the same filename? If you don't necessarily need load-balancing to multiple servers, it should be faster and more reliable to use active/passive and some (automated) failover between them. If you use active/active servers, you either need some clustered filesystem (not that efficient) or NFS (kind of sucks too). signature.asc Description: This is a digitally signed message part
Re: [Dovecot] Clustering dovecot?
On Wed, 2009-05-27 at 14:02 -0400, dove...@corwyn.net wrote: > > Would this mean I would configure two identical dovecot systems, and > point them both at the same SAN space? (so instead of having a yes, and use NAS not SAN you could also use DNS load balancing in a away that has 0 cache TTL on MX addresses, thats better than a software based LB if you cant afford a hardware LB
Re: [Dovecot] Clustering dovecot?
At 01:49 PM 5/27/2009, Rick Romero wrote: But what it sounds like you're saying is that there's no good way to run dovecot so that if the single server it's on fails I can keep service availability? If you have 2 dovecot servers with your mailboxes stored on a SAN (over NFS), you can front your 2 dovecot machines with a load balancer like UltraMonkey (LHA/Heartbeat). Would this mean I would configure two identical dovecot systems, and point them both at the same SAN space? (so instead of having a "clustered" environment both instances of dovecot are just using shared filespace?) Isn't there a risk there of both dovecot instances writing a file/email with the same filename? (and I presume also have my user mysql config also stored and shared on the SAN between the two servers?). RickS
Re: [Dovecot] Clustering dovecot?
dove...@corwyn.net wrote: > At 01:17 PM 5/27/2009, Seth Mattinen wrote: >> You're probably looking for Dovecot replication, which as far as I'm >> aware, has not made itself known in a functional form yet. I really want >> to see master/master replication. > > I at least don't think I am - I think I'd rather run dovecot with two > front-end servers clustered to a backend SAN. If I"m running two > separate servers then all of my clients (which i can point to only 1 IP) > I think woudl go down if "their" server went down. You're going to need something in front of the two servers to abstract the connections (another poster recommended ultramonkey) if you want them both to be serviced by the same IP address. My suggestion was to use active/standby, where the standby doesn't "up" its mail IP unless it detects the primary is dead, typically through some kind of heartbeat; not separate clients across the two servers. ~Seth
Re: [Dovecot] Clustering dovecot?
Quoting dove...@corwyn.net: At 01:17 PM 5/27/2009, Seth Mattinen wrote: You're probably looking for Dovecot replication, which as far as I'm aware, has not made itself known in a functional form yet. I really want to see master/master replication. I at least don't think I am - I think I'd rather run dovecot with two front-end servers clustered to a backend SAN. If I"m running two separate servers then all of my clients (which i can point to only 1 IP) I think woudl go down if "their" server went down. But what it sounds like you're saying is that there's no good way to run dovecot so that if the single server it's on fails I can keep service availability? If you have 2 dovecot servers with your mailboxes stored on a SAN (over NFS), you can front your 2 dovecot machines with a load balancer like UltraMonkey (LHA/Heartbeat). Rick
Re: [Dovecot] Clustering dovecot?
At 01:17 PM 5/27/2009, Seth Mattinen wrote: You're probably looking for Dovecot replication, which as far as I'm aware, has not made itself known in a functional form yet. I really want to see master/master replication. I at least don't think I am - I think I'd rather run dovecot with two front-end servers clustered to a backend SAN. If I"m running two separate servers then all of my clients (which i can point to only 1 IP) I think woudl go down if "their" server went down. But what it sounds like you're saying is that there's no good way to run dovecot so that if the single server it's on fails I can keep service availability? Rick
Re: [Dovecot] Clustering dovecot?
dove...@corwyn.net wrote: > > > > Currently I have a mail server running postfix and dovecot with users in > mysql. I have Outlook (blich) clients checking IMAP mail on the server. > I'd like to build some fault-tolerance into that system in case I have > hardware failure. > > I've read through: http://wiki.dovecot.org/MailLocation/SharedDisk but I > feel I need more information. My basic understanding is that I'll need 2 > clustered systems running with shared disk space and shared IP. > Performance/load-balancing isn't an issue for me, it's just > fault-tolerance that I'm worried about. > > I found a nice long doc for Cyrus IMAP > (http://cyrusimap.web.cmu.edu//ag.html), anything similar for dovecot? > You're probably looking for Dovecot replication, which as far as I'm aware, has not made itself known in a functional form yet. I really want to see master/master replication. I was working on coming up with something myself in an active/standby setup using DRBD. If the heartbeat dies, DRBD on the standby fires up its Dovecot and takes over. Vice-versa for the primary recovering: shut down standby and start primary. It's not totally transparent because it would drop all active connections during the switch. In practice, I find that nobody notices unless it happens multiple times in a row or they happened to be in the middle of a long IMAP command. ~Seth
[Dovecot] Clustering dovecot?
Currently I have a mail server running postfix and dovecot with users in mysql. I have Outlook (blich) clients checking IMAP mail on the server. I'd like to build some fault-tolerance into that system in case I have hardware failure. I've read through: http://wiki.dovecot.org/MailLocation/SharedDisk but I feel I need more information. My basic understanding is that I'll need 2 clustered systems running with shared disk space and shared IP. Performance/load-balancing isn't an issue for me, it's just fault-tolerance that I'm worried about. I found a nice long doc for Cyrus IMAP (http://cyrusimap.web.cmu.edu//ag.html), anything similar for dovecot? Rick Rick Steeves http://www.sinister.net "The more I learn, it seems, the less I know." Frazz
