Hi, Any news about multi-master replication?
Thanks 2013/5/24 <cyrus-devel-requ...@lists.andrew.cmu.edu> > Send Cyrus-devel mailing list submissions to > cyrus-devel@lists.andrew.cmu.edu > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.andrew.cmu.edu/mailman/listinfo/cyrus-devel > or, via email, send a message with subject or body 'help' to > cyrus-devel-requ...@lists.andrew.cmu.edu > > You can reach the person managing the list at > cyrus-devel-ow...@lists.andrew.cmu.edu > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Cyrus-devel digest..." > > > Today's Topics: > > 1. Re: Replication documentation anywhere? (Bron Gondwana) > 2. Re: Replication documentation anywhere? (Karl Pielorz) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 24 May 2013 11:10:16 +1000 > From: Bron Gondwana <br...@fastmail.fm> > Subject: Re: Replication documentation anywhere? > To: cyrus-devel@lists.andrew.cmu.edu > Message-ID: > < > 1369357816.25848.140661234996321.590d3...@webmail.messagingengine.com> > > Content-Type: text/plain > > On Fri, May 24, 2013, at 01:34 AM, Karl Pielorz wrote: > > We're running cyrus-imapd-2.4.17 on FreeBSD. I've been looking at the > > replication built into Cyrus, but can't find much (if any) > > documentation on it. > > > > e.g. The shipped 'install-replication.html' file ends at: > > > > " ... You can also run cyr_synclog(8) instead, which will insert the > > record into the rolling replication log. > > > > Failover " > > > > And that's it. Is there anywhere I can find more info on replication / > > failover / setup, a 'howto' - or anything? > > I'm afraid there isn't much :( Feel free to ask questions about > specific things you run in to, and we can use that as a basis to put > together more detailed documentation. > > Failover is kind of messy at the moment, because it's so site-dependent > how you want to manage your failover. Our process at FastMail looks > like this: > > 1) update database to mark the server as "moving" so new connections get > paused at nginx/web server level and then wait 2 seconds for the > config to be updated. > 2) send a signal to the 'master' process to shut the server down. > 3) wait for up to 10 seconds while grepping the process list every > second for ongoing processes related to the same instance (we use -C > $imapd_conf because we run many instances of Cyrus per server) > 4) if the processes aren't dead after 10 seconds, kill them > individually. > 5) if THAT fails, kill -9. (yeah, I know - evil!) > 6) check the $confdir/sync directory for log files, and run them with > sync_client to ensure all replication is up to date. > > If anything before this failed, we bring this master back up and report > that the failover didn't succeed. > > 7) shut down the replica > 8) restart this side with the replica configuration > 9) restart the other side with the master configuration > > At the moment, we still move a master IP address to the instance which > is running as master, meaning clients can reconnect to the same IP > address after the failover. This is on its way out - we're now at the > point where almost everything can read configuration from our > "fmstatus.json" file which is updated every second on every host, so > they know where the master is actually located. > > Obviously, a ton of this is really site-specific to us. > > Soon (yes, soon!) we will be shifting to a full multi-master setup, > where failover is as simple as pointing clients to the other end of > the replication pair, and killing off existing connections so they > reconnect (with some sync_log checking and force-running), which should > shave quite a few seconds off the sync time and mean that long running > squatter jobs and other things don't get nuked off at the same time. > > But yeah, it's not quite a turn-key system :( > > Bron. > > -- > Bron Gondwana > br...@fastmail.fm > > > ------------------------------ > > Message: 2 > Date: Fri, 24 May 2013 09:41:17 +0100 > From: Karl Pielorz <kpielorz_...@tdx.co.uk> > Subject: Re: Replication documentation anywhere? > To: Bron Gondwana <br...@fastmail.fm>, > cyrus-devel@lists.andrew.cmu.edu > Message-ID: <b2787ce797592c5f029d5...@mail-pc.tdx.co.uk> > Content-Type: text/plain; charset=us-ascii; format=flowed > > > --On 24 May 2013 11:10 +1000 Bron Gondwana <br...@fastmail.fm> wrote: > > > I'm afraid there isn't much :( Feel free to ask questions about > > specific things you run in to, and we can use that as a basis to put > > together more detailed documentation. > > Hi, > > Thanks for your reply (which I've read through now!) - if we have an > existing mailstore (with ~2-3 million emails) and we setup a replica is > 'sync_client' going to be able to bring the replica to the same state as > the master? (given time, and not too many changes on the master?) - or are > we better off using another method (such as FS snapshot -> replica). > > Presumably the replication doesn't do anything for users (we use sasl for > auth, i.e. saslpasswd2 to create the users) - if we intend to have users > 'hitting' the replica (when the master as failed) presumably we'd need to > create those users / passwords on the replica as well? > > In a nutshell - when we have the master / replica setup, if we 'switch' to > having users hit the replica (because the master has failed) - it appears > we can simply 'switch' the replication configs over, and then have what was > the master 're-sync' with the new master? > > I guess what I'm trying to figure out is if the replication / 'sync_client' > is the 'magic' solution is appears to be - i.e. given enough bandwidth, > time / 'grunt' & low enough volume of changes on the master - it'll drag an > empty mailstore up to be an identical replica, or re-sync a once failed > master to be a new replica. > > Thanks again for your time, > > -Karl > > > ------------------------------ > > _______________________________________________ > Cyrus-devel mailing list > Cyrus-devel@lists.andrew.cmu.edu > https://lists.andrew.cmu.edu/mailman/listinfo/cyrus-devel > > > End of Cyrus-devel Digest, Vol 92, Issue 8 > ****************************************** >