Certainly one can just copy the entire mail spool... getting MOST of hte messages, and the .headers, .cache, .index, etc. Is this safe?
Personally, I wouldn't just copy the spool from beginning to end while cyrus was running - all your indexes etc will become out of sync if mail is delivered to a mailbox while it's being copied. *ick*. You also run the risk of a mangled mailboxes.db if it's being modified while it's being copied.
What I'm doing here is using LVM on linux to take a snapshot of the mail spool (actually all of /var). I make a plain-text copy of the mailboxes.db then take the snapshot. I mount the snapshot on a temporary mountpoint, add imap/, spool/imap/, /etc/imapd.conf, /etc/cyrus.conf and /usr/local/cyrus (where all the binaries etc live on my system) into a CPIO archive, gzip it, and scp that to another host. That host later writes the cyrus backup archive to tape.
This is automated and happens nightly. Additionally, both the most recent and second-most-recent backup files are stored live on the other server, so I can quickly recover from a failure of the primary cyrus server that doesn't affect anything else.
Note that my method does /not/ ensure that Cyrus isn't modifying the mail spools as I take the snapshot, nor does it ensure that mailboxes aren't changed between the conversion of the mailboxes.db to text and the snapshot. It's not perfect. For my site, however, it's quite sufficient - at worst I'll have an AWOL mailbox or two, a mailbox slightly out of sync with it's indexes, or a truncated message.
I'm also running into an issue where occasionally the snapshot fails - it's taken while reiserfs is in the middle of a transaction, and the journal can't be replayed on mount because the snapshot is read-only. This is more of an LVM issue than anything else. I get around it by syncing the disk before taking the snapshot, and by deleting the snapshot and trying again if it's not useable.
I'd love to know of a way to ask cyrus to defer message delivery briefly and make sure everything is in sync. That way I could signal cyrus to "sync up and pause", do the db conversions, snapshot the volume, and resume cyrus. I'd prefer to avoid killing and restarting the master during the normal backup process.
What about the databases? Are they save to just copy? are they recoverable? Can they be rebuilt from the plain messages?
My understanding is that the mailbox contents can be rebuilt from the messages, though I seem to remember you'll lose all your flags like read, etc. The mailboxes.db _can not_ be reconstructed easily and must be backed up; the rest, I just don't know enough about. I understand that you can afford to lose the rest of the DBs.
Isn't that what matters most, the messages?
The messages and and your mailboxes DB, yes. Though users will be /very/ unhappy if you lose their subscriptions and/or read status. Recovery without a copy of your mailboxes db is, I understand, a total nightmare. I make a plaintext copy so that if I'm restoring the mail spools etc onto a host with a different berkley DB environment I can get up and running quickly and easily.
Anyway, I'm extremely far from an expert on this, having been running a Cyrus server for only a few months. If I'm wrong on some points, please let me know folks.
This area is one where I'm going to try to add some info to the wiki. There's some there, but it seems a little sparse and maybe I can help.
Craig Ringer
