Cyrus Backup Strategy
My last thing to do prior to converting to production is a backup strategy. I have been doing this with tar something like: tar -C /var/lib-czf lib-date.tar.gzimap tar -C /var/spool -czf spool-date.tar.gz imap tar -cf cyrus-date.tar This is producing a pretty big file(s): lib-date.tar.gz- 2.2M spool-date.tar.gz - 11.0M and ultimately cyrus-date.tar - 13.5M and I was thinking maybe there is a better way that someone else has come up with. Anyone doing backups a different way?
Re: Cyrus Backup Strategy
pnelson wrote: My last thing to do prior to converting to production is a backup strategy. I have been doing this with tar something like: tar -C /var/lib-czf lib-date.tar.gzimap tar -C /var/spool -czf spool-date.tar.gz imap tar -cf cyrus-date.tar This is producing a pretty big file(s): lib-date.tar.gz- 2.2M spool-date.tar.gz - 11.0M and ultimately cyrus-date.tar - 13.5M and I was thinking maybe there is a better way that someone else has come up with. Anyone doing backups a different way? Are you then dumping these files to tape? Otherwise, just having another copy on disk doesn't protect against many potential causes of data loss. We just backup the mail files normally with Veritas Netbackup (be sure to disable true image restore with the millions of files), with the only custom bit being to dump a copy of the mailbox list to a text file before the backups run (this is a precaution since we are backing up structured database files without synchronization with the program writing them, so there is no guarantee the resulting backup is useful). Our full backup for all Cyrus files is 98G and 4.2M files, taking 5-8 hours depending on other data hitting the tape drives. Daily incremental backups average 1-3G and 10-30k files. -- John A. Tamplin Unix System Administrator Emory University, School of Public Health +1 404/727-9931
Re: Cyrus Backup Strategy
On Fri, 2003-06-13 at 11:29, John Alton Tamplin wrote: pnelson wrote: My last thing to do prior to converting to production is a backup strategy. I have been doing this with tar something like: tar -C /var/lib-czf lib-date.tar.gzimap tar -C /var/spool -czf spool-date.tar.gz imap tar -cf cyrus-date.tar This is producing a pretty big file(s): lib-date.tar.gz- 2.2M spool-date.tar.gz - 11.0M and ultimately cyrus-date.tar - 13.5M and I was thinking maybe there is a better way that someone else has come up with. Anyone doing backups a different way? Are you then dumping these files to tape? Otherwise, just having another copy on disk doesn't protect against many potential causes of data loss. Yes dumping to removable medium. We just backup the mail files normally with Veritas Netbackup (be sure to disable true image restore with the millions of files), with the only custom bit being to dump a copy of the mailbox list to a text file before the backups run (this is a precaution since we are backing up structured database files without synchronization with the program writing them, so there is no guarantee the resulting backup is useful). Our full backup for all Cyrus files is 98G and 4.2M files, taking 5-8 hours depending on other data hitting the tape drives. Daily incremental backups average 1-3G and 10-30k files. So a cyrus backup needs to contain: /var/spool/imap /var/lib/imap/mailboxes.db to be useful as a backup, right? With these to things I can recover... I'm making sure I understand this completely. How do you dump mailboxes to/from text? What is the restore process? Thanks
Re: Cyrus Backup Strategy
On Fri, 13 Jun 2003, pnelson wrote: On Fri, 2003-06-13 at 11:29, John Alton Tamplin wrote: pnelson wrote: My last thing to do prior to converting to production is a backup strategy. I have been doing this with tar something like: tar -C /var/lib-czf lib-date.tar.gzimap tar -C /var/spool -czf spool-date.tar.gz imap tar -cf cyrus-date.tar This is producing a pretty big file(s): lib-date.tar.gz- 2.2M spool-date.tar.gz - 11.0M and ultimately cyrus-date.tar - 13.5M and I was thinking maybe there is a better way that someone else has come up with. Anyone doing backups a different way? Are you then dumping these files to tape? Otherwise, just having another copy on disk doesn't protect against many potential causes of data loss. Yes dumping to removable medium. We just backup the mail files normally with Veritas Netbackup (be sure to disable true image restore with the millions of files), with the only custom bit being to dump a copy of the mailbox list to a text file before the backups run (this is a precaution since we are backing up structured database files without synchronization with the program writing them, so there is no guarantee the resulting backup is useful). Our full backup for all Cyrus files is 98G and 4.2M files, taking 5-8 hours depending on other data hitting the tape drives. Daily incremental backups average 1-3G and 10-30k files. So a cyrus backup needs to contain: /var/spool/imap /var/lib/imap/mailboxes.db to be useful as a backup, right? With these to things I can recover... I'm making sure I understand this completely. I am not sure what your directory structure looks like, but you need to make sure to backup sieve and quota directories. Your procedure may require hand recovery after restore before the mail store is usable. I recommend using filesystem snapshot (i assume you do not stop the mail server during backup) before performing backup. This will make your backup 'sane.' How do you dump mailboxes to/from text? What is the restore process? Thanks -- Igor
Re: Cyrus Backup Strategy
On Fri, 2003-06-13 at 12:11, Igor Brezac wrote: On Fri, 13 Jun 2003, pnelson wrote: On Fri, 2003-06-13 at 11:29, John Alton Tamplin wrote: pnelson wrote: My last thing to do prior to converting to production is a backup strategy. I have been doing this with tar something like: tar -C /var/lib-czf lib-date.tar.gzimap tar -C /var/spool -czf spool-date.tar.gz imap tar -cf cyrus-date.tar This is producing a pretty big file(s): lib-date.tar.gz- 2.2M spool-date.tar.gz - 11.0M and ultimately cyrus-date.tar - 13.5M and I was thinking maybe there is a better way that someone else has come up with. Anyone doing backups a different way? Are you then dumping these files to tape? Otherwise, just having another copy on disk doesn't protect against many potential causes of data loss. Yes dumping to removable medium. We just backup the mail files normally with Veritas Netbackup (be sure to disable true image restore with the millions of files), with the only custom bit being to dump a copy of the mailbox list to a text file before the backups run (this is a precaution since we are backing up structured database files without synchronization with the program writing them, so there is no guarantee the resulting backup is useful). Our full backup for all Cyrus files is 98G and 4.2M files, taking 5-8 hours depending on other data hitting the tape drives. Daily incremental backups average 1-3G and 10-30k files. So a cyrus backup needs to contain: /var/spool/imap /var/lib/imap/mailboxes.db to be useful as a backup, right? With these to things I can recover... I'm making sure I understand this completely. I am not sure what your directory structure looks like, but you need to make sure to backup sieve and quota directories. Your procedure may require hand recovery after restore before the mail store is usable. I recommend using filesystem snapshot (i assume you do not stop the mail server during backup) before performing backup. This will make your backup 'sane.' How do you dump mailboxes to/from text? What is the restore process? Good point. So the latest process would be: So a cyrus backup needs to contain: su cyrus -c ctl_mboxlist -d mailboxes-date.txt /var/spool/imap /var/lib/imap/quota /var/lib/imap/user /var/lib/imap/sieve Not sure how the filesystem snapshot works? No I do not want to stop the server.
Re: Cyrus Backup Strategy
On Fri, 13 Jun 2003, pnelson wrote: On Fri, 2003-06-13 at 12:11, Igor Brezac wrote: On Fri, 13 Jun 2003, pnelson wrote: On Fri, 2003-06-13 at 11:29, John Alton Tamplin wrote: pnelson wrote: My last thing to do prior to converting to production is a backup strategy. I have been doing this with tar something like: tar -C /var/lib-czf lib-date.tar.gzimap tar -C /var/spool -czf spool-date.tar.gz imap tar -cf cyrus-date.tar This is producing a pretty big file(s): lib-date.tar.gz- 2.2M spool-date.tar.gz - 11.0M and ultimately cyrus-date.tar - 13.5M and I was thinking maybe there is a better way that someone else has come up with. Anyone doing backups a different way? Are you then dumping these files to tape? Otherwise, just having another copy on disk doesn't protect against many potential causes of data loss. Yes dumping to removable medium. We just backup the mail files normally with Veritas Netbackup (be sure to disable true image restore with the millions of files), with the only custom bit being to dump a copy of the mailbox list to a text file before the backups run (this is a precaution since we are backing up structured database files without synchronization with the program writing them, so there is no guarantee the resulting backup is useful). Our full backup for all Cyrus files is 98G and 4.2M files, taking 5-8 hours depending on other data hitting the tape drives. Daily incremental backups average 1-3G and 10-30k files. So a cyrus backup needs to contain: /var/spool/imap /var/lib/imap/mailboxes.db to be useful as a backup, right? With these to things I can recover... I'm making sure I understand this completely. I am not sure what your directory structure looks like, but you need to make sure to backup sieve and quota directories. Your procedure may require hand recovery after restore before the mail store is usable. I recommend using filesystem snapshot (i assume you do not stop the mail server during backup) before performing backup. This will make your backup 'sane.' How do you dump mailboxes to/from text? What is the restore process? Good point. So the latest process would be: So a cyrus backup needs to contain: su cyrus -c ctl_mboxlist -d mailboxes-date.txt This is not neccassary, but not a bad idea. /var/spool/imap /var/lib/imap/quota /var/lib/imap/user /var/lib/imap/sieve Not sure how the filesystem snapshot works? On solaris do 'man fssnap'. veritas filesystem supports the snapshot feature as does Linux (at least the newer versions), I am not sure which filesystem. Basically, this feature provides a temporary stable and read-only view (snapshot) of a filesystem. The writes occur on a separate filesystem or device. -- Igor
