Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
On Thu, 2010-02-11 at 10:46 +0500, CoolAtt NNA wrote: from the wiki of Lazy Expunge: The plugin is configured by defining namespaces where the mails are moved. You can decide if you want the namespaces to be visible to clients, or if you want to show them only via some special webmail interface. You can use either one or three namespaces How do I access the namespaces(the back up mailboxes) ? Well, a) You can make the namespaces visible (list=yes) and they'll show up just like regular mailboxes. Might confuse users. b) You make the namespaces hidden (list=no, hidden=yes) and the only way to access them is by explicitly configuring client's namespace path to that. That's of course a pretty painful thing to do. Maybe by using two different accounts that would be slightly better. But the main reason for lazy_expunge is to allow a) custom-built webmails or b) sysadmins to do the unexpunging. signature.asc Description: This is a digitally signed message part
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
i use drbd to keep the mailstore redundant and rsnapshot for backup.
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
On 10/02/2010 19:22, Terry Barnum wrote: There's also rsnapshot which I've been using for hardlinked, incremental backups for a couple years now. Very flexible and has saved us a couple times. http://www.rsnapshot.org It's rarely mentioned, but I prefer snapback2 instead - basically the same idea, but it has a rotation schedule which is X daily, Y weekly and Z Monthly backups - more efficient in my mind Ed W
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
CoolAtt NNA wrote: Hi All.. Am working on continous backup of mailboxes using rsync(for e.g by running rsync every 2 min) Things gets more complicated when users create Subfolders in INBOX , SENT , etc.. If anyone among you did that plz guide advise. We have a slightly different method of doing a back-up, at least of people's inboxes (which is all I archive). I've written a program, which is run by cron once a minute, which searches for messages within all the users' inboxes and then hard-links them into a backup store elsewhere on the disk. Then, once a day a reaper process runs which checks for all files in that store with only one link and have not been accessed for a set amount of time (in our case 90 days) and then deletes those files. In other words, the messages are kept for 90 days after they have been deleted or moved out of the user's inbox. I've attached the source code tar file as you or someone else may find it useful. In your case you could rsync the back-up store instead of the user's main directory structure and maybe only do it once per day. Steve -- --- IT Systems Administrator, E-Mail:- st...@earth.ox.ac.uk Department of Earth Sciences,Tel:- +44 (0)1865 282110 University of Oxford, Parks Road, Oxford, UK.Fax:- +44 (0)1865 272072 backupinboxes.tar.gz Description: GNU Zip compressed data
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
On Wed, 2010-02-10 at 13:57 +, Stephen Usher wrote: I've written a program, which is run by cron once a minute, which searches for messages within all the users' inboxes and then hard-links them into a backup store elsewhere on the disk. Then, once a day a reaper process runs which checks for all files in that store with only one link and have not been accessed for a set amount of time (in our case 90 days) and then deletes those files. In other words, the messages are kept for 90 days after they have been deleted or moved out of the user's inbox. http://wiki.dovecot.org/Plugins/Lazyexpunge also does pretty much that, although for all mailboxes, not just for INBOX. And it doesn't automatically expire them. signature.asc Description: This is a digitally signed message part
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
On Fri, Jan 29, 2010 at 09:59:22AM +0200, Timo Sirainen wrote: If you didn't give --delete parameter to rsync, there could be duplicates, yes. Dovecot just logs about them and gives new UIDs to them, so you'll end up having duplicate mails. I suppose some day I should just change it to log a warning, but delete if the files are otherwise identical. Would this be easy to change? It solves this problem, although perhaps not in the most efficient way... Geert -- Geert Hendrickx -=- g...@telenet.be -=- PGP: 0xC4BB9E9F This e-mail was composed using 100% recycled spam messages!
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
On 2010-02-10 2:22 PM, Terry Barnum wrote: There's also rsnapshot which I've been using for hardlinked, incremental backups for a couple years now. Very flexible and has saved us a couple times. http://www.rsnapshot.org +1 for rsnapshot - although it doesn't answer the 'continuous' backup aspect of this thread... -- Best regards, Charles
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
On Feb 10, 2010, at 11:53 AM, Charles Marcus wrote: On 2010-02-10 2:22 PM, Terry Barnum wrote: There's also rsnapshot which I've been using for hardlinked, incremental backups for a couple years now. Very flexible and has saved us a couple times. http://www.rsnapshot.org +1 for rsnapshot - although it doesn't answer the 'continuous' backup aspect of this thread... I'm pretty sure rsnapshot could be configured to run every x minutes. -Terry Terry Barnum digital OutPost San Diego, CA http://www.dop.com 800/464-6434
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
On Wed, 2010-02-10 at 15:14 +0100, Geert Hendrickx wrote: On Fri, Jan 29, 2010 at 09:59:22AM +0200, Timo Sirainen wrote: If you didn't give --delete parameter to rsync, there could be duplicates, yes. Dovecot just logs about them and gives new UIDs to them, so you'll end up having duplicate mails. I suppose some day I should just change it to log a warning, but delete if the files are otherwise identical. Would this be easy to change? It solves this problem, although perhaps not in the most efficient way... Sure, it'll be easy to change, for whoever does the change. :) I've a patch somewhere in mailing list archives that just changes the code to assume identical names means identical mails. signature.asc Description: This is a digitally signed message part
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
from the wiki of Lazy Expunge: The plugin is configured by defining namespaces where the mails are moved. You can decide if you want the namespaces to be visible to clients, or if you want to show them only via some special webmail interface. You can use either one or three namespaces How do I access the namespaces(the back up mailboxes) ? We are using Thunderbird,Microsoft Outlook RoundCube Webmail as email clients. From: te...@dop.com To: dovecot@dovecot.org Date: Wed, 10 Feb 2010 11:22:06 -0800 Subject: Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ? On Feb 10, 2010, at 6:08 AM, Timo Sirainen wrote: On Wed, 2010-02-10 at 13:57 +, Stephen Usher wrote: I've written a program, which is run by cron once a minute, which searches for messages within all the users' inboxes and then hard-links them into a backup store elsewhere on the disk. Then, once a day a reaper process runs which checks for all files in that store with only one link and have not been accessed for a set amount of time (in our case 90 days) and then deletes those files. In other words, the messages are kept for 90 days after they have been deleted or moved out of the user's inbox. http://wiki.dovecot.org/Plugins/Lazyexpunge also does pretty much that, although for all mailboxes, not just for INBOX. And it doesn't automatically expire them. There's also rsnapshot which I've been using for hardlinked, incremental backups for a couple years now. Very flexible and has saved us a couple times. http://www.rsnapshot.org -Terry Terry Barnum digital OutPost San Diego, CA http://www.dop.com 800/464-6434 _ Hotmail: Trusted email with powerful SPAM protection. https://signup.live.com/signup.aspx?id=60969
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tue, 2 Feb 2010, CoolAtt NNA wrote: luigi, can you tell me what will happen if I create a directory in a user's mailbox and restore some emails there. now if the user's quota is x mb after restoration the mailbox becomes greater than x mb. will this cause any prob ? This depends on your quota calculation mechanism. As you mention maildirsize, see http://wiki.dovecot.org/Quota/Maildir You should remove that file, unless you use that file to store the user limit, Dovecot will recalculate the size then. I don't use Maildir++ quota, but Timo usually answers that way :-) the list archive should have several hints. === If you use the filesystem backend, the system enforces the quota. There are (had been) systems, that does not enforce quota, if the user gets overquota by non-user actions, e.g. when root chowns files. Regards, - -- Steffen Kaiser -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) iQEVAwUBS2frcL+Vh58GPL/cAQJEdQgApfXH2KQBVO4+N+26gA6asEMBAUhZz1Ux zhHnw6IHmvullfQlXU9d7Q1KYvLglzZtKhfBK9ykk5D70P+QuQY5YdVoL9JA3E5f 25nPSwv9rp56VrJqOaX//rDvmbFCW45rpa/golXsXV9vYo648oN3Z4ivxIbuKWpA +p/raG5llC7yhd41jegZW10U7WooG19DR987blNRSrV4yOlogug7nKvje7/Y22XK wsSPdm0hqX/SZrNlKaPb11iKbL2xprhmLRs9wE6TnlQM+zX2dx/rIosHH4S2+6P4 Jh4owUzvsN4jeY0yq6xrRZ1sqwl9Cewwg+xKKAmY81zfsHV6ZZKtMg== =luFR -END PGP SIGNATURE-
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
L.M.J linuxmasterj...@free.fr: Unfortunately, I'm using mbox :-/ In that case, --fuzzy got less interest but --delete-after is definitely a good one. If you're talking mbox then -fuzzy has no interest, that's right. But rsync is still a good match, actually a better match than with maildir. The rsync program is very good at only transferring the changed part of a file.
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
For the duplicates , when an email is flagged , am working on a script that will remove the duplicates. Am thinking about the following files: dovecot.index dovecot.index.cache dovecot.index.log maildirsize subscriptions dovecot-uidlist Will they affect the restore process ?? = CoolAtt = Date: Fri, 29 Jan 2010 14:20:05 +0100 From: al...@boxed.no To: dovecot@dovecot.org CC: t...@iki.fi; li...@luigirosa.com Subject: Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ? On 29.01.2010 08:59, Timo Sirainen wrote: On 29.1.2010, at 7.57, Luigi Rosa wrote: Timo Sirainen said the following on 29/01/10 06:48: The main potential problem with that is that if a message's flag is changed, the filename changes so it's now stored twice in the backup server. So when restoring mails, be sure not to restore duplicates. Using a different dedicated directory for restored messages could lead to the same problem? If you didn't give --delete parameter to rsync, there could be duplicates, yes. Dovecot just logs about them and gives new UIDs to them, so you'll end up having duplicate mails. I suppose some day I should just change it to log a warning, but delete if the files are otherwise identical. Using rsync will have the flags issue and others to be dealt with; lvm (or any other kind) of snapshot and rdiff-backup gives you a consistent snapshot, as well as the option of saving a lot of space and time as only changes gets backed up. As a bonus you can also restore anything to a given point in time, either induvidual messages or whole folders or even the whole mailbox. -A _ Hotmail: Powerful Free email with security by Microsoft. https://signup.live.com/signup.aspx?id=60969
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 CoolAtt NNA said the following on 02/02/10 08:02: Am thinking about the following files: dovecot.index dovecot.index.cache dovecot.index.log maildirsize subscriptions dovecot-uidlist Will they affect the restore process ?? Based on some experience, if you remove dovecot-uidlist AND the user uses POP3 AND the user leaves mail on server, the mail client could download the messages again because it sees them as unread. Ciao, luigi - -- / +--[Luigi Rosa]-- \ Computers don't actually think. You just think they think. (We think.) -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAktnz3EACgkQ3kWu7Tfl6ZQRmgCeI2NJ8Zu9wDsnNK0Qirw4NBEo LJYAoMImNr0Ni/cgt7VQcYCyW4+NuzEA =1MO/ -END PGP SIGNATURE-
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
luigi, can you tell me what will happen if I create a directory in a user's mailbox and restore some emails there. now if the user's quota is x mb after restoration the mailbox becomes greater than x mb. will this cause any prob ? Date: Tue, 2 Feb 2010 08:08:33 +0100 From: li...@luigirosa.com To: dovecot@dovecot.org Subject: Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ? -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 CoolAtt NNA said the following on 02/02/10 08:02: Am thinking about the following files: dovecot.index dovecot.index.cache dovecot.index.log maildirsize subscriptions dovecot-uidlist Will they affect the restore process ?? Based on some experience, if you remove dovecot-uidlist AND the user uses POP3 AND the user leaves mail on server, the mail client could download the messages again because it sees them as unread. Ciao, luigi - -- / +--[Luigi Rosa]-- \ Computers don't actually think. You just think they think. (We think.) -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAktnz3EACgkQ3kWu7Tfl6ZQRmgCeI2NJ8Zu9wDsnNK0Qirw4NBEo LJYAoMImNr0Ni/cgt7VQcYCyW4+NuzEA =1MO/ -END PGP SIGNATURE- _ Hotmail: Trusted email with powerful SPAM protection. https://signup.live.com/signup.aspx?id=60969
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 CoolAtt NNA said the following on 02/02/10 08:36: luigi, can you tell me what will happen if I create a directory in a user's mailbox and restore some emails there. now if the user's quota is x mb after restoration the mailbox becomes greater than x mb. will this cause any prob ? Unfortunately I do not have any experience with quota. I think that this scenario is like a situation where an external program (or a MTA) creates files in the maildir structure without using Dovecot API interface. I think that Dovecot sees the directory change and updates the quota data, but it's just a guess. Ciao, luigi - -- / +--[Luigi Rosa]-- \ A cynic is a man who, when he smells flowers, looks around for a coffin. --H.L. Mencken -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAktn12kACgkQ3kWu7Tfl6ZRAZgCeJIGzAXapzXt6DicafmW3MzPx yMQAn2u8GTjcQrGCBw94YcQBoSrG6Ukk =aqUw -END PGP SIGNATURE-
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
L.M.J linuxmasterj...@free.fr:
...
rsync -a --delete $i ${PATHTOSAFEPLACE}/$(date +%u)/${i}/
...
The downside to not using --fuzzy, is that the entire article is
transferred again if a message names changes slightly (ie. if we're
talking maildir here), as a result of status change.
Note if you decide to use --fuzzy and want to delete articles, you
should chose --delete-after. Otherwise rsync will delete the
candidates for fuzzy name matching, before having a chance to make use
of them.
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
Le Sun, 31 Jan 2010 21:04:12 +0100, Steinar Bang s...@dod.no a écrit : The downside to not using --fuzzy, is that the entire article is transferred again if a message names changes slightly (ie. if we're talking maildir here), as a result of status change. Note if you decide to use --fuzzy and want to delete articles, you should chose --delete-after. Otherwise rsync will delete the candidates for fuzzy name matching, before having a chance to make use of them. Unfortunately, I'm using mbox :-/ In that case, --fuzzy got less interest but --delete-after is definitely a good one. Thanks -- LMJ May the source be with you my young padawan http://sites.google.com/site/imatruelinuxmasterjedi/
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
On January 31, 2010 9:04:12 PM +0100 Steinar Bang s...@dod.no wrote:
L.M.J linuxmasterj...@free.fr:
...
rsync -a --delete $i ${PATHTOSAFEPLACE}/$(date +%u)/${i}/
...
The downside to not using --fuzzy, is that the entire article is
transferred again if a message names changes slightly (ie. if we're
talking maildir here), as a result of status change.
zfs really shines here. Only the filename change would be transmitted.
And if you're using opensolaris you can add in AVS to get real time
continous synchronous replication if you want. (I would also consider
having the MTA duplicate the mail to a 2nd mail server and use another
method to synchronize deletions and flag changes.)
-frank
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
Timo Sirainen t...@iki.fi: The main potential problem with that is that if a message's flag is changed, the filename changes so it's now stored twice in the backup server. So when restoring mails, be sure not to restore duplicates. I use the --fuzzy flag to handle the renaming issue. Here's my crontab entry on the machine holding the backup: 0 * * * * rsync --archive --fuzzy --delete-after imapserver:Maildir . As you can see I only sync once an hour. I don't have enough email traffic to justify doing it continously. The --delete-after is so rsync won't delete files that have been removed on the source, before figuring out that a file has been renamed and not been deleted.
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
On Fri, Jan 29, 2010 at 10:21:39AM +0500, CoolAtt NNA wrote: Hi Alex, I mirror the mailboxes as follows: rsync -avt --log-file=$log_file /home/vmail/ 10.0.0.x:/home/vmail I run it in a script for e,g every 1 min. If running this from cron every minute, make sure to use a lockfile or something to make sure you don't end up with multiple rsync jobs running at the same time... Geert -- Geert Hendrickx -=- g...@telenet.be -=- PGP: 0xC4BB9E9F This e-mail was composed using 100% recycled spam messages!
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
For email email am having the email duplication issue when am doing the backup. Am having multiple copies of a mail file on the backup server. Does it mean that if I use --fuzzy --delete-after only the most recent mailfile will be on the backup server ?? e.g On LIVE server : my-file:2,S On BACKUP server : my-file:2,S + my-file:2, + my-file:2,SF my-file:2,S is the current file. So on the backup only my-file:2,S must exist. Will it solve the prob ?? To: dovecot@dovecot.org From: s...@dod.no Date: Fri, 29 Jan 2010 08:58:27 +0100 Subject: Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ? Timo Sirainen t...@iki.fi: The main potential problem with that is that if a message's flag is changed, the filename changes so it's now stored twice in the backup server. So when restoring mails, be sure not to restore duplicates. I use the --fuzzy flag to handle the renaming issue. Here's my crontab entry on the machine holding the backup: 0 * * * * rsync --archive --fuzzy --delete-after imapserver:Maildir . As you can see I only sync once an hour. I don't have enough email traffic to justify doing it continously. The --delete-after is so rsync won't delete files that have been removed on the source, before figuring out that a file has been renamed and not been deleted. _ Hotmail: Powerful Free email with security by Microsoft. https://signup.live.com/signup.aspx?id=60969
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
I tried the --fuzzy --delete-after options. It works fine when the files are renamed. But the prob is that when a mail is deleted it gets deleted from the backup server as well. Is there a way to prevent deletion when using these options? Thanks CoolAtt To: dovecot@dovecot.org From: s...@dod.no Date: Fri, 29 Jan 2010 08:58:27 +0100 Subject: Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ? Timo Sirainen t...@iki.fi: The main potential problem with that is that if a message's flag is changed, the filename changes so it's now stored twice in the backup server. So when restoring mails, be sure not to restore duplicates. I use the --fuzzy flag to handle the renaming issue. Here's my crontab entry on the machine holding the backup: 0 * * * * rsync --archive --fuzzy --delete-after imapserver:Maildir . As you can see I only sync once an hour. I don't have enough email traffic to justify doing it continously. The --delete-after is so rsync won't delete files that have been removed on the source, before figuring out that a file has been renamed and not been deleted. _ Hotmail: Trusted email with Microsoft’s powerful SPAM protection. https://signup.live.com/signup.aspx?id=60969
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
On 29.01.2010 08:59, Timo Sirainen wrote: On 29.1.2010, at 7.57, Luigi Rosa wrote: Timo Sirainen said the following on 29/01/10 06:48: The main potential problem with that is that if a message's flag is changed, the filename changes so it's now stored twice in the backup server. So when restoring mails, be sure not to restore duplicates. Using a different dedicated directory for restored messages could lead to the same problem? If you didn't give --delete parameter to rsync, there could be duplicates, yes. Dovecot just logs about them and gives new UIDs to them, so you'll end up having duplicate mails. I suppose some day I should just change it to log a warning, but delete if the files are otherwise identical. Using rsync will have the flags issue and others to be dealt with; lvm (or any other kind) of snapshot and rdiff-backup gives you a consistent snapshot, as well as the option of saving a lot of space and time as only changes gets backed up. As a bonus you can also restore anything to a given point in time, either induvidual messages or whole folders or even the whole mailbox. -A
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
Le Thu, 28 Jan 2010 13:07:32 +0500,
CoolAtt NNA cool...@live.com a écrit :
Am working on continous backup of mailboxes using rsync(for e.g by running
rsync every 2 min)
Things gets more complicated when users create Subfolders in INBOX , SENT ,
etc..
If anyone among you did that plz guide advise.
I'm going to share what I've did. This is not rocket science but enough for my
SOHO needs.
Each user mail boxes are stored in ~HOME/mail/
# cat /opt/crontab-scripts/rsync-local-email-backup.sh
[...]
for i in /home/*/mail/
rsync -a --delete $i ${PATHTOSAFEPLACE}/$(date +%u)/${i}/
RETURNCODE=$?
[...]
done
[...]
In clear, each day, I copy all mailboxes to a different (date +%u - 0 to 6)
safe sub-directory.
I admit, it's very simple but enough for my needs and certainly could be
improved..
Best regards,
--
LMJ
May the source be with you my young padawan
http://sites.google.com/site/imatruelinuxmasterjedi/
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
The answer, if you can sell the cost, might be network storage. Most of them provide the capability of take inode snapshots CoolAtt NNA wrote: Hi All.. Am working on continous backup of mailboxes using rsync(for e.g by running rsync every 2 min) Things gets more complicated when users create Subfolders in INBOX , SENT , etc.. If anyone among you did that plz guide advise. Thanks CoolAtt _ Your E-mail and More On-the-Go. Get Windows Live Hotmail Free. https://signup.live.com/signup.aspx?id=60969 -- Once upon a time, the Internet was a friendly, neighbors-helping-neighbors small town, and no one locked their doors. Now it's like an apartment in Bed-Stuy: you need three heavy duty pick-proof locks, one of those braces that goes from the lock to the floor, and bars on the windows Stewart Dean, Unix System Admin, Bard College, New York 12504 sd...@bard.edu voice: 845-758-7475, fax: 845-758-7035
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
On 28.01.2010 09:07, CoolAtt NNA wrote: Hi All.. Am working on continous backup of mailboxes using rsync(for e.g by running rsync every 2 min) Things gets more complicated when users create Subfolders in INBOX , SENT , etc.. If anyone among you did that plz guide advise. You don't state much on what this runs on, but I do snapshot backups of my IMAP server consistently by using LVM, and just match my cronjob to snapshot it at a few given times, mounting the snapshot and backing up with rdiff-backup. Very simple, very easy, and very realiable. :) Thanks CoolAtt -A
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
On 1/28/10 12:07 AM, CoolAtt NNA wrote: Hi All.. Am working on continous backup of mailboxes using rsync(for e.g by running rsync every 2 min) Things gets more complicated when users create Subfolders in INBOX , SENT , etc.. How? If anyone among you did that plz guide advise. Sure: rsync -e 'ssh -ax -i mykey' \ -a --delete --stats \ mailserver.host:/var/mail/vmail \ /localbackup ~Seth
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
Hi Alex, I mirror the mailboxes as follows: rsync -avt --log-file=$log_file /home/vmail/ 10.0.0.x:/home/vmail I run it in a script for e,g every 1 min. By doing so if an email is deleted on the live server it still remain on the backup server. Then I give each mail file a retention period of n days so that it automatically gets deleted from the backup server. *** CoolAtt *** Date: Thu, 28 Jan 2010 20:57:15 +0100 From: al...@boxed.no To: dovecot@dovecot.org Subject: Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ? On 28.01.2010 09:07, CoolAtt NNA wrote: Hi All.. Am working on continous backup of mailboxes using rsync(for e.g by running rsync every 2 min) Things gets more complicated when users create Subfolders in INBOX , SENT , etc.. If anyone among you did that plz guide advise. You don't state much on what this runs on, but I do snapshot backups of my IMAP server consistently by using LVM, and just match my cronjob to snapshot it at a few given times, mounting the snapshot and backing up with rdiff-backup. Very simple, very easy, and very realiable. :) Thanks CoolAtt -A _ Hotmail: Trusted email with Microsoft’s powerful SPAM protection. https://signup.live.com/signup.aspx?id=60969
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 CoolAtt NNA said the following on 29/01/10 06:21: rsync -avt --log-file=$log_file /home/vmail/ 10.0.0.x:/home/vmail I run it in a script for e,g every 1 min. By doing so if an email is deleted on the live server it still remain on the backup server. Then I give each mail file a retention period of n days so that it automatically gets deleted from the backup server. I use a similar strategy, with a more relaxed synchronization. But instead of setting an expire on backup system, I simply do a rsync --del every night. Ciao, luigi - -- / +--[Luigi Rosa]-- \ Oo-ta goo-ta, Solo? --Greedo, Star Wars -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAktidI8ACgkQ3kWu7Tfl6ZRfEACfRfJR4z99TIEgylKSywVAa4px l1kAnRA6xyigbYl+BVZoi6d5Vi5/n4Yk =f8lP -END PGP SIGNATURE-
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
Hi Luigi, Thats great. Can you please explain how you do the restore? I plan to do restore of 1 or more mail files. If I simply copy a mailfile e.g MailDir/cur/my-email-file:2,S to MailDir/cur/ on the live system, are there issues associated with it ? Thanks CoolAtt Date: Fri, 29 Jan 2010 06:39:31 +0100 From: li...@luigirosa.com To: dovecot@dovecot.org Subject: Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ? -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 CoolAtt NNA said the following on 29/01/10 06:21: rsync -avt --log-file=$log_file /home/vmail/ 10.0.0.x:/home/vmail I run it in a script for e,g every 1 min. By doing so if an email is deleted on the live server it still remain on the backup server. Then I give each mail file a retention period of n days so that it automatically gets deleted from the backup server. I use a similar strategy, with a more relaxed synchronization. But instead of setting an expire on backup system, I simply do a rsync --del every night. Ciao, luigi - -- / +--[Luigi Rosa]-- \ Oo-ta goo-ta, Solo? --Greedo, Star Wars -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAktidI8ACgkQ3kWu7Tfl6ZRfEACfRfJR4z99TIEgylKSywVAa4px l1kAnRA6xyigbYl+BVZoi6d5Vi5/n4Yk =f8lP -END PGP SIGNATURE- _ Hotmail: Trusted email with Microsoft’s powerful SPAM protection. https://signup.live.com/signup.aspx?id=60969
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
On Fri, 2010-01-29 at 10:21 +0500, CoolAtt NNA wrote: Hi Alex, I mirror the mailboxes as follows: rsync -avt --log-file=$log_file /home/vmail/ 10.0.0.x:/home/vmail I run it in a script for e,g every 1 min. By doing so if an email is deleted on the live server it still remain on the backup server. Then I give each mail file a retention period of n days so that it automatically gets deleted from the backup server. The main potential problem with that is that if a message's flag is changed, the filename changes so it's now stored twice in the backup server. So when restoring mails, be sure not to restore duplicates. signature.asc Description: This is a digitally signed message part
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Timo Sirainen said the following on 29/01/10 06:48: The main potential problem with that is that if a message's flag is changed, the filename changes so it's now stored twice in the backup server. So when restoring mails, be sure not to restore duplicates. Using a different dedicated directory for restored messages could lead to the same problem? Ciao, luigi - -- / +--[Luigi Rosa]-- \ Dr. Frederick Frankenstein: What a filthy job. Igor: Could be worse. Dr. Frederick Frankenstein: How? Igor: Could be raining. --Young Frankenstein -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAktieNIACgkQ3kWu7Tfl6ZT4CgCgx47smwd+UkL+UsgeAviqj6pq UacAmwa/Fy4zF6FTUc6MoWxvDAorzk2q =xnhw -END PGP SIGNATURE-
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
It means if I create a new mailbox and I restore the emails at mail level(individual files) ,will it work correctly? And if I restore the same mail to an existing mailbox ?? Thanks CoolAtt Date: Fri, 29 Jan 2010 06:56:06 +0100 From: li...@luigirosa.com To: dovecot@dovecot.org Subject: Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ? -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 CoolAtt NNA said the following on 29/01/10 06:47: Can you please explain how you do the restore? I always restore at folder level, not mail level: in this case is the user that choose what to restore and I don't need to read the user's mail. I create a .Restore directory and copy into it the restored folder files: the users simply sees a Restore folder and do whatevere he/she wants. If I simply copy a mailfile e.g MailDir/cur/my-email-file:2,S to MailDir/cur/ on the live system, are there issues associated with it ? With a standard configuration there are no issues at all. Check that the permissions and user/group of restored files are ok. Ciao, luigi - -- / +--[Luigi Rosa]-- \ A government big enough to give you everything you want is big enough to take from you everything you have. --Gerald R. Ford -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAktieHYACgkQ3kWu7Tfl6ZSktwCdHZqyD8+q3hOR43OYhz+qPxiC jisAoKIcZEQ+vL1RQnInjByzUZb5yvEg =riZr -END PGP SIGNATURE- _ Hotmail: Trusted email with powerful SPAM protection. https://signup.live.com/signup.aspx?id=60969
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 CoolAtt NNA said the following on 29/01/10 07:18: It means if I create a new mailbox and I restore the emails at mail level(individual files) ,will it work correctly? And if I restore the same mail to an existing mailbox ?? Not a new mailbox, a new mail folder (maildir directory) of the actual account. I am assuming you are using IMAP ad maildir, of course. Ciao, luigi - -- / +--[Luigi Rosa]-- \ Scott me up, Beamie! -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAktifjUACgkQ3kWu7Tfl6ZQalACfaEsS0ME6ab27uks49W6ERj59 yAcAnRDmaN3ZCKAEzJrK41CjZK28B3aT =O5hj -END PGP SIGNATURE-
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
Am using Dovecot the maildir format. Create a new folder like the following ?? vmail/coolatt/Maildir/restored-emails Date: Fri, 29 Jan 2010 07:20:37 +0100 From: li...@luigirosa.com To: dovecot@dovecot.org Subject: Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ? -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 CoolAtt NNA said the following on 29/01/10 07:18: It means if I create a new mailbox and I restore the emails at mail level(individual files) ,will it work correctly? And if I restore the same mail to an existing mailbox ?? Not a new mailbox, a new mail folder (maildir directory) of the actual account. I am assuming you are using IMAP ad maildir, of course. Ciao, luigi - -- / +--[Luigi Rosa]-- \ Scott me up, Beamie! -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAktifjUACgkQ3kWu7Tfl6ZQalACfaEsS0ME6ab27uks49W6ERj59 yAcAnRDmaN3ZCKAEzJrK41CjZK28B3aT =O5hj -END PGP SIGNATURE- _ Hotmail: Powerful Free email with security by Microsoft. https://signup.live.com/signup.aspx?id=60969
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 CoolAtt NNA said the following on 29/01/10 07:39: Create a new folder like the following ?? vmail/coolatt/Maildir/restored-emails Actually vmail/coolatt/Maildir/.restored-emails would be better :) Done that, you copy from the folder to be restored cur, tmp, new. Some email clients need to shutdown and restart to be aware of the new mail folder. Ciao, luigi - -- / +--[Luigi Rosa]-- \ Moneypenny: In the conference room. Something pretty big. Every double-o man in Europe has been rushed in. And the home secretary too! James Bond: His wife probably lost her dog. --007 Thunderball -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAktihLQACgkQ3kWu7Tfl6ZSMzQCghKGCWeqflRH4SatM/YItp3HG Ba0AoMFM4CVAdHtvsjNv7k0FLWFeYlw+ =6eIt -END PGP SIGNATURE-
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
On Thu, 28 Jan 2010 13:07:32 +0500 CoolAtt NNA cool...@live.com wrote: Hi All.. Am working on continous backup of mailboxes using rsync(for e.g by running rsync every 2 min) Things gets more complicated when users create Subfolders in INBOX , SENT , etc.. If anyone among you did that plz guide advise. Thanks CoolAtt Take a look on ccollect project http://unix.schottelius.org/ccollect/. It made incremental backups using rsync. That will be working for maildirs perfectly. _ Your E-mail and More On-the-Go. Get Windows Live Hotmail Free. https://signup.live.com/signup.aspx?id=60969
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
On 29.1.2010, at 7.57, Luigi Rosa wrote: Timo Sirainen said the following on 29/01/10 06:48: The main potential problem with that is that if a message's flag is changed, the filename changes so it's now stored twice in the backup server. So when restoring mails, be sure not to restore duplicates. Using a different dedicated directory for restored messages could lead to the same problem? If you didn't give --delete parameter to rsync, there could be duplicates, yes. Dovecot just logs about them and gives new UIDs to them, so you'll end up having duplicate mails. I suppose some day I should just change it to log a warning, but delete if the files are otherwise identical.
