Re: [Dovecot] prevent dovecot from modifying maildirs
On Fri, 2013-08-09 at 21:12 +0200, Felix Rubio Dalmau wrote: Michael, Charles, thank you for your comments. I had already seen the wikipedia entries about ZFS, but when I have performed my tests, consisting in: generate a program that writes a 100 MB file to disk and perform the snapshot during the file is being written. Obviously the snapshot shows a view of the file with the information and size it had at the time of the snapshot... so I get a portion of it. In order to save space I prefer no to have such half-files (mails) , and it is for this reason that I am asking if there is any way to prevent this. If you don't disable mail_fsync, you won't get partially written mails. And I'm not sure if you get them even if you do disable mail_fsync. The mails are first written to temporary files, fsynced, then renamed to the final name. So you might end up with some temporary files, but they won't be visible and they'll be deleted automatically. The above applies to all mailbox formats except mbox.
Re: [Dovecot] prevent dovecot from modifying maildirs
Great! Thank you very much, Timo! Felix On Sunday 11 August 2013 22:49:43 Timo Sirainen wrote: On Fri, 2013-08-09 at 21:12 +0200, Felix Rubio Dalmau wrote: Michael, Charles, thank you for your comments. I had already seen the wikipedia entries about ZFS, but when I have performed my tests, consisting in: generate a program that writes a 100 MB file to disk and perform the snapshot during the file is being written. Obviously the snapshot shows a view of the file with the information and size it had at the time of the snapshot... so I get a portion of it. In order to save space I prefer no to have such half-files (mails) , and it is for this reason that I am asking if there is any way to prevent this. If you don't disable mail_fsync, you won't get partially written mails. And I'm not sure if you get them even if you do disable mail_fsync. The mails are first written to temporary files, fsynced, then renamed to the final name. So you might end up with some temporary files, but they won't be visible and they'll be deleted automatically. The above applies to all mailbox formats except mbox.
Re: [Dovecot] prevent dovecot from modifying maildirs
On 2013-08-09 5:44 AM, Felix Rubio Dalmau felixrubiodal...@gmail.com wrote: Is there any possibility to ask dovecot to write nothing to disk, something like a read only state, without shutting it down? I have set up a ZFS+postfix+dovecot mail server, and I need both postfix (postsuper -h ALL) and dovecot (¿?) to stop modifying the disk to create the snapshot to be saved. ??? That is supposed to be (haven't used it yet) one of ZFS's strong points. You don't need to worry about this, it is all handled by the OS. If you think you do need to worry about it, then you'll have to come up with a very good reason why. -- Best regards, */Charles/*
Re: [Dovecot] prevent dovecot from modifying maildirs
Felix Rubio Dalmau skrev den 2013-08-09 11:44: Does anybody has any idea on how to achieve this? stop dovecot/postfix, maybe remember sql/ldap database aswell
Re: [Dovecot] prevent dovecot from modifying maildirs
On 2013-08-09 11:44, Felix Rubio Dalmau wrote: Is there any possibility to ask dovecot to write nothing to disk, something like a read only state, without shutting it down? I have set up a ZFS+postfix+dovecot mail server, and I need both postfix (postsuper -h ALL) and dovecot (¿?) to stop modifying the disk to create the snapshot to be saved. Does anybody has any idea on how to achieve this? As mentioned before, that's done by ZFS. Just create a snapshot using your ZFS functionality (zfs snapshot name). Regards, Michael
Re: [Dovecot] prevent dovecot from modifying maildirs
On 2013-08-09 8:04 AM, Michael Grimm trash...@odo.in-berlin.de wrote: On 2013-08-09 11:44, Felix Rubio Dalmau wrote: Is there any possibility to ask dovecot to write nothing to disk, something like a read only state, without shutting it down? I have set up a ZFS+postfix+dovecot mail server, and I need both postfix (postsuper -h ALL) and dovecot (¿?) to stop modifying the disk to create the snapshot to be saved. Does anybody has any idea on how to achieve this? As mentioned before, that's done by ZFS. Just create a snapshot using your ZFS functionality (zfs snapshot name). And the same goes for LVM snapshots. The only thing you might want/need to worry about is SQL databases... for these I always do periodic dumps (how often dicatated by how long the dumps take vs how often the data changes vs how critical the data is). -- Best regards, */Charles/*
Re: [Dovecot] prevent dovecot from modifying maildirs
Actually the problem is that the snapshot is instantaneous, so if some commands are being run at the time of the snapshot I could get a snapshot of a non-consistent FS, right? Then, the single solution goes through just shuting down dovecot (and other services, like apache) the seconds required to perform the snapshot, whilst holding the postfix queues to ensure not loosing any mail. Does anybody have a better approach? Regards! Felix On Friday 09 August 2013 08:19:59 Charles Marcus wrote: On 2013-08-09 8:04 AM, Michael Grimm trash...@odo.in-berlin.de wrote: On 2013-08-09 11:44, Felix Rubio Dalmau wrote: Is there any possibility to ask dovecot to write nothing to disk, something like a read only state, without shutting it down? I have set up a ZFS+postfix+dovecot mail server, and I need both postfix (postsuper -h ALL) and dovecot (¿?) to stop modifying the disk to create the snapshot to be saved. Does anybody has any idea on how to achieve this? As mentioned before, that's done by ZFS. Just create a snapshot using your ZFS functionality (zfs snapshot name). And the same goes for LVM snapshots. The only thing you might want/need to worry about is SQL databases... for these I always do periodic dumps (how often dicatated by how long the dumps take vs how often the data changes vs how critical the data is). smime.p7s Description: S/MIME cryptographic signature
Re: [Dovecot] prevent dovecot from modifying maildirs
On 2013-08-09 1:43 PM, Felix Rubio Dalmau felixrubiodal...@gmail.com wrote: Actually the problem is that the snapshot is instantaneous, so if some commands are being run at the time of the snapshot I could get a snapshot of a non-consistent FS, right? No, that is not right. If it was it would totally defeat the entire purpose of snapshots (which *is* to get a consistent view of the filesystem). Then, the single solution goes through just shuting down dovecot (and other services, like apache) the seconds required to perform the snapshot, whilst holding the postfix queues to ensure not loosing any mail. Does anybody have a better approach? Yes - read up on how snapshots work (in your case ZFS snapshots) and stop wasting brain cycles worrying about a non existent problem. As I said earlier, the only exception to this consistency question is with SQL databases. -- Best regards, */Charles /*
Re: [Dovecot] prevent dovecot from modifying maildirs
On 09.08.2013, at 19:43, Felix Rubio Dalmau felixrubiodal...@gmail.com wrote: Actually the problem is that the snapshot is instantaneous, so if some commands are being run at the time of the snapshot I could get a snapshot of a non-consistent FS, right? As mentioned before: Wrong. Have a look at http://en.wikipedia.org/wiki/Zfs#Snapshots_and_clones and http://en.wikipedia.org/wiki/Snapshot_(computer_storage) and alike. Regards, Michael
Re: [Dovecot] prevent dovecot from modifying maildirs
Michael, Charles, thank you for your comments. I had already seen the wikipedia entries about ZFS, but when I have performed my tests, consisting in: generate a program that writes a 100 MB file to disk and perform the snapshot during the file is being written. Obviously the snapshot shows a view of the file with the information and size it had at the time of the snapshot... so I get a portion of it. In order to save space I prefer no to have such half-files (mails) , and it is for this reason that I am asking if there is any way to prevent this. I'm not an expert in field of file systems, so I can be asking very naive questions... but I appreciate your comments on this topic. Thank you Felix On Friday 09 August 2013 20:39:31 Michael Grimm wrote: On 09.08.2013, at 19:43, Felix Rubio Dalmau felixrubiodal...@gmail.com wrote: Actually the problem is that the snapshot is instantaneous, so if some commands are being run at the time of the snapshot I could get a snapshot of a non-consistent FS, right? As mentioned before: Wrong. Have a look at http://en.wikipedia.org/wiki/Zfs#Snapshots_and_clones and http://en.wikipedia.org/wiki/Snapshot_(computer_storage) and alike. Regards, Michael
Re: [Dovecot] prevent dovecot from modifying maildirs
In order to save space I prefer no to have such half-files (mails) , and it is for this reason that I am asking if there is any way to prevent this. I'm not an expert in field of file systems, so I can be asking very naive questions... but I appreciate your comments on this topic. Filesystem consistency is different than, for lack of a better term, dovecot consistency. The latter places additional constraints on the content of files, rather than treating them as sequences of disk blocks. I don't think you can ensure the latter without stopping all dovecot processes (and maybe the MTA/LDA as well), which will be much more difficult and disruptive than just taking a snapshot. (I think some snapshot aware application like databases and hypervisors allows the operator to put the application in a state of acquiesence, long enough to take a snapshot, then unblock all processes after the snapshot.) I'd take the suggestion of the previous poster: take snapshots, don't worry, be happy. The small amount of inconsistencies is trivial compared with quickly recovering a mail store after disaster. Joseph Tam jtam.h...@gmail.com
