Re: Recommended way to pause postfix local delivery while taking snapshot for backup
My 2 cents. I had a Digital Ocean snapshot that turned out to be unbootable. (I have no explanation.) My snapshot procedure since then is I shut down all the services, power down, take the snapshot and restore from that snapshot. Me paranoid? Yep. I also keep two snapshots, both of which have been proven to be bootable. You would think an image of a droplet that was running should be bootable, but real life can be full of surprises. Regarding rsync, at least you won't lose user data and configuration files. But restoration from rsync still sounds like work to me. Digital Ocean had one situation where a backup was bad. https://murze.be/2016/02/today-digitalocean-lost-our-entire-server/ I'm picking on Digital Ocean, but Vultr and Linode have similar incidents. If my flow was to do powered (operating) images...well I wouldn't. I'd go for rsync. I would have a script scheme to rebuild (reconfigure) the OS from a stock droplet.
Re: Recommended way to pause postfix local delivery while taking snapshot for backup
On 09/04/17 20:52, Dominic Raferd wrote: > Is there a best/recommended way to pause postfix local deliveries so > that I can take an LVM snapshot of the local mails for backup purposes? > The pause only has to be momentary, while the snapshot is taken, but the > files need to be in a consistent state. If anyone also knows the way to > pause Dovecot imap/pop3 similarly (as this could also be accessing the > same files), that would be helpful too. I'm fairly sure you can just snapshot and any mail that has at least been queued will be safe, either in the queue or delivered. There is a small chance that the snapshot will be taken in between the point where the message is delivered and when it is removed from the queue in which case you can end up with a duplicate of that message when you restore as postfix will see the message in the queue and deliver it again. This is all minor in comparison to the mail that would be lost after the snapshot is taken and when the system dies before you restore (after the system dies you're actually safe because sending MXes will defer their mail to you). No amount of pausing during the snapshot will recover mail that you accepted after the snapshot. Even if you were to loose any mail that was in postfix during the snapshot (which won't happen) it's still negligible compared to what you'll loose after the snapshot. So at the end of the day I really wouldn't worry so much about this. Take your snapshot, and take your backup and understand that if you ever get to the point where everything blows up you *will* loose some mail, there is no real way around that. Peter
Re: Recommended way to pause postfix local delivery while taking snapshot for backup
Thanks to all for the replies. This is an existing somewhat elderly setup and I don't want to make major changes (file system, LMTP etc) at this stage. For postfix, Wietse's suggestion should work nicely. I guess I should build in a delay of a second or two after issuing the command to ensure that any local deliveries in progress are completed before the snapshot is taken - this is not a very big/busy server. For dovecot well I will just stop it. So something like this: postconf defer_transports=local postfix reload sleep 2s /etc/init.d/dovecot stop # (this is on Debian 6.0, no systemd) # take lvm snapshot here, then: postconf defer_transports= postfix reload /etc/init.d/dovecot start # backup from snapshot can now proceed On 9 April 2017 at 15:29, Viktor Dukhovni wrote: > > > On Apr 9, 2017, at 4:52 AM, Dominic Raferd > wrote: > > > > Is there a best/recommended way to pause postfix local deliveries so that > > I can take an LVM snapshot of the local mails for backup purposes? > > The "best" way is to use a file system that supports snapshots, such as > "zfs". Otherwise, what Wietse said about "defer_transports". Covers > the Postfix part. > > > The pause only has to be momentary, while the snapshot is taken, but the > > files need to be in a consistent state. If anyone also knows the way to > > pause Dovecot imap/pop3 similarly (as this could also be accessing the > > same files), that would be helpful too. > > You'd have to stop Dovecot, terminating all active connections, or use > a file-system that supports snapshots. > > With maildir, you can probably not worry too much about consistency, > and just do periodic rsync without stopping Postfix or Dovecot. > > -- > Viktor. > >
Re: Recommended way to pause postfix local delivery while taking snapshot for backup
> On Apr 9, 2017, at 4:52 AM, Dominic Raferd wrote: > > Is there a best/recommended way to pause postfix local deliveries so that > I can take an LVM snapshot of the local mails for backup purposes? The "best" way is to use a file system that supports snapshots, such as "zfs". Otherwise, what Wietse said about "defer_transports". Covers the Postfix part. > The pause only has to be momentary, while the snapshot is taken, but the > files need to be in a consistent state. If anyone also knows the way to > pause Dovecot imap/pop3 similarly (as this could also be accessing the > same files), that would be helpful too. You'd have to stop Dovecot, terminating all active connections, or use a file-system that supports snapshots. With maildir, you can probably not worry too much about consistency, and just do periodic rsync without stopping Postfix or Dovecot. -- Viktor.
Re: Recommended way to pause postfix local delivery while taking snapshot for backup
Dominic Raferd: > Is there a best/recommended way to pause postfix local deliveries so that I > can take an LVM snapshot of the local mails for backup purposes? The pause > only has to be momentary, while the snapshot is taken, but the files need > to be in a consistent state. If anyone also knows the way to pause Dovecot > imap/pop3 similarly (as this could also be accessing the same files), that > would be helpful too. # postconf defer_transports=xxx # postfix reload For a suitable value of xxx (name in master.cf). This tells the queue manager to stop generating delivery requests. It does not suspend a delivery request that is already in progress; by design, Postfix has no single program that knows 'everything'. As documented, defer_transports also does not block forced delivery commands such as "postfix flush" and "sendmail -q". Wietse
Re: Recommended way to pause postfix local delivery while taking snapshot for backup
On 2017-04-09 10:52:58 (+0200), Dominic Raferd wrote: Is there a best/recommended way to pause postfix local deliveries so that I can take an LVM snapshot of the local mails for backup purposes? The pause only has to be momentary, while the snapshot is taken, but the files need to be in a consistent state. If anyone also knows the way to pause Dovecot imap/pop3 similarly (as this could also be accessing the same files), that would be helpful too. You can kill two birds with one stone if you deliver mails using LMTP instead of letting Postfix local(8) deliver directly to mailboxes. As I understand it, Dovecot's LMTP implementation will ensure the mail store is always in a consistent state: either a message will be stored and indexed or it won't. If you want to ensure no deliveries are attempted while you're taking a snapshot, running LMTP over a TCP socket and blocking new connections with the firewall would do it. Postfix will queue messages for later delivery when it can't connect to the LMTP server. Philip -- Philip Paeps Senior Reality Engineer Ministry of Information
Re: Recommended way to pause postfix local delivery while taking snapshot for backup
Am 09.04.2017 um 10:52 schrieb Dominic Raferd: > Is there a best/recommended way to pause postfix local deliveries so > that I can take an LVM snapshot of the local mails for backup purposes? > The pause only has to be momentary, while the snapshot is taken, but the > files need to be in a consistent state. If anyone also knows the way to > pause Dovecot imap/pop3 similarly (as this could also be accessing the > same files), that would be helpful too. why not simply write copies of all mails via bcc as archive and/or use dsync or rsync to write backups with dovecot ? Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Schleißheimer Straße 26/MG, 80333 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
Recommended way to pause postfix local delivery while taking snapshot for backup
Is there a best/recommended way to pause postfix local deliveries so that I can take an LVM snapshot of the local mails for backup purposes? The pause only has to be momentary, while the snapshot is taken, but the files need to be in a consistent state. If anyone also knows the way to pause Dovecot imap/pop3 similarly (as this could also be accessing the same files), that would be helpful too.
