On 7/1/20 3:58 PM, Robert Haas wrote:
On Wed, Jul 1, 2020 at 3:50 PM Magnus Hagander <mag...@hagander.net> wrote:
As far as I've seen, the one thing that people have problems with in the
exclusive mode backups are precisely the fact that they have to keep a
persistent conneciton open, and thus it cannot work together with backup
software that is limited to only supporting running a pre- and a post script.
Something like I have suggested here is to solve *that* problem. I don't think anybody
actually explicitly wants "exclusive backups" -- they want a backup solution
that plugs into their world of pre/post scripts. And if we can make that one work in a
safer way than the current exclusive backups, ohw is that not an improvement?
Yeah, I guess that's a pretty fair point. I have to confess to having
somewhat limited enthusiasm for adding a third mode here, but it might
be worth it.
It seems pretty well inevitable to me that people are going to forget
to end them. I am not sure exactly what the consequences of that will
be, but if for example there's a limited number of shared memory slots
to store information about these backups, then if you leak any, you'll
eventually run out of slots and your backups will start failing. I
feel like that's a going to happen to about 75% of the people who try
to use this new backup mode at some point in time, but maybe I'm a
pessimist.[1]
I did consider going down this road and I came to more or less the same
conclusion. Backups would accumulate until we just ran out of space
and/or started throwing errors at some limit. Hardly very attractive.
Also, full pages writes will be left on indefinitely.
If we could jigger things so that you don't need to stop the backup at
all, you only start it, and whether you ever finish copying everything
is something about which the system need not know or care, that would
be a lot nicer. I'm not sure I see how to do that, though.
Well, the only thing pg_stop_backup() *really* needs to know is the
starting WAL position. pg_start_backup() gets that info so if it passes
it back to pg_stop_backup() that could be enough. Or as was proposed
above, it just passes the backup_label back to pg_stop_backup() for
parsing. To write the .backup file to WAL (which I personally think is
pretty useless) you'd need backup_label.
The issue I have not been able to work around is that full page writes
need to be on for backups and they won't get turned back off if you
don't end all the backups that get started.
Here's a thought. What if we just stored the oldest starting LSN and a
count of how many backups have been requested. When the backup ends it
checks that backup count is > 0 and starting LSN is <= its starting LSN.
If not, it throws an error. When backups go to 0 FPWs are turned off if
they were off before the first backup.
That way, we could have a single function to cancel all backups in
progress. Even if a new one started, one ending that was started prior
to the cancel would know that it could not end successfully and error.
For the vast majority of people who have full page writes on, it
wouldn't really matter how many backups were running so we could
probably just skip all of that.
If we wanted to really make it more foolproof we could add a checksum to
the backup label to make sure the user doesn't damage it before passing
it back.
Regards,
--
-David
da...@pgmasters.net
