Remove Deprecated Exclusive Backup Mode

2018-11-26 Thread David Steele
Hackers, I propose we remove the deprecated exclusive backup mode of pg_start_backup()/pg_stop_backup() for Postgres 12. The exclusive backup mode has a serious known issue. If Postgres terminates ungracefully during a backup (due to hardware, kernel, Postgres issues, etc.) then Postgres ma

Re: Remove Deprecated Exclusive Backup Mode

2020-06-30 Thread David Steele
Steele Date: Tue, 30 Jun 2020 17:23:51 -0400 Subject: [PATCH] Remove Deprecated Exclusive Backup Mode. Remove the exclusive backup mode which has been deprecated since PostgreSQL 9.6. --- doc/src/sgml/backup.sgml | 150 +- doc/src/sgml/func.sgml

Re: Remove Deprecated Exclusive Backup Mode

2020-06-30 Thread Stephen Frost
Greetings, * David Steele (da...@pgmasters.net) wrote: > Lastly, there is some concern about getting the backup label too late when > doing snapshots or using traditional backup software. It would certainly be > possible to return the backup_label and tablespace_map from > pg_start_backup() and le

Re: Remove Deprecated Exclusive Backup Mode

2020-06-30 Thread David Steele
On 6/30/20 6:13 PM, Stephen Frost wrote: Greetings, * David Steele (da...@pgmasters.net) wrote: Lastly, there is some concern about getting the backup label too late when doing snapshots or using traditional backup software. It would certainly be possible to return the backup_label and tablespa

Re: Remove Deprecated Exclusive Backup Mode

2020-06-30 Thread Stephen Frost
Greetings, * David Steele (da...@pgmasters.net) wrote: > On 6/30/20 6:13 PM, Stephen Frost wrote: > >* David Steele (da...@pgmasters.net) wrote: > >>Lastly, there is some concern about getting the backup label too late when > >>doing snapshots or using traditional backup software. It would certain

Re: Remove Deprecated Exclusive Backup Mode

2020-06-30 Thread Bruce Momjian
On Tue, Jun 30, 2020 at 07:10:46PM -0400, Stephen Frost wrote: > De-deprecating the exclusive backup mode and returning that info at the > start and telling users "put this anywhere EXCEPT this specific place, > and make sure you DO put these files in that exact place when you > restore" would perh

Re: Remove Deprecated Exclusive Backup Mode

2020-06-30 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jun 30, 2020 at 07:10:46PM -0400, Stephen Frost wrote: > > De-deprecating the exclusive backup mode and returning that info at the > > start and telling users "put this anywhere EXCEPT this specific place, > > and make sure you DO put

Re: Remove Deprecated Exclusive Backup Mode

2020-06-30 Thread Bruce Momjian
On Tue, Jun 30, 2020 at 07:32:47PM -0400, Stephen Frost wrote: > Greetings, > > * Bruce Momjian (br...@momjian.us) wrote: > > On Tue, Jun 30, 2020 at 07:10:46PM -0400, Stephen Frost wrote: > > > De-deprecating the exclusive backup mode and returning that info at the > > > start and telling users "

Re: Remove Deprecated Exclusive Backup Mode

2020-07-01 Thread Laurenz Albe
On Tue, 2020-06-30 at 17:38 -0400, David Steele wrote: > Rebased patch is attached. I remain -1 on removing the exclusive backup API. People who don't mind manual intervention when PostgreSQL doesn't start after a crash in backup mode could safely use it. We have been over this, and there is not

Re: Remove Deprecated Exclusive Backup Mode

2020-07-01 Thread Stephen Frost
Greetings, * Laurenz Albe (laurenz.a...@cybertec.at) wrote: > On Tue, 2020-06-30 at 17:38 -0400, David Steele wrote: > > Rebased patch is attached. > > I remain -1 on removing the exclusive backup API. It's still deprecated and I'm certainly against removing that status until/unless someone actu

Re: Remove Deprecated Exclusive Backup Mode

2020-07-01 Thread Laurenz Albe
On Wed, 2020-07-01 at 08:47 -0400, Stephen Frost wrote: > > I remain -1 on removing the exclusive backup API. > > It's still deprecated and I'm certainly against removing that status > until/unless someone actually fixes it (including documentation), and if > we're not going to do that then we sho

Re: Remove Deprecated Exclusive Backup Mode

2020-07-01 Thread Stephen Frost
Greetings, * Laurenz Albe (laurenz.a...@cybertec.at) wrote: > On Wed, 2020-07-01 at 08:47 -0400, Stephen Frost wrote: > > > I remain -1 on removing the exclusive backup API. > > > > It's still deprecated and I'm certainly against removing that status > > until/unless someone actually fixes it (in

Re: Remove Deprecated Exclusive Backup Mode

2020-07-01 Thread Bruce Momjian
On Wed, Jul 1, 2020 at 04:46:59PM +0200, Laurenz Albe wrote: > On Wed, 2020-07-01 at 08:47 -0400, Stephen Frost wrote: > > > I remain -1 on removing the exclusive backup API. > > > > It's still deprecated and I'm certainly against removing that status > > until/unless someone actually fixes it (i

Re: Remove Deprecated Exclusive Backup Mode

2020-07-01 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Wed, Jul 1, 2020 at 04:46:59PM +0200, Laurenz Albe wrote: > > On Wed, 2020-07-01 at 08:47 -0400, Stephen Frost wrote: > > > > I remain -1 on removing the exclusive backup API. > > > > > > It's still deprecated and I'm certainly against re

Re: Remove Deprecated Exclusive Backup Mode

2020-07-01 Thread Magnus Hagander
On Wed, Jul 1, 2020 at 2:47 PM Stephen Frost wrote: > > This would presumably be for the exclusive API as a way to make it not > completely broken, maybe. > > If we wanted to try and make this work in a non-exclusive manner then > we'd need to do something like have the user save some info out at

Re: Remove Deprecated Exclusive Backup Mode

2020-07-01 Thread Stephen Frost
Greetings, * Magnus Hagander (mag...@hagander.net) wrote: > On Wed, Jul 1, 2020 at 2:47 PM Stephen Frost wrote: > > This would presumably be for the exclusive API as a way to make it not > > completely broken, maybe. > > > > If we wanted to try and make this work in a non-exclusive manner then >

Re: Remove Deprecated Exclusive Backup Mode

2020-07-01 Thread Magnus Hagander
On Wed, Jul 1, 2020 at 6:29 PM Stephen Frost wrote: > Greetings, > > * Magnus Hagander (mag...@hagander.net) wrote: > > On Wed, Jul 1, 2020 at 2:47 PM Stephen Frost wrote: > > > This would presumably be for the exclusive API as a way to make it not > > > completely broken, maybe. > > > > > > If

Re: Remove Deprecated Exclusive Backup Mode

2020-07-01 Thread Stephen Frost
Greetings, * Magnus Hagander (mag...@hagander.net) wrote: > On Wed, Jul 1, 2020 at 6:29 PM Stephen Frost wrote: > > * Magnus Hagander (mag...@hagander.net) wrote: > > > On Wed, Jul 1, 2020 at 2:47 PM Stephen Frost wrote: > > > > This would presumably be for the exclusive API as a way to make it

Re: Remove Deprecated Exclusive Backup Mode

2020-07-01 Thread Robert Haas
On Wed, Jul 1, 2020 at 11:48 AM Magnus Hagander wrote: > We don't even need to make it an exclusive mode -- we can allow > *nonexclusive* backups but remove the requirement to run start and stop > backup in the same connection, which I believe is the problem that people > have with the exclusiv

Re: Remove Deprecated Exclusive Backup Mode

2020-07-01 Thread Bruce Momjian
On Wed, Jul 1, 2020 at 01:23:21PM -0400, Stephen Frost wrote: > > Or, how about we require them to provide the backup label contents in its > > entirety? Which I believe does contain that WAL portion? The downside of > > that is that it would be multiline which might screw with shellscripts. > >

Re: Remove Deprecated Exclusive Backup Mode

2020-07-01 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Jul 1, 2020 at 11:48 AM Magnus Hagander wrote: > > We don't even need to make it an exclusive mode -- we can allow > > *nonexclusive* backups but remove the requirement to run start and stop > > backup in the same connection, whi

Re: Remove Deprecated Exclusive Backup Mode

2020-07-01 Thread Magnus Hagander
On Wed, Jul 1, 2020 at 9:43 PM Robert Haas wrote: > On Wed, Jul 1, 2020 at 11:48 AM Magnus Hagander > wrote: > > We don't even need to make it an exclusive mode -- we can allow > *nonexclusive* backups but remove the requirement to run start and stop > backup in the same connection, which I beli

Re: Remove Deprecated Exclusive Backup Mode

2020-07-01 Thread Robert Haas
On Wed, Jul 1, 2020 at 3:50 PM Magnus Hagander wrote: > As far as I've seen, the one thing that people have problems with in the > exclusive mode backups are precisely the fact that they have to keep a > persistent conneciton open, and thus it cannot work together with backup > software that is

Re: Remove Deprecated Exclusive Backup Mode

2020-07-01 Thread Magnus Hagander
On Wed, Jul 1, 2020 at 9:59 PM Robert Haas wrote: > On Wed, Jul 1, 2020 at 3:50 PM Magnus Hagander > wrote: > > As far as I've seen, the one thing that people have problems with in the > exclusive mode backups are precisely the fact that they have to keep a > persistent conneciton open, and thus

Re: Remove Deprecated Exclusive Backup Mode

2020-07-01 Thread David Steele
On 7/1/20 3:58 PM, Robert Haas wrote: On Wed, Jul 1, 2020 at 3:50 PM Magnus Hagander wrote: As far as I've seen, the one thing that people have problems with in the exclusive mode backups are precisely the fact that they have to keep a persistent conneciton open, and thus it cannot work toget

Re: Remove Deprecated Exclusive Backup Mode

2020-07-01 Thread Bruce Momjian
On Wed, Jul 1, 2020 at 10:24:15PM +0200, Magnus Hagander wrote: > On Wed, Jul 1, 2020 at 9:59 PM Robert Haas wrote: > Yeah, I guess that's a pretty fair point. I have to confess to having > somewhat limited enthusiasm for adding a third mode here, but it might > be worth it. > The int

Re: Remove Deprecated Exclusive Backup Mode

2020-07-01 Thread Bruce Momjian
On Wed, Jul 1, 2020 at 04:27:58PM -0400, David Steele wrote: > On 7/1/20 3:58 PM, Robert Haas wrote: > > If we could jigger things so that you don't need to stop the backup at > > all, you only start it, and whether you ever finish copying everything > > is something about which the system need no

Re: Remove Deprecated Exclusive Backup Mode

2020-07-01 Thread Robert Haas
On Wed, Jul 1, 2020 at 4:27 PM David Steele wrote: > Well, the only thing pg_stop_backup() *really* needs to know is the > starting WAL position. pg_start_backup() gets that info so if it passes > it back to pg_stop_backup() that could be enough. Or as was proposed > above, it just passes the back

Re: Remove Deprecated Exclusive Backup Mode

2020-07-01 Thread David Steele
On 7/1/20 4:27 PM, David Steele wrote: Here's a thought. What if we just stored the oldest starting LSN and a count of how many backups have been requested. When the backup ends it checks that backup count is > 0 and starting LSN is <= its starting LSN. If not, it throws an error. When backup

Re: Remove Deprecated Exclusive Backup Mode

2020-07-01 Thread Robert Haas
On Wed, Jul 1, 2020 at 4:29 PM Bruce Momjian wrote: > Agreed. I don't think three modes would help anyone. Well, I think that if and when we remove the existing exclusive mode, we're going to break a bunch of people's backup scripts. I think it's appropriate to do that eventually, but I'm not in

Re: Remove Deprecated Exclusive Backup Mode

2020-07-01 Thread Magnus Hagander
On Wed, Jul 1, 2020 at 10:28 PM David Steele wrote: > On 7/1/20 3:58 PM, Robert Haas wrote: > > On Wed, Jul 1, 2020 at 3:50 PM Magnus Hagander > wrote: > >> As far as I've seen, the one thing that people have problems with in > the exclusive mode backups are precisely the fact that they have to

Re: Remove Deprecated Exclusive Backup Mode

2020-07-01 Thread Bruce Momjian
On Wed, Jul 1, 2020 at 04:36:03PM -0400, Robert Haas wrote: > On Wed, Jul 1, 2020 at 4:29 PM Bruce Momjian wrote: > > Agreed. I don't think three modes would help anyone. > > Well, I think that if and when we remove the existing exclusive mode, > we're going to break a bunch of people's backup

Re: Remove Deprecated Exclusive Backup Mode

2020-07-01 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Jul 1, 2020 at 4:29 PM Bruce Momjian wrote: > > Agreed. I don't think three modes would help anyone. > > Well, I think that if and when we remove the existing exclusive mode, > we're going to break a bunch of people's backup scri

Re: Remove Deprecated Exclusive Backup Mode

2020-07-01 Thread David Steele
On 7/1/20 4:39 PM, Magnus Hagander wrote: On Wed, Jul 1, 2020 at 10:28 PM David Steele Here's a thought. What if we just stored the oldest starting LSN and a count of how many backups have been requested. When the backup ends it checks that backup count is > 0 and starting LSN is <= i

Re: Remove Deprecated Exclusive Backup Mode

2020-07-01 Thread David Steele
On 7/1/20 4:32 PM, Bruce Momjian wrote: On Wed, Jul 1, 2020 at 04:27:58PM -0400, David Steele wrote: On 7/1/20 3:58 PM, Robert Haas wrote: If we could jigger things so that you don't need to stop the backup at all, you only start it, and whether you ever finish copying everything is something

Re: Remove Deprecated Exclusive Backup Mode

2020-07-01 Thread Bruce Momjian
On Wed, Jul 1, 2020 at 05:20:16PM -0400, David Steele wrote: > On 7/1/20 4:32 PM, Bruce Momjian wrote: > > On Wed, Jul 1, 2020 at 04:27:58PM -0400, David Steele wrote: > > > On 7/1/20 3:58 PM, Robert Haas wrote: > > > > If we could jigger things so that you don't need to stop the backup at > > >

Re: Remove Deprecated Exclusive Backup Mode

2020-07-01 Thread Magnus Hagander
On Wed, Jul 1, 2020 at 11:08 PM David Steele wrote: > On 7/1/20 4:39 PM, Magnus Hagander wrote: > > On Wed, Jul 1, 2020 at 10:28 PM David Steele > Here's a thought. What if we just stored the oldest starting LSN and > a > > count of how many backups have been requested. When the backup e

Re: Remove Deprecated Exclusive Backup Mode

2020-07-01 Thread Jehan-Guillaume de Rorthais
On Wed, 1 Jul 2020 15:58:57 -0400 Robert Haas wrote: > On Wed, Jul 1, 2020 at 3:50 PM Magnus Hagander wrote: > > As far as I've seen, the one thing that people have problems with in the > > exclusive mode backups are precisely the fact that they have to keep a > > persistent conneciton open, and

Re: Remove Deprecated Exclusive Backup Mode

2020-07-02 Thread Laurenz Albe
On Wed, 2020-07-01 at 16:46 -0400, Stephen Frost wrote: > I continue to find it curious that we stress a great deal over (very > likely) poorly written backup scripts that haven't been updated in the > 5+? years since exclusive mode was deprecated, but we happily add new > keywords in new major ver

Re: Remove Deprecated Exclusive Backup Mode

2020-07-02 Thread Magnus Hagander
On Thu, Jul 2, 2020 at 1:06 AM Jehan-Guillaume de Rorthais wrote: > On Wed, 1 Jul 2020 15:58:57 -0400 > Robert Haas wrote: > > > On Wed, Jul 1, 2020 at 3:50 PM Magnus Hagander > wrote: > > > As far as I've seen, the one thing that people have problems with in > the > > > exclusive mode backups

Re: Remove Deprecated Exclusive Backup Mode

2020-07-02 Thread Magnus Hagander
On Thu, Jul 2, 2020 at 10:35 AM Laurenz Albe wrote: > On Wed, 2020-07-01 at 16:46 -0400, Stephen Frost wrote: > > I continue to find it curious that we stress a great deal over (very > > likely) poorly written backup scripts that haven't been updated in the > > 5+? years since exclusive mode was

Re: Remove Deprecated Exclusive Backup Mode

2020-07-02 Thread David Steele
On 7/1/20 5:44 PM, Magnus Hagander wrote: On Wed, Jul 1, 2020 at 11:08 PM David Steele > wrote: But yeah, it would be possible to kill somebody else's session with some finagling. Still, worse case would be an error'd backup rather than a corrupt one.

Re: Remove Deprecated Exclusive Backup Mode

2020-07-02 Thread Jehan-Guillaume de Rorthais
On Thu, 2 Jul 2020 12:32:14 +0200 Magnus Hagander wrote: [...] > > non-exclusive backup...this is not that easy anymore. And > > pg_is_in_backup() is quite misleading if the admin found it without reading > > doc. Maybe an admin > > Yeah, as it is now it should really be called pg_is_in_exclusiv

Re: Remove Deprecated Exclusive Backup Mode

2020-07-02 Thread Stephen Frost
Greetings, * Laurenz Albe (laurenz.a...@cybertec.at) wrote: > On Wed, 2020-07-01 at 16:46 -0400, Stephen Frost wrote: > > If someone could explain what is so special about *this* part of the > > system that we absolutely can't possibly accept any change that might > > break user's scripts, and why

Re: Remove Deprecated Exclusive Backup Mode

2020-07-02 Thread Stephen Frost
Greetings, * Magnus Hagander (mag...@hagander.net) wrote: > On Thu, Jul 2, 2020 at 1:06 AM Jehan-Guillaume de Rorthais > wrote: > > function to list in progress non-exclusive backup and related backend pid > > might > > be a good start? > > I think it would make perfect sense to show manual (exc

Re: Remove Deprecated Exclusive Backup Mode

2020-07-10 Thread David Steele
On 7/2/20 7:12 AM, David Steele wrote: None of this really solves the problem of what happens when the user dumps the backup_label into the data directory. With traditional backup software that's pretty much going to be the only choice. Is telling them not to do it and washing our hands of it

Re: Remove Deprecated Exclusive Backup Mode

2019-02-15 Thread Andres Freund
On 2019-01-05 13:19:20 -0500, Stephen Frost wrote: > Greetings, > > * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: > > On 12/12/2018 05:31, Robert Haas wrote: > > > Most of the features I've been involved in removing have been > > > deprecated for 5+ years. The first release where t

Re: Remove Deprecated Exclusive Backup Mode

2019-02-17 Thread David Steele
On 2/16/19 5:57 AM, Andres Freund wrote: On 2019-01-05 13:19:20 -0500, Stephen Frost wrote: Greetings, * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: On 12/12/2018 05:31, Robert Haas wrote: Most of the features I've been involved in removing have been deprecated for 5+ years. T

Re: Remove Deprecated Exclusive Backup Mode

2019-02-22 Thread Magnus Hagander
On Mon, Feb 18, 2019 at 6:13 AM David Steele wrote: > On 2/16/19 5:57 AM, Andres Freund wrote: > > On 2019-01-05 13:19:20 -0500, Stephen Frost wrote: > >> Greetings, > >> > >> * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: > >>> On 12/12/2018 05:31, Robert Haas wrote: > Most of

Re: Remove Deprecated Exclusive Backup Mode

2019-02-22 Thread Stephen Frost
Greetings, * Magnus Hagander (mag...@hagander.net) wrote: > On Mon, Feb 18, 2019 at 6:13 AM David Steele wrote: > > On 2/16/19 5:57 AM, Andres Freund wrote: > > > On 2019-01-05 13:19:20 -0500, Stephen Frost wrote: > > >> * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: > > >>> On 12/1

Re: Remove Deprecated Exclusive Backup Mode

2019-02-22 Thread David Steele
On 2/22/19 6:32 PM, Stephen Frost wrote: * Magnus Hagander (mag...@hagander.net) wrote: On Mon, Feb 18, 2019 at 6:13 AM David Steele wrote: On 2/16/19 5:57 AM, Andres Freund wrote: I don't agree with a general 5 year deprecation window either. But it seems pretty clear that there's no major

Re: Remove Deprecated Exclusive Backup Mode

2019-02-22 Thread Fujii Masao
On Tue, Nov 27, 2018 at 12:13 PM David Steele wrote: > > Hackers, > > I propose we remove the deprecated exclusive backup mode of > pg_start_backup()/pg_stop_backup() for Postgres 12. -1 for the removal. I think that there are still many users of an exclusive backup API, and it's not so easy to m

Re: Remove Deprecated Exclusive Backup Mode

2019-02-22 Thread Robert Haas
On Fri, Feb 22, 2019 at 12:35 PM Fujii Masao wrote: > -1 for the removal. I think that there are still many users of an exclusive > backup API, and it's not so easy to migrate their backup scripts so that > only non-exclusive one is used because of the reason I wrote in another > thread. > https:

Re: Remove Deprecated Exclusive Backup Mode

2019-02-22 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Fri, Feb 22, 2019 at 12:35 PM Fujii Masao wrote: > > -1 for the removal. I think that there are still many users of an exclusive > > backup API, and it's not so easy to migrate their backup scripts so that > > only non-exclusive one is

Re: Remove Deprecated Exclusive Backup Mode

2019-02-24 Thread Christophe Pettus
> On Feb 22, 2019, at 15:18, Stephen Frost wrote: > Getting a solid and resiliant backup to work from a shell script is, imv > anyway (though I might have a bit of experience, having tried numerous > times myself and then realizing that it just isn't practical...), a > downright fool's errand.

Re: Remove Deprecated Exclusive Backup Mode

2019-02-24 Thread Stephen Frost
Greetings, * Christophe Pettus (x...@thebuild.com) wrote: > > On Feb 22, 2019, at 15:18, Stephen Frost wrote: > > Getting a solid and resiliant backup to work from a shell script is, imv > > anyway (though I might have a bit of experience, having tried numerous > > times myself and then realizing

Re: Remove Deprecated Exclusive Backup Mode

2019-02-24 Thread Andres Freund
Hi,, On 2019-02-24 11:52:54 -0800, Christophe Pettus wrote: > > On Feb 22, 2019, at 15:18, Stephen Frost wrote: > > Getting a solid and resiliant backup to work from a shell script is, imv > > anyway (though I might have a bit of experience, having tried numerous > > times myself and then realizi

Re: Remove Deprecated Exclusive Backup Mode

2019-02-24 Thread Christophe Pettus
> On Feb 24, 2019, at 12:00, Stephen Frost wrote: > > Do they realize how that existing backup strategy is flawed? Undoubtedly, some do, some don't. However, given that it has been the *only* backup API for a very long time, many organizations have spent a lot of time closing all of the ho

Re: Remove Deprecated Exclusive Backup Mode

2019-02-24 Thread Stephen Frost
Greetings, * Christophe Pettus (x...@thebuild.com) wrote: > > On Feb 24, 2019, at 12:00, Stephen Frost wrote: > > > > Do they realize how that existing backup strategy is flawed? > > Undoubtedly, some do, some don't. However, given that it has been the *only* > backup API for a very long time

Re: Remove Deprecated Exclusive Backup Mode

2019-02-24 Thread Laurenz Albe
Andres Freund wrote: > On 2019-02-24 11:52:54 -0800, Christophe Pettus wrote: > > > On Feb 22, 2019, at 15:18, Stephen Frost wrote: > > > Getting a solid and resiliant backup to work from a shell script is, imv > > > anyway (though I might have a bit of experience, having tried numerous > > > time

Re: Remove Deprecated Exclusive Backup Mode

2019-02-24 Thread Laurenz Albe
Stephen Frost wrote: > Yes, it *is* impossible to do safe backups with the existing API. There > is an unquestionable race condition where a system restart will cause > your system to not come back up without you going in and removing the > backup_label file- and the only way you make that race wi

Re: Remove Deprecated Exclusive Backup Mode

2019-02-24 Thread Christophe Pettus
> On Feb 24, 2019, at 13:00, Stephen Frost wrote: > > No, it *hasn't* been the only backup API for a long time- that was only > true up until 9.6 was released, since then we've had both, and it's made > everything a downright mess because of exactly these arguments. 9.6 has been out for about

Re: Remove Deprecated Exclusive Backup Mode

2019-02-24 Thread Stephen Frost
Greetings, * Laurenz Albe (laurenz.a...@cybertec.at) wrote: > Stephen Frost wrote: > > Yes, it *is* impossible to do safe backups with the existing API. There > > is an unquestionable race condition where a system restart will cause > > your system to not come back up without you going in and rem

Re: Remove Deprecated Exclusive Backup Mode

2019-02-24 Thread Stephen Frost
Greetings, * Christophe Pettus (x...@thebuild.com) wrote: > > On Feb 24, 2019, at 13:00, Stephen Frost wrote: > > > > No, it *hasn't* been the only backup API for a long time- that was only > > true up until 9.6 was released, since then we've had both, and it's made > > everything a downright me

Re: Remove Deprecated Exclusive Backup Mode

2019-02-24 Thread Christophe Pettus
> On Feb 24, 2019, at 13:44, Stephen Frost wrote: > > Right, and PG12 will be out for another *5* years beyond that, meaning > people will have had 8.5 years to move from the exclusive API to the > non-exclusive one. The thing is that for 90% of installations, the clock will start ticking whe

Re: Remove Deprecated Exclusive Backup Mode

2019-02-24 Thread Stephen Frost
Greetings, * Christophe Pettus (x...@thebuild.com) wrote: > > On Feb 24, 2019, at 13:44, Stephen Frost wrote: > > Right, and PG12 will be out for another *5* years beyond that, meaning > > people will have had 8.5 years to move from the exclusive API to the > > non-exclusive one. > > The thing i

Re: Remove Deprecated Exclusive Backup Mode

2019-02-24 Thread Andres Freund
Hi, On 2019-02-24 17:19:22 -0500, Stephen Frost wrote: > You say above that the new interface is unquestionably an improvement FWIW, I think we didn't design it even remotely as well as we ought to have. It was both a mistake to not offer a version of non-exclusive backups that works with a clien

Re: Remove Deprecated Exclusive Backup Mode

2019-02-24 Thread Christophe Pettus
> On Feb 24, 2019, at 14:19, Stephen Frost wrote: > You say above that the new interface is unquestionably an improvement > and here say that we shouldn't deprecate the old one in favor of it > (even though we actually already have... but that's beside the point I'm > trying to make here), so w

Re: Remove Deprecated Exclusive Backup Mode

2019-02-24 Thread Andres Freund
Hi, On 2019-02-24 14:35:04 -0800, Christophe Pettus wrote: > > On Feb 24, 2019, at 14:19, Stephen Frost wrote: > > You say above that the new interface is unquestionably an improvement > > and here say that we shouldn't deprecate the old one in favor of it > > (even though we actually already hav

Re: Remove Deprecated Exclusive Backup Mode

2019-02-24 Thread David Steele
On 2/24/19 11:36 PM, Stephen Frost wrote: Greetings, * Laurenz Albe (laurenz.a...@cybertec.at) wrote: Stephen Frost wrote: Yes, it *is* impossible to do safe backups with the existing API. There is an unquestionable race condition where a system restart will cause your system to not come back

Re: Remove Deprecated Exclusive Backup Mode

2019-02-24 Thread David Steele
On 2/25/19 12:35 AM, Christophe Pettus wrote: On Feb 24, 2019, at 14:19, Stephen Frost wrote: You say above that the new interface is unquestionably an improvement and here say that we shouldn't deprecate the old one in favor of it (even though we actually already have... but that's beside

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Adrien NAYRAT
On 2/22/19 7:10 PM, Robert Haas wrote: On Fri, Feb 22, 2019 at 12:35 PM Fujii Masao wrote: -1 for the removal. I think that there are still many users of an exclusive backup API, and it's not so easy to migrate their backup scripts so that only non-exclusive one is used because of the reason I

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Dagfinn Ilmari Mannsåker
David Steele writes: > On 2/25/19 12:35 AM, Christophe Pettus wrote: >> >> >>> On Feb 24, 2019, at 14:19, Stephen Frost wrote: >>> You say above that the new interface is unquestionably an improvement >>> and here say that we shouldn't deprecate the old one in favor of it >>> (even though we act

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Stephen Frost
Greetings, * Adrien NAYRAT (adrien.nay...@anayrat.info) wrote: > On 2/22/19 7:10 PM, Robert Haas wrote: > >On Fri, Feb 22, 2019 at 12:35 PM Fujii Masao wrote: > >>-1 for the removal. I think that there are still many users of an exclusive > >>backup API, and it's not so easy to migrate their back

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Stephen Frost
Greetings, * Dagfinn Ilmari Mannsåker (ilm...@ilmari.org) wrote: > David Steele writes: > > On 2/25/19 12:35 AM, Christophe Pettus wrote: > >>> On Feb 24, 2019, at 14:19, Stephen Frost wrote: > >>> You say above that the new interface is unquestionably an improvement > >>> and here say that we s

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2019-02-24 17:19:22 -0500, Stephen Frost wrote: > > You say above that the new interface is unquestionably an improvement > > FWIW, I think we didn't design it even remotely as well as we ought to > have. It was both a mistake to not off

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Laurenz Albe
Stephen Frost wrote: > > It will be annoying if after this removal, companies must change their > > backup strategy by using specific postgres tools (pgbackrest, barman). > > You don't write your own database system using CSV files and shell > magic, do you? I have to say that it continues to bog

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Stephen Frost
Greetings, * Laurenz Albe (laurenz.a...@cybertec.at) wrote: > Stephen Frost wrote: > > > It will be annoying if after this removal, companies must change their > > > backup strategy by using specific postgres tools (pgbackrest, barman). > > > > You don't write your own database system using CSV f

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Christophe Pettus
> On Feb 25, 2019, at 05:14, Stephen Frost wrote: > > You don't write your own database system using CSV files and shell > magic, do you? I have to say that it continues to boggle my mind that > people insist that *this* part of the system has to be able to be > implementable using shell scri

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Christophe Pettus
> On Feb 25, 2019, at 05:18, Stephen Frost wrote: > So.. We've more-or-less come full circle back to where the thread had > left off last time- the plan is to update the documentation to make it > clearer that the exclusive mode is deprecated and that it's going to be > removed, and then we'll

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Christophe Pettus
> On Feb 24, 2019, at 22:49, David Steele wrote: > This is a good idea -- but I'm still a bit mystified why the ability to run a > backup in shell script is considered to be a hard requirement. Just about any > language you can name performs non-exclusive backups with ease -- except > shell.

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Stephen Frost
Greetings, * Christophe Pettus (x...@thebuild.com) wrote: > > On Feb 25, 2019, at 05:18, Stephen Frost wrote: > > So.. We've more-or-less come full circle back to where the thread had > > left off last time- the plan is to update the documentation to make it > > clearer that the exclusive mode i

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Robert Haas
On Sun, Feb 24, 2019 at 3:00 PM Stephen Frost wrote: > Ok, then please do so, and please be prepared to continue to maintain > the documentation of both methods moving forward, because others have > tried and have (rightfully, in my opinion) decided that it's frankly not > worth the effort and ult

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Robert Haas
On Mon, Feb 25, 2019 at 1:49 AM David Steele wrote: > Exclusive backup will not be removed for PG12. There wasn't support for > it so I push it out to PG13. > > There does appear to be support for removing it in PG13, though, and I'd > like to see that done sooner than later. Given the additiona

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Sun, Feb 24, 2019 at 3:00 PM Stephen Frost wrote: > > Ok, then please do so, and please be prepared to continue to maintain > > the documentation of both methods moving forward, because others have > > tried and have (rightfully, in my

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Robert Haas
On Mon, Feb 25, 2019 at 8:42 AM Stephen Frost wrote: > Alright, then how about we provide a bit of help for everyone who's got > a system built around recovery.conf today, instead of just completely > ripping that out? > > If we're happy to do that then I really can't agree with these arguments >

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Robert Haas
On Fri, Feb 22, 2019 at 6:18 PM Stephen Frost wrote: > What we need is a backup tool included in core that users feel > comfortable using instead of trying to write their own. I agree. That's a great idea. Let's talk about how to make that happen. Providing a tool that gives people MORE AND BE

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Mon, Feb 25, 2019 at 8:42 AM Stephen Frost wrote: > > Alright, then how about we provide a bit of help for everyone who's got > > a system built around recovery.conf today, instead of just completely > > ripping that out? > > > > If we'

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Robert Haas
On Mon, Feb 25, 2019 at 11:09 AM Stephen Frost wrote: > I do think we can ask that people who wish to have a capability included > in PG (or continue to be included when there are serious known issues > with it...) be prepared to either build and maintain it themselves or to > convince someone els

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Fri, Feb 22, 2019 at 6:18 PM Stephen Frost wrote: > > What we need is a backup tool included in core that users feel > > comfortable using instead of trying to write their own. > > I agree. That's a great idea. Let's talk about how t

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Robert Haas
On Mon, Feb 25, 2019 at 11:23 AM Stephen Frost wrote: > If that argument did matter, we could go back and find the prior > discussions about the issues around the exclusive backup mode and about > removing it, or next year we could point to this thread about it, or the > year after, and say "well,

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Robert Haas
On Mon, Feb 25, 2019 at 11:33 AM Stephen Frost wrote: > > Removing an option that people are currently using, and which they > > find better than other available options for reasons with which I > > understand that you disagree, will make users more sad. Happy is > > better. > > I don't want to s

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Mon, Feb 25, 2019 at 11:23 AM Stephen Frost wrote: > > If that argument did matter, we could go back and find the prior > > discussions about the issues around the exclusive backup mode and about > > removing it, or next year we could p

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Mon, Feb 25, 2019 at 11:33 AM Stephen Frost wrote: > > > Removing an option that people are currently using, and which they > > > find better than other available options for reasons with which I > > > understand that you disagree, will

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Mon, Feb 25, 2019 at 11:09 AM Stephen Frost wrote: > > I do think we can ask that people who wish to have a capability included > > in PG (or continue to be included when there are serious known issues > > with it...) be prepared to eit

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Christophe Pettus
> On Feb 25, 2019, at 08:55, Stephen Frost wrote: > > I honestly do doubt that they have had the same experiences that I have > had Well, I guarantee you that no two people on this list have had identical experiences. :) I certainly have been bitten by the problems with the current system.

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Fujii Masao
On Mon, Feb 25, 2019 at 10:49 PM Laurenz Albe wrote: > > Stephen Frost wrote: > > > It will be annoying if after this removal, companies must change their > > > backup strategy by using specific postgres tools (pgbackrest, barman). > > > > You don't write your own database system using CSV files a

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread David Steele
On 2/25/19 7:50 PM, Fujii Masao wrote: On Mon, Feb 25, 2019 at 10:49 PM Laurenz Albe wrote: I'm not playing devil's advocate here to annoy you. I see the problems with the exclusive backup, and I see how it can hurt people. I just think that removing exclusive backup without some kind of help

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Bruce Momjian
On Mon, Feb 25, 2019 at 11:33:33AM -0500, Stephen Frost wrote: > I don't want to see more users stumbling over the issues with the > exclusive backup interface. A better interface exists, and has existed > since 9.6. Do you really think we would be having this discussion if the non-exclusive back

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread David Steele
Hi Christophe, On 2/25/19 7:24 PM, Christophe Pettus wrote: On Feb 25, 2019, at 08:55, Stephen Frost wrote: I honestly do doubt that they have had the same experiences that I have had Well, I guarantee you that no two people on this list have had identical experiences. :) I certainly ha

  1   2   3   >