Re: pg_receivewal - couple of improvements

2022-02-06 Thread Julien Rouhaud
Hi, On Mon, Feb 07, 2022 at 12:03:03PM +0530, Bharath Rupireddy wrote: > > What if someone doesn't use pg_receivewal as an archive location? The > pg_receivewal can also be used for synchronous replication quorum > right? In this situation, I don't mind if some of the WAL files are > missing in

Re: pg_receivewal - couple of improvements

2022-02-06 Thread Bharath Rupireddy
On Mon, Feb 7, 2022 at 8:23 AM Michael Paquier wrote: > > On Sun, Feb 06, 2022 at 01:01:41PM +0530, Bharath Rupireddy wrote: > > With the start position as an option, users can just provide the LSN > > from which they want to stream the WAL, in the above case, it can be > > from primary's latest

Re: pg_receivewal - couple of improvements

2022-02-06 Thread Michael Paquier
On Sun, Feb 06, 2022 at 01:01:41PM +0530, Bharath Rupireddy wrote: > With the start position as an option, users can just provide the LSN > from which they want to stream the WAL, in the above case, it can be > from primary's latest checkpoint LSN. This still strikes me as a dangerous thing to

Re: pg_receivewal - couple of improvements

2022-02-05 Thread Bharath Rupireddy
On Sun, Feb 6, 2022 at 12:16 PM Michael Paquier wrote: > > On Thu, Feb 03, 2022 at 10:01:42PM +0800, Julien Rouhaud wrote: > > I don't get it. If you're missing WAL it means that will you have to do > > that > > tedious manual work to retrieve them no matter what. So on top of that > >

Re: pg_receivewal - couple of improvements

2022-02-05 Thread Michael Paquier
On Thu, Feb 03, 2022 at 10:01:42PM +0800, Julien Rouhaud wrote: > I don't get it. If you're missing WAL it means that will you have to do that > tedious manual work to retrieve them no matter what. So on top of that > tedious > work, you also have to make sure that you don't provide a bogus

Re: pg_receivewal - couple of improvements

2022-02-03 Thread Julien Rouhaud
On Thu, Feb 03, 2022 at 06:40:55PM +0530, Bharath Rupireddy wrote: > > Isn't complex for anyone to go to the archive location which involves > extra steps - getting authentication tokens, searching there for the > required WAL file, downloading it, unzipping it, copying back to > pg_receivewal

Re: pg_receivewal - couple of improvements

2022-02-03 Thread Bharath Rupireddy
On Wed, Feb 2, 2022 at 9:28 PM Julien Rouhaud wrote: > > On Wed, Feb 02, 2022 at 09:14:03PM +0530, Bharath Rupireddy wrote: > > > > FYI that thread is closed, it committed the change (f61e1dd [1]) that > > pg_receivewal can read from its replication slot restart lsn. > > > > I know that providing

Re: pg_receivewal - couple of improvements

2022-02-02 Thread Julien Rouhaud
On Wed, Feb 02, 2022 at 09:14:03PM +0530, Bharath Rupireddy wrote: > > FYI that thread is closed, it committed the change (f61e1dd [1]) that > pg_receivewal can read from its replication slot restart lsn. > > I know that providing the start pos as an option came up there [2], > but I wanted to

Re: pg_receivewal - couple of improvements

2022-02-02 Thread Bharath Rupireddy
On Wed, Feb 2, 2022 at 9:05 PM Julien Rouhaud wrote: > > Hi, > > On Wed, Feb 02, 2022 at 08:53:13PM +0530, Bharath Rupireddy wrote: > > > > Here are some improvements we can make to pg_receivewal that were > > emanated after working with it in production environments: > > > > 1) As a user, I,

Re: pg_receivewal - couple of improvements

2022-02-02 Thread Julien Rouhaud
Hi, On Wed, Feb 02, 2022 at 08:53:13PM +0530, Bharath Rupireddy wrote: > > Here are some improvements we can make to pg_receivewal that were > emanated after working with it in production environments: > > 1) As a user, I, sometimes, want my pg_receivewal to start streaming > from the LSN that

pg_receivewal - couple of improvements

2022-02-02 Thread Bharath Rupireddy
Hi, Here are some improvements we can make to pg_receivewal that were emanated after working with it in production environments: 1) As a user, I, sometimes, want my pg_receivewal to start streaming from the LSN that I provide as an input i.e. startpos instead of it calculating the stream start