Re: Question about StartLogicalReplication() error path

2021-06-19 Thread Amit Kapila
On Thu, Jun 17, 2021 at 4:25 AM Jeff Davis wrote: > > On Tue, 2021-06-15 at 15:19 +0900, Kyotaro Horiguchi wrote: > > I don't think the message is neded, but I don't oppose it as far as > > the level is LOG and the messages were changed as something like > > this: > > > > > > - elog(

Re: Question about StartLogicalReplication() error path

2021-06-16 Thread Jeff Davis
On Tue, 2021-06-15 at 15:19 +0900, Kyotaro Horiguchi wrote: > I don't think the message is neded, but I don't oppose it as far as > the level is LOG and the messages were changed as something like > this: > > > - elog(DEBUG1, "cannot stream from %X/%X, minimum is > %X/%X, forwarding

Re: Question about StartLogicalReplication() error path

2021-06-15 Thread Amit Kapila
On Tue, Jun 15, 2021 at 3:21 AM Jeff Davis wrote: > > On Mon, 2021-06-14 at 13:13 -0400, Robert Haas wrote: > > I'm happy to hear other opinions, but I think I would be inclined to > > vote in favor of #1 and/or #2 but against #3. > > What about upgrading it to, say, LOG? It seems like it would ha

Re: Question about StartLogicalReplication() error path

2021-06-14 Thread Kyotaro Horiguchi
At Mon, 14 Jun 2021 14:51:35 -0700, Jeff Davis wrote in > On Mon, 2021-06-14 at 13:13 -0400, Robert Haas wrote: > > I'm happy to hear other opinions, but I think I would be inclined to > > vote in favor of #1 and/or #2 but against #3. > > What about upgrading it to, say, LOG? It seems like it wo

Re: Question about StartLogicalReplication() error path

2021-06-14 Thread Jeff Davis
On Mon, 2021-06-14 at 13:13 -0400, Robert Haas wrote: > I'm happy to hear other opinions, but I think I would be inclined to > vote in favor of #1 and/or #2 but against #3. What about upgrading it to, say, LOG? It seems like it would happen pretty infrequently, and in the event something strange h

Re: Question about StartLogicalReplication() error path

2021-06-14 Thread Robert Haas
On Mon, Jun 14, 2021 at 12:50 PM Jeff Davis wrote: > It seems that there's not much agreement in a behavior change here. I > suggest one or more of the following: > > 1. change the logical rep protocol docs to match the current behavior > a. also briefly explain in the docs why it's different

Re: Question about StartLogicalReplication() error path

2021-06-14 Thread Jeff Davis
On Sat, 2021-06-12 at 16:17 +0530, Amit Kapila wrote: > AFAIU, currently, in such a case, the subscriber (client) won't > advance the flush location (confirmed_flush_lsn). So, we won't lose > any data. I think you are talking about the official Logical Replication specifically, rather than an arbi

Re: Question about StartLogicalReplication() error path

2021-06-12 Thread Amit Kapila
On Fri, Jun 11, 2021 at 11:52 AM Jeff Davis wrote: > > On Fri, 2021-06-11 at 10:13 +0530, Amit Kapila wrote: > > I think because there is no need to process the WAL that has been > > confirmed by the client. Do you see any problems with this scheme? > > Several: > > * Replication setups are comple

Re: Question about StartLogicalReplication() error path

2021-06-11 Thread Andres Freund
On 2021-06-11 12:07:24 -0700, Jeff Davis wrote: > On Fri, 2021-06-11 at 11:56 -0700, Andres Freund wrote: > > That doesn't work as easily in older versions because there was no > > SQL > > support in replication connections until PG 10... > > 9.6 will be EOL this year. I don't really see why such

Re: Question about StartLogicalReplication() error path

2021-06-11 Thread Andres Freund
Hi, On 2021-06-11 16:05:10 -0400, Robert Haas wrote: > You seem to see this as some kind of major problem and I guess I don't > agree. I think it's pretty clear what the motivation was for the > current behavior, because I believe it's well-explained by the comment > and the three people who have

Re: Question about StartLogicalReplication() error path

2021-06-11 Thread Robert Haas
On Fri, Jun 11, 2021 at 2:49 PM Jeff Davis wrote: > It doesn't add any overhead. > > All the client would have to do is "SELECT confirmed_flush_lsn FROM > pg_replication_slots WHERE slot_name='myslot'", and compare it to the > stored value. If the stored value is earlier than the > confirmed_flush

Re: Question about StartLogicalReplication() error path

2021-06-11 Thread Jeff Davis
On Fri, 2021-06-11 at 11:56 -0700, Andres Freund wrote: > That doesn't work as easily in older versions because there was no > SQL > support in replication connections until PG 10... 9.6 will be EOL this year. I don't really see why such old versions are relevant to this discussion. Regards,

Re: Question about StartLogicalReplication() error path

2021-06-11 Thread Andres Freund
On 2021-06-11 11:49:19 -0700, Jeff Davis wrote: > All the client would have to do is "SELECT confirmed_flush_lsn FROM > pg_replication_slots WHERE slot_name='myslot'", and compare it to the > stored value. That doesn't work as easily in older versions because there was no SQL support in replicatio

Re: Question about StartLogicalReplication() error path

2021-06-11 Thread Jeff Davis
On Fri, 2021-06-11 at 10:40 -0700, Andres Freund wrote: > Especially because it very well might break existing working > setups... Please address my concerns[1]. Regards, Jeff Davis [1] https://www.postgresql.org/message-id/e22a4606333ce1032e29fe2fb1aa9036e6f0ca98.camel%40j-davis.com

Re: Question about StartLogicalReplication() error path

2021-06-11 Thread Jeff Davis
On Fri, 2021-06-11 at 13:15 -0400, Robert Haas wrote: > on it, but it seems to me that the comment does explain this, and > that > it's basically the same explanation as what Amit said. It only addresses the "pro" side of the behavior, not the "con". It's a bit like saying "Given that we are in th

Re: Question about StartLogicalReplication() error path

2021-06-11 Thread Andres Freund
Hi, On 2021-06-11 13:15:11 -0400, Robert Haas wrote: > On Fri, Jun 11, 2021 at 2:23 AM Jeff Davis wrote: > > * The comment acknowledges that a user might expect an error in that > > case; but doesn't really address why the user would expect an error, > > and why it's OK to violate that expectatio

Re: Question about StartLogicalReplication() error path

2021-06-11 Thread Robert Haas
On Fri, Jun 11, 2021 at 2:23 AM Jeff Davis wrote: > * The comment acknowledges that a user might expect an error in that > case; but doesn't really address why the user would expect an error, > and why it's OK to violate that expectation. This code was written by Andres, so he'd be the best perso

Re: Question about StartLogicalReplication() error path

2021-06-10 Thread Jeff Davis
On Fri, 2021-06-11 at 10:13 +0530, Amit Kapila wrote: > Because sometimes clients don't have to do anything for xlog records. > One example is WAL for DDL where logical decoding didn't produce > anything for the client but later with keepalive we send the LSN of > WAL where DDL has finished and the

Re: Question about StartLogicalReplication() error path

2021-06-10 Thread Amit Kapila
On Fri, Jun 11, 2021 at 7:38 AM Jeff Davis wrote: > > StartLogicalReplication() calls CreateDecodingContext(), which says: > > else if (start_lsn < slot->data.confirmed_flush) > { > /* > * It might seem like we should error out in this case, but it's > * pretty common for a clien

Question about StartLogicalReplication() error path

2021-06-10 Thread Jeff Davis
StartLogicalReplication() calls CreateDecodingContext(), which says: else if (start_lsn < slot->data.confirmed_flush) { /* * It might seem like we should error out in this case, but it's * pretty common for a client to acknowledge a LSN it doesn't have to * do anything for,