Re: Catalog_xmin is not advanced when a logical slot is lost

2022-11-22 Thread Alvaro Herrera
On 2022-Nov-21, sirisha chamarthi wrote: > On Mon, Nov 21, 2022 at 10:56 AM Alvaro Herrera > wrote: > > Instead I'm going to do what Ashutosh mentioned at the start, which is > > to verify both the restart_lsn and the invalidated_at, when deciding > > whether to ignore the slot. > > Sounds

Re: Catalog_xmin is not advanced when a logical slot is lost

2022-11-21 Thread sirisha chamarthi
On Mon, Nov 21, 2022 at 10:56 AM Alvaro Herrera wrote: > On 2022-Nov-21, sirisha chamarthi wrote: > > > I have a old .partial file in the data directory to reproduce this. > > I don't think the .partial file is in itself important. But I think > this whole thing is a distraction. Yes, sorry

Re: Catalog_xmin is not advanced when a logical slot is lost

2022-11-21 Thread Alvaro Herrera
On 2022-Nov-21, sirisha chamarthi wrote: > I have a old .partial file in the data directory to reproduce this. I don't think the .partial file is in itself important. But I think this whole thing is a distraction. I managed to reproduce it eventually, by messing with the slot and WAL at

Re: Catalog_xmin is not advanced when a logical slot is lost

2022-11-21 Thread sirisha chamarthi
On Mon, Nov 21, 2022 at 10:40 AM sirisha chamarthi < sirichamarth...@gmail.com> wrote: > > > On Mon, Nov 21, 2022 at 10:11 AM Alvaro Herrera > wrote: > >> On 2022-Nov-21, sirisha chamarthi wrote: >> >> > It appears to be. wal_sender is setting restart_lsn to a valid LSN even >> > when the slot

Re: Catalog_xmin is not advanced when a logical slot is lost

2022-11-21 Thread sirisha chamarthi
On Mon, Nov 21, 2022 at 10:11 AM Alvaro Herrera wrote: > On 2022-Nov-21, sirisha chamarthi wrote: > > > It appears to be. wal_sender is setting restart_lsn to a valid LSN even > > when the slot is invalidated. > > > postgres@pgvm:~$ /usr/local/pgsql/bin/pg_receivewal -S s1 -D . > >

Re: Catalog_xmin is not advanced when a logical slot is lost

2022-11-21 Thread Alvaro Herrera
On 2022-Nov-21, sirisha chamarthi wrote: > It appears to be. wal_sender is setting restart_lsn to a valid LSN even > when the slot is invalidated. > postgres@pgvm:~$ /usr/local/pgsql/bin/pg_receivewal -S s1 -D . > pg_receivewal: error: unexpected termination of replication stream: ERROR: >

Re: Catalog_xmin is not advanced when a logical slot is lost

2022-11-21 Thread sirisha chamarthi
On Mon, Nov 21, 2022 at 9:12 AM Alvaro Herrera wrote: > On 2022-Nov-21, sirisha chamarthi wrote: > > > On Mon, Nov 21, 2022 at 8:05 AM Alvaro Herrera > > wrote: > > > > Thank you. I had pushed mine for CirrusCI to test, and it failed the > > > assert I added in slot.c: > > >

Re: Catalog_xmin is not advanced when a logical slot is lost

2022-11-21 Thread Alvaro Herrera
On 2022-Nov-21, sirisha chamarthi wrote: > On Mon, Nov 21, 2022 at 8:05 AM Alvaro Herrera > wrote: > > Thank you. I had pushed mine for CirrusCI to test, and it failed the > > assert I added in slot.c: > > https://cirrus-ci.com/build/4786354503548928 > > Not yet sure why, looking into it. > >

Re: Catalog_xmin is not advanced when a logical slot is lost

2022-11-21 Thread sirisha chamarthi
On Mon, Nov 21, 2022 at 8:05 AM Alvaro Herrera wrote: > On 2022-Nov-21, sirisha chamarthi wrote: > > > > > I am a fan of stricter, all-assumption-covering conditions. In case > we > > > > don't want to check restart_lsn, an Assert might be useful to > validate > > > > our assumption. > > > > > >

Re: Catalog_xmin is not advanced when a logical slot is lost

2022-11-21 Thread Alvaro Herrera
On 2022-Nov-21, sirisha chamarthi wrote: > > > I am a fan of stricter, all-assumption-covering conditions. In case we > > > don't want to check restart_lsn, an Assert might be useful to validate > > > our assumption. > > > > Agreed. I'll throw in an assert. > > Changed this in the patch to

Re: Catalog_xmin is not advanced when a logical slot is lost

2022-11-21 Thread sirisha chamarthi
Thanks Alvaro, Ashutosh for your comments. On Mon, Nov 21, 2022 at 6:20 AM Alvaro Herrera wrote: > On 2022-Nov-21, Ashutosh Bapat wrote: > > > Maybe. In that case pg_get_replication_slots() should be changed. We > > should use the same criteria to decide whether a slot is invalidated > > or

Re: Catalog_xmin is not advanced when a logical slot is lost

2022-11-21 Thread Alvaro Herrera
On 2022-Nov-21, Ashutosh Bapat wrote: > Maybe. In that case pg_get_replication_slots() should be changed. We > should use the same criteria to decide whether a slot is invalidated > or not at all the places. Right. > I am a fan of stricter, all-assumption-covering conditions. In case we > don't

Re: Catalog_xmin is not advanced when a logical slot is lost

2022-11-21 Thread Ashutosh Bapat
On Mon, Nov 21, 2022 at 5:39 PM Alvaro Herrera wrote: > > On 2022-Nov-21, Ashutosh Bapat wrote: > > > I think the condition should be > > > > if (!XLogRecPtrIsInvalid(invalidated_at_lsn)) LSN and XID are > > different data types. > > Yeah, this bit is wrong. I agree with your suggestion to just

Re: Catalog_xmin is not advanced when a logical slot is lost

2022-11-21 Thread Alvaro Herrera
On 2022-Nov-21, Ashutosh Bapat wrote: > I think the condition should be > > if (!XLogRecPtrIsInvalid(invalidated_at_lsn)) LSN and XID are > different data types. Yeah, this bit is wrong. I agree with your suggestion to just keep a boolean flag, as we don't need more than that. > and to be

Re: Catalog_xmin is not advanced when a logical slot is lost

2022-11-21 Thread Ashutosh Bapat
Hi Sirisha, Thanks for identifying the bug and the solution. Some review comments inlined. On Mon, Nov 21, 2022 at 2:49 PM Alvaro Herrera wrote: > > On 2022-Nov-20, sirisha chamarthi wrote: > > > Hi Hackers, > > > > forking this thread from the discussion [1] as suggested by Amit. > > > >

Re: Catalog_xmin is not advanced when a logical slot is lost

2022-11-21 Thread Alvaro Herrera
On 2022-Nov-20, sirisha chamarthi wrote: > Hi Hackers, > > forking this thread from the discussion [1] as suggested by Amit. > > Catalog_xmin is not advanced when a logical slot is invalidated (lost) > until the invalidated slot is dropped. This patch ignores invalidated slots > while computing

Catalog_xmin is not advanced when a logical slot is lost

2022-11-20 Thread sirisha chamarthi
Hi Hackers, forking this thread from the discussion [1] as suggested by Amit. Catalog_xmin is not advanced when a logical slot is invalidated (lost) until the invalidated slot is dropped. This patch ignores invalidated slots while computing the oldest xmin. Attached a small patch to address this