Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-09-03 Thread shveta malik
On Wed, Sep 3, 2025 at 11:58 AM Ajin Cherian wrote: > > Attaching v10 with the above changes. > The patch does not apply on HEAD. Can you please rebase? thanks Shveta

Re: Conflict detection for update_deleted in logical replication

2025-09-02 Thread shveta malik
On Tue, Sep 2, 2025 at 3:30 PM shveta malik wrote: > > > > > > > > > Here is V70 patch set. > > > > > Please find a few comments on v70-003: 1) Doc of dead_tuple_retention_active says: True if retain_dead_tuples is enabled and the retention duration for

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-09-02 Thread shveta malik
Few trivial comments for doc:

Re: Conflict detection for update_deleted in logical replication

2025-09-02 Thread shveta malik
On Mon, Sep 1, 2025 at 5:45 PM shveta malik wrote: > > > > > Here is V70 patch set. > > > > The patch v70-0001 looks good to me. Verified, all the old issues are > resolved. > > Will resume review of v70-0002 now. > Please find a few comments on v70

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-09-01 Thread shveta malik
On Tue, Sep 2, 2025 at 10:24 AM shveta malik wrote: > > Few trivial comments for doc: Sorry, the email got sent without comments. 1) +It is important to note that when wal_level is set to +replica the effective WAL level can automatically change comma after replica missi

Re: Conflict detection for update_deleted in logical replication

2025-09-01 Thread shveta malik
On Mon, Sep 1, 2025 at 5:07 PM Zhijie Hou (Fujitsu) wrote: > > > I reviewed the patch internally and tweaked a small detail of the apply worker > to reduce the waiting time in the main loop when max_retention_duration is > defined (set wait_time = min(wait_time, max_retention_duration)). Also, I

Re: Issue with logical replication slot during switchover

2025-08-31 Thread shveta malik
On Sat, Aug 30, 2025 at 11:43 AM Shlok Kyal wrote: > > Hi Fabrice, > > Thanks for providing the patch. I reviewed your patch and have > following comment: > > 1. I think we should add a commit message in the patch. It would help > to give an understanding of the patch. > > 2. I tried applying patc

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-08-31 Thread shveta malik
On Fri, Aug 29, 2025 at 4:14 PM Ashutosh Bapat wrote: > > On Fri, Aug 29, 2025 at 2:37 PM shveta malik wrote: > > > > On Fri, Aug 29, 2025 at 2:20 PM Ashutosh Bapat > > wrote: > > > > > > @@ -1276,7 +1331,7 @@ wait_for_slot_activity(bool some_slot_upd

Re: Conflict detection for update_deleted in logical replication

2025-08-29 Thread shveta malik
On Fri, Aug 29, 2025 at 11:49 AM Zhijie Hou (Fujitsu) wrote: > > Here is the new version patch set which also addressed Shveta's comments[1]. > Thanks for the patch. On 001 alone, I’m observing a behavior where, if sub1 has stopped retention, and I then create a new subscription sub2, the worker

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-08-29 Thread shveta malik
On Fri, Aug 29, 2025 at 2:20 PM Ashutosh Bapat wrote: > > On Fri, Aug 29, 2025 at 11:42 AM Ajin Cherian wrote: > > > > On Fri, Aug 29, 2025 at 3:01 PM shveta malik wrote: > > > > > > On Tue, Aug 26, 2025 at 9:58 AM Ajin Cherian wrote: > > >

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-08-29 Thread shveta malik
On Fri, Aug 29, 2025 at 9:59 AM Masahiko Sawada wrote: > > On Thu, Aug 28, 2025 at 4:29 AM shveta malik wrote: > > > > On Wed, Aug 27, 2025 at 12:12 PM Masahiko Sawada > > wrote: > > > > > > On Tue, Aug 26, 2025 at 2:32 AM shveta malik > > >

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-08-28 Thread shveta malik
On Tue, Aug 26, 2025 at 9:58 AM Ajin Cherian wrote: > > On Fri, Aug 22, 2025 at 3:44 PM shveta malik wrote: > > > > On Wed, Aug 20, 2025 at 10:53 AM Ajin Cherian wrote: > > > > > > > > > I've removed them. > > > Attaching patch v8

Re: Issue with logical replication slot during switchover

2025-08-28 Thread shveta malik
On Thu, Aug 28, 2025 at 9:11 PM Fabrice Chapuis wrote: > > What is the procedure to create this patch. Thank you for your help. > We use 'git format-patch' to generate formatted patches. I have given a few links ([1],[2],[3)] on know-how. I usually use these steps: git add git add git c

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-08-28 Thread shveta malik
On Wed, Aug 27, 2025 at 12:12 PM Masahiko Sawada wrote: > > On Tue, Aug 26, 2025 at 2:32 AM shveta malik wrote: > > > > On Tue, Aug 26, 2025 at 12:54 AM Masahiko Sawada > > wrote: > > > > > > I've attached the updated patch that incorporated t

Re: Conflict detection for update_deleted in logical replication

2025-08-28 Thread shveta malik
On Thu, Aug 28, 2025 at 8:02 AM Zhijie Hou (Fujitsu) wrote: > > > I noticed that Cfbot failed to compile the document due to a typo after > renaming > the subscription option. Here are the updated V67 patches to fix that, only > the doc > in 0001 is modified. > Please find a few comments: pat

Re: Issue with logical replication slot during switchover

2025-08-28 Thread shveta malik
On Wed, Aug 27, 2025 at 7:55 PM Fabrice Chapuis wrote: > > For the first step (a), the pg_create_logical_replication_slot interface is > extended. > The slot on the new attached standby will be dropped and recreated if the > flag allow_overwrite is set to true. > I tested the modified source, co

Re: Avoid retaining conflict-related data when no tables are subscribed

2025-08-27 Thread shveta malik
On Thu, Aug 28, 2025 at 7:54 AM Zhijie Hou (Fujitsu) wrote: > > Hi, > > My colleague Nisha reported an issue to me off-list: dead tuples can't > be removed when retain_dead_tuples is enabled for a subscription with no > tables. > > This appears to stem from the inability to advance the non-remova

Re: Conflict detection for update_deleted in logical replication

2025-08-26 Thread shveta malik
Please find some more comments: 1) In CheckSubDeadTupleRetention(), shall we have below instead of retain_dead_tuples check in all conditions? if (retain_dead_tuples) guc checks (wal_level and tracl_commit) else max retention check 2) Currently stop and resume messages are: ~~ LOG: logic

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-08-26 Thread shveta malik
On Tue, Aug 26, 2025 at 12:54 AM Masahiko Sawada wrote: > > I've attached the updated patch that incorporated the comments and is > rebased to the current HEAD. > Thanks for the patch, please find a few comments concerning LOG messages: 1) slotsync worker gives LOG: LOG: replication slot synchr

Re: Logical Replication of sequences

2025-08-25 Thread shveta malik
On Thu, Aug 21, 2025 at 10:08 PM vignesh C wrote: > > I have also addressed all the comments from [1] in the attached > v20250823 version patch. > [1] - > https://www.postgresql.org/message-id/CAA4eK1%2BoVQW8oP%3DLo1X8qac6dzg-fgGQ6R_F_psfokUEqe%2Ba6w%40mail.gmail.com > Thank You for the patches

Re: Conflict detection for update_deleted in logical replication

2025-08-25 Thread shveta malik
On Mon, Aug 25, 2025 at 12:09 PM shveta malik wrote: > > On Mon, Aug 25, 2025 at 10:06 AM Zhijie Hou (Fujitsu) > wrote: > > > > Attach the V65 patch set which addressed above and > > Shveta's comments[1]. > > > > Thank You for the patches, please

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-08-24 Thread shveta malik
On Sat, Aug 23, 2025 at 3:51 AM Masahiko Sawada wrote: > > > I found that we don't need to expose LogicalDecodingCtlData in > logicalctl.h header file. I've updated some cosmetic changes including > that point. > > I think the patch is getting pretty good shape Yes, I agree. > and am aiming at >

Re: Conflict detection for update_deleted in logical replication

2025-08-24 Thread shveta malik
On Mon, Aug 25, 2025 at 10:06 AM Zhijie Hou (Fujitsu) wrote: > > Attach the V65 patch set which addressed above and > Shveta's comments[1]. > Thank You for the patches, please find a few comments on v64 itself (I think valid on v65 as well): 1) in resume_conflict_info_retention(), shall we rewri

Re: Conflict detection for update_deleted in logical replication

2025-08-22 Thread shveta malik
On Fri, Aug 22, 2025 at 5:10 PM Amit Kapila wrote: > > On Mon, Aug 4, 2025 at 3:11 PM Amit Kapila wrote: > > > > On Mon, Aug 4, 2025 at 11:46 AM shveta malik wrote: > > > > > > 7) > > > Shall we rename 'max_conflict_retention_duration'

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-08-21 Thread shveta malik
On Wed, Aug 20, 2025 at 10:53 AM Ajin Cherian wrote: > > > I've removed them. > Attaching patch v8 addressing the above comments. > Thanks for the patch. Please find a few comments: 1) When the API is in progress, and meanwhile in another session we turn off hot_standby_feedback, the API session

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-08-21 Thread shveta malik
On Thu, Aug 21, 2025 at 10:34 PM Masahiko Sawada wrote: > > On Wed, Aug 20, 2025 at 3:11 AM shveta malik wrote: > > > > Please find a few comments: > > Thank you for reviewing the patch! > > > > > 1) > > ReplicationSlotsDropDBSlots: > > + bool

Re: Conflict detection for update_deleted in logical replication

2025-08-21 Thread shveta malik
On Thu, Aug 21, 2025 at 2:09 PM Zhijie Hou (Fujitsu) wrote: > > On Thursday, August 21, 2025 2:01 PM shveta malik > wrote: > > > > On Wed, Aug 20, 2025 at 12:12 PM Zhijie Hou (Fujitsu) > > wrote: > > > > > > > > > I agree. Here is V63 versi

Re: Logical Replication of sequences

2025-08-20 Thread shveta malik
On Wed, Aug 20, 2025 at 2:25 PM vignesh C wrote: > > On Tue, 19 Aug 2025 at 23:33, Masahiko Sawada wrote: > > > > I imagined something like case 2. For logical replication of tables, > > if we support DDL replication (i.e., CREATE/ALTER/DROP TABLE), all > > changes the apply worker executes are s

Re: Conflict detection for update_deleted in logical replication

2025-08-20 Thread shveta malik
On Wed, Aug 20, 2025 at 12:12 PM Zhijie Hou (Fujitsu) wrote: > > > I agree. Here is V63 version which implements this approach. > Thank You for the patches. > The retention status is recorded in the pg_subscription catalog > (subretentionactive) to prevent unnecessary retention initiation upon s

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-08-20 Thread shveta malik
Please find a few comments: 1) ReplicationSlotsDropDBSlots: + bool dropped = false; We can name 'dropped ' as 'dropped_logical' similar to ReplicationSlotCleanup. 2) ReplicationSlotsDropDBSlots() + + if (dropped && nlogicalslots == 0) + DisableLogicalDecodingIfNecessary(); I could not understan

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-08-19 Thread shveta malik
On Tue, Aug 19, 2025 at 10:55 AM Ajin Cherian wrote: > > Attaching patch v7 addressing all the above comments. > Thank You for the patches. Please find a few comments: 1) We are not resetting 'slot_persistence_pending' to false anywhere. So once it hits the flow which sets it to true, it will ne

Re: Conflict detection for update_deleted in logical replication

2025-08-14 Thread shveta malik
Few more comments: 1) src/sgml/monitoring.sgml: + + True if retain_dead_tuples + is enabled and the duration for which information useful for conflict + detection is retained by this apply worker does not exceed + max_conflict_retention_duration; NULL for + par

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-08-13 Thread shveta malik
On Thu, Aug 14, 2025 at 7:28 AM Ajin Cherian wrote: > > > Patch v6 attached. > Thanks Ajin. Please find my comments: 1) SyncReplicationSlots: + remote_slots = fetch_or_refresh_remote_slots(wrconn, NIL); + + /* Retry until all slots are sync ready atleast */ + for (;;) + { + bool some_slot_update

Re: Issue with logical replication slot during switchover

2025-08-13 Thread shveta malik
On Wed, Aug 13, 2025 at 9:29 PM Fabrice Chapuis wrote: > > Thanks for sharing this. In fact, I agree, introducing an allow_overwrite > slot property makes seems cleaner than a GUC for this specific use case. > > a) At first, an extension of pg_create_logical_replication_slot() could be > propos

Re: Conflict detection for update_deleted in logical replication

2025-08-13 Thread shveta malik
On Wed, Aug 13, 2025 at 4:15 PM shveta malik wrote: > > On Wed, Aug 13, 2025 at 10:41 AM Zhijie Hou (Fujitsu) > wrote: > > > > > > Here is the V61 patch set which addressed above comments and the comment by > > Nisha[2]. > > > > Thank You for the

Re: Conflict detection for update_deleted in logical replication

2025-08-13 Thread shveta malik
On Wed, Aug 13, 2025 at 5:42 PM Amit Kapila wrote: > > > > > 4) > > For the DETAIL part of resume and stop messages, how about these: > > > > The retention duration for information used in conflict detection has > > exceeded the limit of xx. > > The retention duration for information used in confl

Re: Conflict detection for update_deleted in logical replication

2025-08-13 Thread shveta malik
On Wed, Aug 13, 2025 at 10:41 AM Zhijie Hou (Fujitsu) wrote: > > > Here is the V61 patch set which addressed above comments and the comment by > Nisha[2]. > Thank You for the patch. I tested the patch, please find a few comments: 1) Now when it stops-retention and later resumes it due to the fa

Re: Issue with logical replication slot during switchover

2025-08-12 Thread shveta malik
On Fri, Aug 8, 2025 at 7:01 PM Fabrice Chapuis wrote: > > Thanks Shveta for coming on this point again and fixing the link. > The idea is to check if the slot has same name to try to resynchronize it > with the primary. > ok the check on the failover status for the remote slot is perhaps redundan

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-08-12 Thread shveta malik
On Mon, Aug 11, 2025 at 1:37 PM Ajin Cherian wrote: > > On Fri, Aug 8, 2025 at 11:22 PM Ashutosh Bapat > wrote: > > > > > > There's also a minor merge conflict because func.sgml is not split > > into multiple files. > > > > Yes, I fixed this. > Thanks for the patch. Please find a few comments:

Re: Conflict detection for update_deleted in logical replication

2025-08-12 Thread shveta malik
025 at 2:21 PM Amit Kapila > > > wrote: > > > > > > > > On Tue, Aug 12, 2025 at 2:06 PM shveta malik > > > wrote: > > > > > > > > > > 2) > > > > > postgres=# create subscription sub2 connection 'dbname

Re: Conflict detection for update_deleted in logical replication

2025-08-12 Thread shveta malik
On Mon, Aug 11, 2025 at 2:40 PM Zhijie Hou (Fujitsu) wrote: > > I agree. So, following the above points and some off-list discussions, I have > revised the option to be a subscription option in the V60 version. > Thank You for the patches. Tried to test the new sub-level parameter, have few comme

Re: Excessive LOG messages from replication slot sync worker

2025-08-11 Thread shveta malik
On Tue, Aug 12, 2025 at 8:52 AM Fujii Masao wrote: > > Hi, > > While testing the replication slot synchronization feature, I set up > three instances on the same machine: > > - Physical replication primary (also Logical replication publisher) > - Physical replication standby > - Logical replicatio

Re: Proposal: Conflict log history table for Logical Replication

2025-08-08 Thread shveta malik
On Fri, Aug 8, 2025 at 10:01 AM Dilip Kumar wrote: > > On Fri, Aug 8, 2025 at 8:58 AM shveta malik wrote: > > > > On Thu, Aug 7, 2025 at 3:08 PM Dilip Kumar wrote: > > > > > > So logically for PostgreSQL its an > > > user table but yeah

Re: Conflict detection for update_deleted in logical replication

2025-08-07 Thread shveta malik
On Thu, Aug 7, 2025 at 10:10 AM Zhijie Hou (Fujitsu) wrote: > > On Tuesday, August 5, 2025 10:09 AM Zhijie Hou (Fujitsu) > wrote: > > Here is V57 patch set which addressed most of comments. > > > > In this version, I also fixed a bug that the apply worker continued to find > > dead > > tuples e

Re: Issue with logical replication slot during switchover

2025-08-07 Thread shveta malik
On Thu, Aug 7, 2025 at 6:50 PM Fabrice Chapuis wrote: > > Hi, > > An issue occurred during the initial switchover using PostgreSQL version > 17.5. The setup consists of a cluster with two nodes, managed by Patroni > version 4.0.5. > Logical replication is configured on the same instance, and the

Re: Proposal: Conflict log history table for Logical Replication

2025-08-07 Thread shveta malik
On Thu, Aug 7, 2025 at 3:08 PM Dilip Kumar wrote: > > On Thu, Aug 7, 2025 at 1:43 PM shveta malik wrote: > > > > On Thu, Aug 7, 2025 at 12:25 PM shveta malik wrote: > > Thanks Shveta for your opinion on the design. > > > > On Tue, Aug 5

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-08-07 Thread shveta malik
On Fri, Aug 8, 2025 at 3:30 AM Masahiko Sawada wrote: > > On Tue, Aug 5, 2025 at 11:23 PM shveta malik wrote: > > > > Please find a few comments on v6: > > > > 1) > > +/* > > + * Initialize logical decoding status on shmem at server startup. This > &

Re: Conflict detection for update_deleted in logical replication

2025-08-07 Thread shveta malik
On Thu, Aug 7, 2025 at 6:05 PM Dilip Kumar wrote: > > On Mon, Aug 4, 2025 at 3:11 PM Amit Kapila wrote: > > > > On Mon, Aug 4, 2025 at 11:46 AM shveta malik wrote: > > > > > > 7) > > > Shall we rename 'max_conflict_retention_duration'

Re: Proposal: Conflict log history table for Logical Replication

2025-08-07 Thread shveta malik
On Thu, Aug 7, 2025 at 12:25 PM shveta malik wrote: > > On Tue, Aug 5, 2025 at 5:54 PM Dilip Kumar wrote: > > > > Currently we log conflicts to the server's log file and updates, this > > approach has limitations, 1) Difficult to query and analyze, parsing > >

Re: Proposal: Conflict log history table for Logical Replication

2025-08-06 Thread shveta malik
On Tue, Aug 5, 2025 at 5:54 PM Dilip Kumar wrote: > > Currently we log conflicts to the server's log file and updates, this > approach has limitations, 1) Difficult to query and analyze, parsing > plain text log files for conflict details is inefficient. 2) Lack of > structured data, key conflict

Re: Logical Replication of sequences

2025-08-06 Thread shveta malik
On Wed, Aug 6, 2025 at 4:29 PM shveta malik wrote: > > On Wed, Aug 6, 2025 at 2:28 PM vignesh C wrote: > > > > The attached v20250806 version patch has the changes for the same. > > > > Thank You for the patches. Please find a few comments: > > 1) >

Re: Logical Replication of sequences

2025-08-06 Thread shveta malik
On Wed, Aug 6, 2025 at 2:28 PM vignesh C wrote: > > The attached v20250806 version patch has the changes for the same. > Thank You for the patches. Please find a few comments: 1) * If 'resync_all_sequences' is false: * Add or remove tables and sequences that have been added to or removed

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-08-05 Thread shveta malik
Please find a few comments on v6: 1) +/* + * Initialize logical decoding status on shmem at server startup. This + * must be called ONCE during postmaster or standalone-backend startup, + * before initializing replication slots. + */ +void +StartupLogicalDecodingStatus(bool last_status) The comme

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-08-05 Thread shveta malik
On Wed, Aug 6, 2025 at 6:18 AM Masahiko Sawada wrote: > > > I've attached the updated version patch. > Thank You for the patch. The patch does not apply to the latest head due to conflict with slot-sync fix (commit-Id: 4614d53d). thanks Shveta

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-08-05 Thread shveta malik
On Wed, Aug 6, 2025 at 7:35 AM Ajin Cherian wrote: > > On Tue, Aug 5, 2025 at 4:22 PM Amit Kapila wrote: > > > > On Tue, Aug 5, 2025 at 9:28 AM shveta malik wrote: > > > > > > On Mon, Aug 4, 2025 at 3:41 PM Amit Kapila > > > wrote: > > >

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-08-05 Thread shveta malik
On Tue, Aug 5, 2025 at 5:14 AM Masahiko Sawada wrote: > > On Mon, Aug 4, 2025 at 3:38 AM shveta malik wrote: > > > > On Sat, Aug 2, 2025 at 4:53 AM Masahiko Sawada > > wrote: > > > > > > On Thu, Jul 31, 2025 at 5:00 AM Hayato Kuroda (Fujitsu) >

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-08-04 Thread shveta malik
On Mon, Aug 4, 2025 at 3:41 PM Amit Kapila wrote: > > On Mon, Aug 4, 2025 at 12:19 PM shveta malik wrote: > > > > On Mon, Aug 4, 2025 at 11:31 AM Amit Kapila wrote: > > > > > > On Fri, Aug 1, 2025 at 2:50 PM shveta malik > > > wrote: > > >

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-08-04 Thread shveta malik
On Sat, Aug 2, 2025 at 4:53 AM Masahiko Sawada wrote: > > On Thu, Jul 31, 2025 at 5:00 AM Hayato Kuroda (Fujitsu) > wrote: > > > > Dear Sawada-san, > > > > > I thought we could fix this issue by checking the number of in-use > > > logical slots while holding ReplicationSlotControlLock and > > > L

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-08-03 Thread shveta malik
On Mon, Aug 4, 2025 at 11:31 AM Amit Kapila wrote: > > On Fri, Aug 1, 2025 at 2:50 PM shveta malik wrote: > > > > 5) > > I tried a test where there were 4 slots on the publisher, where one > > was getting used while the others were not. Initiated > > pg_sync_

Re: Conflict detection for update_deleted in logical replication

2025-08-03 Thread shveta malik
On Fri, Aug 1, 2025 at 9:16 PM Zhijie Hou (Fujitsu) wrote: > > > Thanks for confirming. Here is V56 patch set which addressed all the > comments including the comments from Amit[1] and Shveta[2]. > > I have merged V55-0002 into 0001 and updated the list of author > and reviewers based on my knowle

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-08-01 Thread shveta malik
On Fri, Aug 1, 2025 at 12:02 PM shveta malik wrote: > > On Thu, Jul 31, 2025 at 3:11 PM Ajin Cherian wrote: > > > > > > Patch v3 attached. > > > > Thanks for the patch. I tested it, please find a few comments: > > > 1) > it hit

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-07-31 Thread shveta malik
On Thu, Jul 31, 2025 at 3:11 PM Ajin Cherian wrote: > > > Patch v3 attached. > Thanks for the patch. I tested it, please find a few comments: 1) it hits an assert (slotsync_reread_config()-->Assert(sync_replication_slots)) when API is trying to sync and is in wait loop while in another session,

Re: Conflict detection for update_deleted in logical replication

2025-07-31 Thread shveta malik
On Thu, Jul 31, 2025 at 3:49 PM Zhijie Hou (Fujitsu) wrote: > > On Thursday, July 31, 2025 5:26 PM shveta malik > wrote: > > > > On Tue, Jul 29, 2025 at 10:51 AM Zhijie Hou (Fujitsu) > > > > wrote: > > > > > > > > > This is th

Re: Conflict detection for update_deleted in logical replication

2025-07-31 Thread shveta malik
On Tue, Jul 29, 2025 at 10:51 AM Zhijie Hou (Fujitsu) wrote: > > > This is the V54 patch set, with only patch 0001 updated to address the latest > comments. > Thanks for the patch. While performing tests on the latest patch, I found an assert in tablesync worker in FindDeletedTupleInLocalRel (se

Re: Logical Replication of sequences

2025-07-30 Thread shveta malik
On Wed, Jul 30, 2025 at 11:16 AM shveta malik wrote: > > On Mon, Jul 28, 2025 at 3:37 PM vignesh C wrote: > > > Thanks for the comments, the attached v20250728 version patch has the > > changes for the same. > > > Thanks for the patches, please find a few comments:

Re: Logical Replication of sequences

2025-07-29 Thread shveta malik
On Mon, Jul 28, 2025 at 3:37 PM vignesh C wrote: > Thanks for the comments, the attached v20250728 version patch has the > changes for the same. > Thanks for the patches, please find a few comments: 1) WARNING: WITH clause parameters do not affect sequence synchronization a) How about: WITH cl

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-28 Thread shveta malik
On Fri, Jul 25, 2025 at 11:45 AM Masahiko Sawada wrote: > > Thank you for testing the patch! > > I've reworked the locking part in the patch. The attached v4 patch > should address all review comments including your previous > comments[1]. > Thank You for the patch. I have not reviewed fully, but

Re: Conflict detection for update_deleted in logical replication

2025-07-28 Thread shveta malik
On Mon, Jul 28, 2025 at 4:38 PM shveta malik wrote: > > On Fri, Jul 25, 2025 at 4:38 PM Zhijie Hou (Fujitsu) > wrote: > > > > > > The V53-0001 also includes Shveta's comments in [1]. > > > > Thanks, I have not yet completed the review, bu

Re: Conflict detection for update_deleted in logical replication

2025-07-28 Thread shveta malik
On Fri, Jul 25, 2025 at 4:38 PM Zhijie Hou (Fujitsu) wrote: > > > The V53-0001 also includes Shveta's comments in [1]. > Thanks, I have not yet completed the review, but please find a few comments on 001: 1) IsIndexUsableForFindingDeletedTuple() We first have: + /* + * A frozen transaction ID in

Re: Conflict detection for update_deleted in logical replication

2025-07-25 Thread shveta malik
On Fri, Jul 25, 2025 at 2:31 PM Amit Kapila wrote: > > On Fri, Jul 25, 2025 at 12:37 PM shveta malik wrote: > > > > On Thu, Jul 24, 2025 at 9:12 AM shveta malik wrote: > > > > > > > > > 2) > > >

Re: Logical replication launcher did not automatically restart when got SIGKILL

2025-07-25 Thread shveta malik
On Fri, Jul 25, 2025 at 7:17 AM Fujii Masao wrote: > > On Thu, Jul 24, 2025 at 6:46 PM shveta malik wrote: > > Sounds reasonable. > > Thinking out loud, when cleaning up after a backend or background > > worker crash, process_pm_child_exit() is invoked, which sub

Re: Conflict detection for update_deleted in logical replication

2025-07-25 Thread shveta malik
On Thu, Jul 24, 2025 at 9:12 AM shveta malik wrote: > > > 2) > + if (MySubscription->retaindeadtuples && > + FindMostRecentlyDeletedTupleInfo(localrel, remoteslot, > + > &conflicttuple.xmin, > + &g

Re: Logical replication launcher did not automatically restart when got SIGKILL

2025-07-24 Thread shveta malik
On Thu, Jul 24, 2025 at 2:39 PM Fujii Masao wrote: > > On Thu, Jul 17, 2025 at 6:58 PM shveta malik wrote: > > > > On Wed, Jul 16, 2025 at 8:51 AM cca5507 wrote: > > > > > > Hi, > > > > > > The v1-0002 in [1] will call ReportBa

Re: Conflict detection for update_deleted in logical replication

2025-07-23 Thread shveta malik
On Wed, Jul 23, 2025 at 12:53 PM Zhijie Hou (Fujitsu) wrote: > > On Wednesday, July 23, 2025 12:08 PM Amit Kapila > wrote: > > > > On Wed, Jul 23, 2025 at 3:51 AM Masahiko Sawada > > wrote: > > > > > > I've reviewed the 0001 patch and it looks good to me. > > > > > > > Thanks, I have pushed the

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-22 Thread shveta malik
On Tue, Jul 22, 2025 at 5:03 AM Masahiko Sawada wrote: > > Yes, I agree. The main patch focuses on the part where we > automatically change the effective WAL level upon the logical slot > creation and deletion (and potentially remove 'logical' from > wal_level), and other things are implemented as

Re: Skipping schema changes in publication

2025-07-22 Thread shveta malik
I further tested inherited tables flow as well wrt ONLY and EXCEPT, it works well. But while reading docs for the saem, I have few concerns. 1) While explaining ONLY for EXCEPT, create-publication doc says this + This does not apply to a partitioned table, however. The partitions of +

Re: Skipping schema changes in publication

2025-07-22 Thread shveta malik
Shlok, I was trying to validate the interaction of 'publish_via_partition_root' with 'EXCEPT". Found some unexpected behaviour, can you please review: Pub: - CREATE TABLE tab_root (range_col int,i int,j int) PARTITION BY RANGE (range_col); CREATE TABLE tab_part_1 PARTITION OF tab_root FOR

Re: Skipping schema changes in publication

2025-07-22 Thread shveta malik
On Sat, Jul 19, 2025 at 4:17 PM Shlok Kyal wrote: > > On Mon, 30 Jun 2025 at 16:25, shveta malik wrote: > > > > Few more comments on 002: > > > > 5) > > +GetAllTablesPublicationRelations(Oid pubid, bool pubviaroot) > > { > > &

Re: Skipping schema changes in publication

2025-07-21 Thread shveta malik
On Sat, Jul 19, 2025 at 4:17 PM Shlok Kyal wrote: > > On Mon, 30 Jun 2025 at 16:25, shveta malik wrote: > > > > Few more comments on 002: > > > > 5) > > +GetAllTablesPublicationRelations(Oid pubid, bool pubviaroot) > > { > > &

Re: Logical Replication of sequences

2025-07-21 Thread shveta malik
On Mon, Jul 21, 2025 at 2:55 PM Amit Kapila wrote: > > On Mon, Jul 21, 2025 at 2:36 PM vignesh C wrote: > > > > On Mon, 21 Jul 2025 at 11:15, Dilip Kumar wrote: > > > > > > > > 3. Some of the syntaxes works for sequence which doesn't make sense to > > > me, as listed below, I think there are mor

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-21 Thread shveta malik
On Mon, Jul 21, 2025 at 12:23 PM Amit Kapila wrote: > > I am slightly hesitant to introduce multiple ways to enable logical > decoding/replication unless that is the only path as giving multiple > options to achieve the same thing can confuse users as to which one is > preferable and pros/cons of

Re: Logical Replication of sequences

2025-07-20 Thread shveta malik
On Mon, Jul 21, 2025 at 11:15 AM Dilip Kumar wrote: > > 3. Some of the syntaxes works for sequence which doesn't make sense to > me, as listed below, I think there are more > > postgres[154731]=# CREATE PUBLICATION insert_only FOR ALL SEQUENCES > WITH (publish = 'insert'); > CREATE PUBLICATION > >

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-20 Thread shveta malik
On Mon, Jul 21, 2025 at 10:48 AM shveta malik wrote: > > On Fri, Jul 18, 2025 at 3:03 PM Amit Kapila wrote: > > > > On Fri, Jul 18, 2025 at 2:27 PM shveta malik wrote: > > > > > > On Thu, Jul 17, 2025 at 3:06 PM Amit Kapila > > > wrote: > >

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-20 Thread shveta malik
On Fri, Jul 18, 2025 at 3:03 PM Amit Kapila wrote: > > On Fri, Jul 18, 2025 at 2:27 PM shveta malik wrote: > > > > On Thu, Jul 17, 2025 at 3:06 PM Amit Kapila wrote: > > > > > > On Wed, Jun 18, 2025 at 3:23 PM shveta malik > > > wrote: > > &

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-07-20 Thread shveta malik
On Sat, Jul 19, 2025 at 5:10 PM Amit Kapila wrote: > > On Fri, Jul 18, 2025 at 11:31 AM Dilip Kumar wrote: > > > > On Fri, Jul 18, 2025 at 11:25 AM shveta malik > > wrote: > > > > > > Okay. I see your point. Yes, it was non-blocking earlier but

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-18 Thread shveta malik
On Thu, Jul 17, 2025 at 3:06 PM Amit Kapila wrote: > > On Wed, Jun 18, 2025 at 3:23 PM shveta malik wrote: > > > > On Wed, Jun 18, 2025 at 2:39 PM Bertrand Drouvot > > wrote: > > > > > > Hi, > > > > > > On Tue, Jun 10, 2025 at 02:

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-07-17 Thread shveta malik
On Fri, Jul 18, 2025 at 10:52 AM Dilip Kumar wrote: > > On Fri, Jul 18, 2025 at 10:45 AM shveta malik wrote: > > > > On Fri, Jul 18, 2025 at 10:14 AM Dilip Kumar wrote: > > > > > > On Thu, Jul 17, 2025 at 9:34 AM shveta malik > > > wrote: > &g

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-07-17 Thread shveta malik
On Fri, Jul 18, 2025 at 10:14 AM Dilip Kumar wrote: > > On Thu, Jul 17, 2025 at 9:34 AM shveta malik wrote: > > > > On Wed, Jul 16, 2025 at 3:47 PM Ajin Cherian wrote: > > > > > > > I am not able to apply the patch to the latest head or even to a week >

Re: Conflict detection for update_deleted in logical replication

2025-07-17 Thread shveta malik
On Thu, Jul 17, 2025 at 9:56 AM Dilip Kumar wrote: > > On Fri, Jul 11, 2025 at 4:28 PM Amit Kapila wrote: > > > > On Thu, Jul 10, 2025 at 6:46 PM Masahiko Sawada > > wrote: > > > > > > On Wed, Jul 9, 2025 at 9:09 PM Amit Kapila > > > wrote: > > > > > > > > > > > > I think that even with retai

Re: Logical replication launcher did not automatically restart when got SIGKILL

2025-07-17 Thread shveta malik
On Wed, Jul 16, 2025 at 8:51 AM cca5507 wrote: > > Hi, > > The v1-0002 in [1] will call ReportBackgroundWorkerExit() which will send > SIGUSR1 to 'bgw_notify_pid', but it may already exit in HandleChildCrash(), > is this ok? > Shall ReportBackgroundWorkerExit() be skipped for 'crashed' backgrou

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-07-16 Thread shveta malik
On Wed, Jul 16, 2025 at 3:47 PM Ajin Cherian wrote: > > > I am not able to apply the patch to the latest head or even to a week > > back version. Can you please check and rebase? > > > > thanks > > Shveta > > Rebased. > Thanks. Please find a few comments: 1) /* Any slot with NULL in these field

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-07-16 Thread shveta malik
On Wed, Jul 16, 2025 at 3:00 PM Ajin Cherian wrote: > > On Wed, Jul 2, 2025 at 7:56 PM shveta malik wrote: > > > > Few comments: > > > > 1) > > When the API is waiting for the primary to advance, standby fails to > > handle promotion requests. Promot

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-15 Thread shveta malik
On Tue, Jul 15, 2025 at 10:37 PM Masahiko Sawada wrote: > > I've attached updated patches that implement the idea we've discussed. > The patches still need to be polished but the implemented ideas seem > good. Feedback is very welcome. > Thank You for the patches. I just tried my hands on ptach00

Re: Logical replication launcher did not automatically restart when got SIGKILL

2025-07-15 Thread shveta malik
On Tue, Jul 15, 2025 at 2:56 PM cca5507 wrote: > > Hi, hackers > > I found the $SUBJECT, the main reason is that RegisteredBgWorker::rw_pid has > not been cleaned. > > Attach a patch to fix it. > Thank You for reporting this. The problem exists and the patch works as expected. In the patch, we

Re: Logical Replication of sequences

2025-07-14 Thread shveta malik
On Mon, Jul 14, 2025 at 10:03 AM vignesh C wrote: > > On Fri, 11 Jul 2025 at 14:26, shveta malik wrote: > > > > On Wed, Jul 9, 2025 at 4:11 PM vignesh C wrote: > > > > > > > > > 3) > > > > SyncFetchRelationStates: > > > >

Re: [WIP]Vertical Clustered Index (columnar store extension) - take2

2025-07-14 Thread shveta malik
On Mon, Jul 14, 2025 at 2:08 PM Peter Smith wrote: > > Hi Shveta, > > Thanks for your README questions. > > On Fri, Jul 11, 2025 at 1:46 PM shveta malik wrote: > > > > Thank You for working on this. I started going through the README and > > tried run

Re: Logical Replication of sequences

2025-07-11 Thread shveta malik
On Wed, Jul 9, 2025 at 4:11 PM vignesh C wrote: > > > 3) > > SyncFetchRelationStates: > > Earlier the name was FetchTableStates. If we really want to use the > > 'Sync' keyword, we can name it FetchRelationSyncStates, as we are > > fetching sync-status only. Thoughts? > > Instead of FetchRelatio

Re: [WIP]Vertical Clustered Index (columnar store extension) - take2

2025-07-10 Thread shveta malik
Thank You for working on this. I started going through the README and tried running simple tests, have few concerns: 1) I am not able to understand section 4.2 'WOS-to-ROS conversion'. When whiteout-WOS says 'delete 4', what does that mean? 4 is CRID, TXID? And when does delete-vector X represents

Re: Logical Replication of sequences

2025-07-09 Thread shveta malik
Please find a few more comments on July4 patch 6) + + To synchronize sequences from a publisher to a subscriber, first publish + them using + CREATE PUBLICATION ... FOR ALL SEQUENCES. + This sentence looks odd, as we have 'first' but no follow-up sentence after that. Can we please comb

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-07-08 Thread shveta malik
Please find few more comments: 1) In pg_sync_replication_slots() doc, we have this: "Note that this function is primarily intended for testing and debugging purposes and should be used with caution. Additionally, this function cannot be executed if " We can get rid of this info as well and

  1   2   3   4   5   6   7   >