Re: Function to get invalidation cause of a replication slot.

Hi, On 12/20/23 10:55 AM, shveta malik wrote: On Wed, Dec 20, 2023 at 2:20 PM Amit Kapila wrote: PFA v2 patch. Addressed below comments: 1) Added test in 019_replslot_limit.pl 2) 'pg_get_slot_invalidation_cause' now returns error if the given slot does not exist 3) Corrected doc and commit

Re: Function to get invalidation cause of a replication slot.

Hi, On 12/20/23 10:57 AM, shveta malik wrote: On Wed, Dec 20, 2023 at 12:46 PM Drouvot, Bertrand Worth to add some coverage in 019_replslot_limit.pl and 035_standby_logical_decoding.pl? I have recently added a test in 019_replslot_limit.pl in v2. Thanks! Do you suggest adding

Re: Function to get invalidation cause of a replication slot.

Hi, On 12/20/23 9:50 AM, Amit Kapila wrote: On Wed, Dec 20, 2023 at 12:46 PM Drouvot, Bertrand wrote: On 12/20/23 7:01 AM, shveta malik wrote: Hello hackers, Attached is a patch which attempts to implement a new system function pg_get_slot_invalidation_cause('slot_name') to get

Re: Function to get invalidation cause of a replication slot.

Hi, On 12/20/23 7:01 AM, shveta malik wrote: Hello hackers, Attached is a patch which attempts to implement a new system function pg_get_slot_invalidation_cause('slot_name') to get invalidation cause of a replication slot. Thanks! +1 for the idea to display this information through an SQL

Re: Add isCatalogRel in rmgrdesc

Hi, On 12/19/23 9:00 AM, Masahiko Sawada wrote: On Tue, Dec 12, 2023 at 6:15 PM Michael Paquier wrote: On Tue, Dec 12, 2023 at 09:23:46AM +0100, Drouvot, Bertrand wrote: Please find attached a patch to add this field in the related rmgrdesc (i.e all the ones that already provide

Re: Move walreceiver state assignment (to WALRCV_STREAMING) in WalReceiverMain()

Hi, On 12/13/23 3:33 PM, Michael Paquier wrote: On Tue, Dec 12, 2023 at 04:54:32PM -0300, Euler Taveira wrote: Couldn't it give up before starting if you apply your patch? My main concern is due to a slow system, the walrcv_connect() took to long in WalReceiverMain() and the code above kills

Re: Move walreceiver state assignment (to WALRCV_STREAMING) in WalReceiverMain()

Hi, On 12/12/23 8:54 PM, Euler Taveira wrote: On Tue, Dec 12, 2023, at 12:58 PM, Drouvot, Bertrand wrote: Currently walrcv->walRcvState is set to WALRCV_STREAMING at the beginning of WalReceiverMain(). But it seems that after this assignment things could be wrong before the walreice

Move walreceiver state assignment (to WALRCV_STREAMING) in WalReceiverMain()

Hi hackers, Currently walrcv->walRcvState is set to WALRCV_STREAMING at the beginning of WalReceiverMain(). But it seems that after this assignment things could be wrong before the walreicever actually starts streaming (like not being able to connect to the primary). It looks to me that

Re: Add isCatalogRel in rmgrdesc

Hi, On 12/12/23 10:15 AM, Michael Paquier wrote: On Tue, Dec 12, 2023 at 09:23:46AM +0100, Drouvot, Bertrand wrote: Please find attached a patch to add this field in the related rmgrdesc (i.e all the ones that already provide the snapshotConflictHorizon except the one related

Add isCatalogRel in rmgrdesc

Hi hackers, 6af1793954 added a new field namely "isCatalogRel" in some WAL records to help detecting row removal conflict during logical decoding from standby. Please find attached a patch to add this field in the related rmgrdesc (i.e all the ones that already provide the

Re: Synchronizing slots from primary to standby

Hi, On 12/8/23 10:06 AM, Amit Kapila wrote: On Wed, Dec 6, 2023 at 4:53 PM shveta malik wrote: PFA v43, changes are: I wanted to discuss 0003 patch about cascading standby's. It is not clear to me whether we want to allow physical standbys to further wait for cascading standby to sync

Re: Synchronizing slots from primary to standby

Hi, On 12/7/23 10:07 AM, shveta malik wrote: On Thu, Dec 7, 2023 at 1:19 PM Drouvot, Bertrand wrote: Might be worth to add comments in the code (around the WalRcv->latestWalEnd checks) that no "lagging" sync are possible if the walreceiver is not started though? I am a bit con

Re: Synchronizing slots from primary to standby

Hi, On 12/6/23 12:23 PM, shveta malik wrote: On Wed, Dec 6, 2023 at 4:28 PM shveta malik wrote: PFA v43, changes are: Thanks! v43-001: 1) Support of 'failover' dump in pg_dump. It was missing earlier. v43-002: 1) Slot-sync worker now checks validity of primary_slot_name by connecting

Re: Synchronizing slots from primary to standby

Hi, On 12/6/23 11:58 AM, shveta malik wrote: On Wed, Dec 6, 2023 at 3:00 PM Drouvot, Bertrand wrote: Hi, On 12/6/23 7:18 AM, shveta malik wrote: On Wed, Dec 6, 2023 at 10:56 AM Amit Kapila wrote: I feel that is indirectly relying on the fact that the primary won't advance logical slots

Re: Synchronizing slots from primary to standby

Hi, On 12/6/23 7:18 AM, shveta malik wrote: On Wed, Dec 6, 2023 at 10:56 AM Amit Kapila wrote: I feel that is indirectly relying on the fact that the primary won't advance logical slots unless physical standby has consumed data. Yes, that is the basis of this discussion. Yes. But now

Re: Synchronizing slots from primary to standby

Hi, On 12/5/23 12:32 PM, Amit Kapila wrote: On Tue, Dec 5, 2023 at 10:38 AM shveta malik wrote: On Mon, Dec 4, 2023 at 10:07 PM Drouvot, Bertrand wrote: Maybe another option could be to have the walreceiver a way to let the slot sync worker knows that it (the walreceiver) was not able

Re: Synchronizing slots from primary to standby

Hi, On 12/5/23 11:29 AM, shveta malik wrote: On Tue, Dec 5, 2023 at 2:18 PM Drouvot, Bertrand wrote: Wouldn't that make sense to move it once we are sure that walrcv_startstreaming() returns true and first_stream is true, here? " if (first_s

Re: Synchronizing slots from primary to standby

Hi, On 12/5/23 6:08 AM, shveta malik wrote: On Mon, Dec 4, 2023 at 10:07 PM Drouvot, Bertrand wrote: Maybe another option could be to have the walreceiver a way to let the slot sync worker knows that it (the walreceiver) was not able to start due to non existing replication slot

Re: Synchronizing slots from primary to standby

Hi, On 12/4/23 6:10 AM, shveta malik wrote: On Fri, Dec 1, 2023 at 5:40 PM Nisha Moond wrote: Review for v41 patch. Thanks for the feedback. ~~~ 2. IIUC, the slotsyncworker's connection to the primary is to execute a query. Its aim is not walsender type connection, but at primary when

Re: Synchronizing slots from primary to standby

Hi, On 12/4/23 4:33 AM, Zhijie Hou (Fujitsu) wrote: On Wednesday, November 29, 2023 5:55 PM Drouvot, Bertrand wrote: I think I'm fine with documenting the fact that the user should not change the failover value. But if he does change it (because at the end nothing prevents it to do so

Re: Synchronizing slots from primary to standby

Hi, On 12/1/23 12:06 PM, Amit Kapila wrote: On Wed, Nov 29, 2023 at 3:24 PM Drouvot, Bertrand wrote: I think I'm fine with documenting the fact that the user should not change the failover value. But if he does change it (because at the end nothing prevents it to do so) then I think

Re: Synchronizing slots from primary to standby

Hi, On 12/1/23 4:19 AM, shveta malik wrote: On Thu, Nov 30, 2023 at 5:37 PM Ajin Cherian wrote: 1. In my opinion, the second message "aborting the wait...moving to the next slot" does not hold much value. There might not even be a "next slot", there might be just one slot. I think the first

Re: Synchronizing slots from primary to standby

Hi, On 11/30/23 1:06 PM, Ajin Cherian wrote: On Wed, Nov 29, 2023 at 8:17 PM Zhijie Hou (Fujitsu) 3. If creation of a slot on the standby fails for one slot because a slot of the same name exists, then thereafter no new sync slots are created on standby. Is this expected? I do see that

Re: Synchronizing slots from primary to standby

Hi, On 11/29/23 3:58 AM, Amit Kapila wrote: On Tue, Nov 28, 2023 at 2:17 PM Drouvot, Bertrand wrote: What do you think about also adding a pg_alter_logical_replication_slot() or such function? That would allow users to alter manually created logical replication slots without the need

Re: Synchronizing slots from primary to standby

Hi, On 11/29/23 6:58 AM, Zhijie Hou (Fujitsu) wrote: On Tuesday, November 28, 2023 8:07 PM Drouvot, Bertrand wrote: Hi, On 11/27/23 9:57 AM, Zhijie Hou (Fujitsu) wrote: On Monday, November 27, 2023 4:51 PM shveta malik wrote: Here is the updated version(v39_2) which include all

Re: Synchronizing slots from primary to standby

Hi, On 11/28/23 10:40 AM, shveta malik wrote: On Tue, Nov 28, 2023 at 12:19 PM Drouvot, Bertrand wrote: Hi, On 11/28/23 4:13 AM, shveta malik wrote: On Mon, Nov 27, 2023 at 4:08 PM Amit Kapila wrote: On Mon, Nov 27, 2023 at 2:27 PM Zhijie Hou (Fujitsu) wrote: Here is the updated

Re: Synchronizing slots from primary to standby

Hi, On 11/27/23 9:57 AM, Zhijie Hou (Fujitsu) wrote: On Monday, November 27, 2023 4:51 PM shveta malik wrote: Here is the updated version(v39_2) which include all the changes made in 0002. Please use for review, and sorry for the confusion. Thanks! As far v39_2-0001: " Altering the

Re: Synchronizing slots from primary to standby

Hi, On 11/2/23 1:27 AM, Zhijie Hou (Fujitsu) wrote: On Tuesday, October 31, 2023 6:45 PM Amit Kapila wrote: We have create_replication_slot and drop_replication_slot in repl_gram.y. How about if introduce alter_replication_slot and handle the 'failover' flag with that? The idea is we will

Re: Synchronizing slots from primary to standby

Hi, On 11/28/23 4:13 AM, shveta malik wrote: On Mon, Nov 27, 2023 at 4:08 PM Amit Kapila wrote: On Mon, Nov 27, 2023 at 2:27 PM Zhijie Hou (Fujitsu) wrote: Here is the updated version(v39_2) which include all the changes made in 0002. Please use for review, and sorry for the confusion.

Re: [PATCH] pg_convert improvement

Hi, On 11/28/23 2:16 AM, Yurii Rashkovskii wrote:  Hi Bertrand, Did some minor changes in the attached: - Started the multi-line comment with an upper case and finished it with a "." and re-worded a bit. - Ran pgindent What do you think about the attached? It looks

Re: Synchronizing slots from primary to standby

Hi, On 11/27/23 1:23 PM, Zhijie Hou (Fujitsu) wrote: On Monday, November 27, 2023 8:05 PM Drouvot, Bertrand wrote: Hi, On 11/6/23 2:30 AM, Zhijie Hou (Fujitsu) wrote: On Friday, November 3, 2023 7:32 PM Amit Kapila I don't see a corresponding change in repl_gram.y. I think

Re: Synchronizing slots from primary to standby

Hi, On 11/27/23 1:23 PM, Zhijie Hou (Fujitsu) wrote: On Monday, November 27, 2023 8:05 PM Drouvot, Bertrand wrote: Did not look in details but it looks like there is more to do here as this is failing (with v39_2): " postgres@primary: psql replication=database psql (17devel) Type

Re: Synchronizing slots from primary to standby

Hi, On 11/6/23 2:30 AM, Zhijie Hou (Fujitsu) wrote: On Friday, November 3, 2023 7:32 PM Amit Kapila I don't see a corresponding change in repl_gram.y. I think the following part of the code needs to be changed: /* CREATE_REPLICATION_SLOT slot [TEMPORARY] LOGICAL plugin [options] */ |

Re: Synchronizing slots from primary to standby

Hi, On 11/27/23 7:02 AM, Zhijie Hou (Fujitsu) wrote: On Monday, November 27, 2023 12:03 PM Zhijie Hou (Fujitsu) wrote: Attach the V38 patch set which addressed all comments in [1][2] except for the ones that mentioned above. [1]

Re: [PATCH] pg_convert improvement

Hi, On 11/24/23 3:32 PM, Yurii Rashkovskii wrote: Hi Bertrand, On Fri, Nov 24, 2023 at 6:26 AM Drouvot, Bertrand mailto:bertranddrouvot...@gmail.com>> wrote: The patch is pretty straightforward, I just have one remark: +       /* if no actual conversion happened, return the or

Re: [PATCH] pg_convert improvement

Hi, On 11/24/23 3:05 PM, Yurii Rashkovskii wrote: Hi, I propose a patch that ensures `pg_convert` doesn't allocate and copy data when no conversion is done. It is an unnecessary overhead, especially when such conversions are done frequently and for large values. +1 for the patch, I think

Re: Synchronizing slots from primary to standby

Hi, On 11/24/23 10:45 AM, Amit Kapila wrote: On Fri, Nov 24, 2023 at 1:53 PM Drouvot, Bertrand wrote: On 11/24/23 4:35 AM, Zhijie Hou (Fujitsu) wrote: On Thursday, November 23, 2023 11:45 PM Drouvot, Bertrand wrote: IOW, the logical slot's LSN can still be advanced after the walreceiver

Re: Synchronizing slots from primary to standby

Hi, On 11/23/23 11:45 AM, Amit Kapila wrote: On Wed, Nov 22, 2023 at 10:02 AM Zhijie Hou (Fujitsu) wrote: Or we could just document that it is user's responsibility to match the failover property in case it changes the slot_name. Personally, I think we should document this behavior

Re: Synchronizing slots from primary to standby

Hi, On 11/24/23 4:35 AM, Zhijie Hou (Fujitsu) wrote: On Thursday, November 23, 2023 11:45 PM Drouvot, Bertrand wrote: IOW, the logical slot's LSN can still be advanced after the walreceiver shutdown if it was far bebind the physical slot's LSN. oh yeah right, it would need much more work

Re: Synchronizing slots from primary to standby

Hi, On 11/23/23 6:13 AM, Amit Kapila wrote: On Tue, Nov 21, 2023 at 4:35 PM Drouvot, Bertrand wrote: On 11/21/23 10:32 AM, shveta malik wrote: On Tue, Nov 21, 2023 at 2:02 PM shveta malik wrote: v37 fails to apply to HEAD due to a recent commit e83aa9f92fdd, rebased the patches. PFA

Re: Synchronizing slots from primary to standby

Hi, On 11/21/23 10:32 AM, shveta malik wrote: On Tue, Nov 21, 2023 at 2:02 PM shveta malik wrote: v37 fails to apply to HEAD due to a recent commit e83aa9f92fdd, rebased the patches. PFA v37_2 patches. Thanks! Regarding the promotion flow: If the primary is available and reachable I

Re: Synchronizing slots from primary to standby

Hi, On 11/21/23 6:16 AM, Amit Kapila wrote: On Mon, Nov 20, 2023 at 6:51 PM Drouvot, Bertrand wrote: As far the 'i' state here, from what I see, it is currently useful for: 1. Cascading standby to not sync slots with state = 'i' from the first standby. 2. Easily report Slots that did

Re: Synchronizing slots from primary to standby

On 11/20/23 11:59 AM, Amit Kapila wrote: On Mon, Nov 20, 2023 at 3:17 PM Drouvot, Bertrand wrote: On 11/18/23 11:45 AM, Amit Kapila wrote: On Fri, Nov 17, 2023 at 5:18 PM Drouvot, Bertrand wrote: On 11/17/23 2:46 AM, Zhijie Hou (Fujitsu) wrote: On Tuesday, November 14, 2023 10:27 PM

Re: Synchronizing slots from primary to standby

Hi, On 11/18/23 11:45 AM, Amit Kapila wrote: On Fri, Nov 17, 2023 at 5:18 PM Drouvot, Bertrand wrote: On 11/17/23 2:46 AM, Zhijie Hou (Fujitsu) wrote: On Tuesday, November 14, 2023 10:27 PM Drouvot, Bertrand wrote: I feel the WaitForWALToBecomeAvailable may not be the best place

Re: Synchronizing slots from primary to standby

Hi, On 11/17/23 2:46 AM, Zhijie Hou (Fujitsu) wrote: On Tuesday, November 14, 2023 10:27 PM Drouvot, Bertrand wrote: On 11/13/23 2:57 PM, Zhijie Hou (Fujitsu) wrote: On Friday, November 10, 2023 4:16 PM Drouvot, Bertrand wrote: Yeah good point, agree to just error out in all the case

Re: Synchronizing slots from primary to standby

Hi, On 11/16/23 1:03 PM, shveta malik wrote: On Thu, Nov 16, 2023 at 3:43 PM Amit Kapila wrote: PFA v35. It has below changes: Thanks for the update! 6) shutdown the slotsync worker on promotion. + /* +* Shutdown the slot sync workers to prevent potential conflicts between +*

Re: Synchronizing slots from primary to standby

Hi, On 11/13/23 2:57 PM, Zhijie Hou (Fujitsu) wrote: On Friday, November 10, 2023 4:16 PM Drouvot, Bertrand wrote: Yeah good point, agree to just error out in all the case then (if we discard the sync_ reserved wording proposal, which seems to be the case as probably not worth the extra work

Re: Split index and table statistics into different types of stats

Hi, On 11/13/23 9:44 PM, Andres Freund wrote: Hi, On 2023-11-13 09:26:56 +0100, Drouvot, Bertrand wrote: --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -799,11 +799,19 @@ ReadBufferExtended(Relation reln, ForkNumber forkNum, BlockNumber blockNum

Re: Synchronizing slots from primary to standby

Hi, On 11/13/23 5:24 AM, shveta malik wrote: On Thu, Nov 9, 2023 at 8:56 AM Amit Kapila wrote: Apart from the above, I would like to discuss the slot sync work distribution strategy of this patch. The current implementation as explained in the commit message [1] works well if the slots

Re: Synchronizing slots from primary to standby

Hi, On 11/10/23 4:31 AM, shveta malik wrote: On Thu, Nov 9, 2023 at 9:15 PM Drouvot, Bertrand wrote: Yeah I think so, because there is a time window when one could "use" the slot after the promotion and before it is removed. Producing things like: " 2023-11-09 15:16:50.294 UT

Re: Synchronizing slots from primary to standby

Hi, On 11/10/23 8:55 AM, Amit Kapila wrote: On Fri, Nov 10, 2023 at 12:50 PM Drouvot, Bertrand wrote: But even if we ERROR out instead of emitting a WARNING, the user would still need to be notified/monitor such errors. I agree that then probably they will come to know earlier because

Re: Synchronizing slots from primary to standby

Hi, On 11/10/23 6:41 AM, Amit Kapila wrote: On Thu, Nov 9, 2023 at 7:29 PM Drouvot, Bertrand wrote: Are you saying that we change the state of the already existing slot on standby? Yes. And, such a state would indicate that we are trying to sync the slot with the same name from

Re: Synchronizing slots from primary to standby

Hi, On 11/9/23 11:54 AM, shveta malik wrote: PFA v32 patches which has below changes: Thanks! 7) Added warning for cases where a user-slot with the same name is already present which slot-sync worker is trying to create. Sync for such slots is skipped. I'm seeing assertion and segfault

Re: Synchronizing slots from primary to standby

Hi, On 11/9/23 3:41 AM, Amit Kapila wrote: On Wed, Nov 8, 2023 at 8:09 PM Drouvot, Bertrand wrote: Unrelated to above, if there is a user slot on standby with the same name which the slot-sync worker is trying to create, then shall it emit a warning and skip the sync of that slot or shall

Re: Synchronizing slots from primary to standby

Hi, On 11/8/23 12:50 PM, shveta malik wrote: On Wed, Nov 8, 2023 at 3:19 PM Drouvot, Bertrand wrote: Hi, On 11/8/23 9:57 AM, Amit Kapila wrote: On Wed, Nov 8, 2023 at 12:32 PM Drouvot, Bertrand wrote: Hi, On 11/8/23 4:50 AM, Amit Kapila wrote: I think if we want to follow

Re: Synchronizing slots from primary to standby

Hi, On 11/8/23 9:57 AM, Amit Kapila wrote: On Wed, Nov 8, 2023 at 12:32 PM Drouvot, Bertrand wrote: Hi, On 11/8/23 4:50 AM, Amit Kapila wrote: I think if we want to follow this approach then we need to also monitor these slots for any change in the consecutive cycles and if we are able

Re: Synchronizing slots from primary to standby

Hi, On 11/8/23 4:50 AM, Amit Kapila wrote: On Tue, Nov 7, 2023 at 7:58 PM Drouvot, Bertrand wrote: If we think this window is too short we could: - increase it or - don't drop the slot once created (even if there is no activity on the primary during PrimaryCatchupWaitAttempt attempts) so

Re: Synchronizing slots from primary to standby

Hi, On 11/7/23 11:55 AM, Amit Kapila wrote: On Tue, Nov 7, 2023 at 3:51 PM Drouvot, Bertrand wrote: On 10/31/23 10:37 AM, shveta malik wrote: On Fri, Oct 27, 2023 at 8:43 PM Drouvot, Bertrand Agree with your test case, but in my case I was not using pub/sub. I was not clear, so when I

Re: Synchronizing slots from primary to standby

Hi, On 10/31/23 10:37 AM, shveta malik wrote: On Fri, Oct 27, 2023 at 8:43 PM Drouvot, Bertrand Agree with your test case, but in my case I was not using pub/sub. I was not clear, so when I said: - create logical_slot1 on the primary (and don't start using it) I meant don't start

Re: Synchronizing slots from primary to standby

Hi, On 10/27/23 10:35 AM, Amit Kapila wrote: On Wed, Oct 25, 2023 at 3:15 PM Drouvot, Bertrand I think even if we provide such an API, we need to have logic to get the slots from the primary and create them. Yeah, my idea was to add an API (in addition to what is already in place). Say

Re: Synchronizing slots from primary to standby

Hi, On 10/27/23 10:51 AM, shveta malik wrote: On Wed, Oct 25, 2023 at 3:15 PM Drouvot, Bertrand wrote: I discussed this with my colleague Hou-San and we think that one possibility could be to somehow accelerate the increment of restart_lsn on primary. This can be achieved by connecting

Re: Synchronizing slots from primary to standby

Hi, On 10/27/23 11:56 AM, shveta malik wrote: On Wed, Oct 25, 2023 at 3:15 PM Drouvot, Bertrand wrote: Hi, On 10/25/23 5:00 AM, shveta malik wrote: On Tue, Oct 24, 2023 at 11:54 AM Drouvot, Bertrand wrote: Hi, On 10/23/23 2:56 PM, shveta malik wrote: On Mon, Oct 23, 2023 at 5:52 PM

Re: Reconcile stats in find_tabstat_entry() and get rid of PgStat_BackendFunctionEntry

Hi, On 10/27/23 8:07 AM, Michael Paquier wrote: The part that I found disturbing is here: + tabentry = (PgStat_TableStatus *) entry_ref->pending; + tablestatus = palloc(sizeof(PgStat_TableStatus)); + *tablestatus = *tabentry; This causes tablestatus->trans to point to the

Re: Synchronizing slots from primary to standby

Hi, On 10/26/23 10:40 AM, Amit Kapila wrote: On Wed, Oct 25, 2023 at 8:49 PM Drouvot, Bertrand wrote: Good point, I think we should enhance the WalSndWait() logic to address this case. Agree. I think it would need to take care of the new CV and probably provide a way for the caller

Re: Reconcile stats in find_tabstat_entry() and get rid of PgStat_BackendFunctionEntry

Hi, On 3/27/23 8:35 AM, Michael Paquier wrote: On Fri, Mar 24, 2023 at 08:00:44PM -0700, Andres Freund wrote: I don't understand what we're optimizing for here. These functions are very very very far from being a hot path. The xact functions are barely ever used. Compared to the cost of query

Re: Synchronizing slots from primary to standby

Hi, On 10/9/23 12:30 PM, shveta malik wrote: PFA v22 patch-set. It has below changes: patch 001: 1) Now physical walsender wakes up logical walsender(s) by using a new CV as suggested in [1] Thanks! I think that works fine as long as the standby is up and running and catching up. The

Re: Synchronizing slots from primary to standby

Hi, On 10/25/23 6:57 AM, Amit Kapila wrote: On Tue, Oct 24, 2023 at 3:35 PM Drouvot, Bertrand wrote: On 10/24/23 7:44 AM, Ajin Cherian wrote: On Mon, Oct 23, 2023 at 11:22 PM Drouvot, Bertrand wrote: 2) When we create a subscription, another slot is created during the subscription

Re: Synchronizing slots from primary to standby

Hi, On 10/25/23 5:00 AM, shveta malik wrote: On Tue, Oct 24, 2023 at 11:54 AM Drouvot, Bertrand wrote: Hi, On 10/23/23 2:56 PM, shveta malik wrote: On Mon, Oct 23, 2023 at 5:52 PM Drouvot, Bertrand wrote: We are waiting for DEFAULT_NAPTIME_PER_CYCLE (3 minutes) before checking

Re: Synchronizing slots from primary to standby

Hi, On 10/24/23 7:44 AM, Ajin Cherian wrote: On Mon, Oct 23, 2023 at 11:22 PM Drouvot, Bertrand wrote: @@ -602,6 +602,9 @@ CreateDecodingContext(XLogRecPtr start_lsn, SnapBuildSetTwoPhaseAt(ctx->snapshot_builder, start_lsn); } + /* set failover in the slot, as reques

Re: Synchronizing slots from primary to standby

Hi, On 10/23/23 2:56 PM, shveta malik wrote: On Mon, Oct 23, 2023 at 5:52 PM Drouvot, Bertrand wrote: We are waiting for DEFAULT_NAPTIME_PER_CYCLE (3 minutes) before checking if there is new synced slot(s) to be created on the standby. Do we want to keep this behavior for V1? I think

Re: Synchronizing slots from primary to standby

Hi, On 10/20/23 5:27 AM, shveta malik wrote: On Wed, Oct 18, 2023 at 4:24 PM Amit Kapila wrote: PFA v25 patch set. The changes are: 1) 'enable_failover' is changed to 'failover' 2) Alter subscription changes to support 'failover' 3) Fixes a bug in patch001 wherein any change in

Re: Synchronizing slots from primary to standby

Hi, On 10/18/23 6:43 AM, shveta malik wrote: On Tue, Oct 17, 2023 at 9:06 PM Drouvot, Bertrand +static void +ProcessRepliesAndTimeOut(void) +{ + CHECK_FOR_INTERRUPTS(); + + /* Process any requests or signals received recently */ + if (ConfigReloadPending

Re: Synchronizing slots from primary to standby

Hi, On 10/13/23 10:35 AM, shveta malik wrote: On Thu, Oct 12, 2023 at 9:18 AM shveta malik wrote: PFA v24 patch set which has below changes: 1) 'enable_failover' displayed in pg_replication_slots. 2) Support for 'enable_failover' in pg_create_logical_replication_slot(). It is an optional

Re: Add a new BGWORKER_BYPASS_ROLELOGINCHECK flag

Hi, On 10/12/23 2:26 AM, Michael Paquier wrote: On Wed, Oct 11, 2023 at 08:48:04AM +0200, Drouvot, Bertrand wrote: Yeah, agree. Changed in v12 attached. I have tweaked a few comments, and applied that. Thanks. Oh and you also closed the CF entry, thanks! Regards, -- Bertrand Drouvot

Re: Add a new BGWORKER_BYPASS_ROLELOGINCHECK flag

Hi, On 10/11/23 5:40 AM, Michael Paquier wrote: On Wed, Oct 11, 2023 at 08:26:42AM +0900, Michael Paquier wrote: /* flags for InitPostgres() */ #define INIT_PG_LOAD_SESSION_LIBS 0x0001 #define INIT_PG_OVERRIDE_ALLOW_CONNS 0x0002 +#define INIT_PG_BYPASS_ROLE_LOGIN 0x0004 In

Re: Add a new BGWORKER_BYPASS_ROLELOGINCHECK flag

On 10/10/23 9:21 AM, Michael Paquier wrote: On Tue, Oct 10, 2023 at 09:18:04AM +0200, Drouvot, Bertrand wrote: Please forget about it ;-) That's called an ENOCOFFEE :D Exactly, good one! ;-) Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web

Re: Add a new BGWORKER_BYPASS_ROLELOGINCHECK flag

On 10/10/23 9:12 AM, Drouvot, Bertrand wrote: Hi, Any reason why INIT_PG_BYPASS_ROLE_LOGIN is not 0x0003? Please forget about it ;-) Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Add a new BGWORKER_BYPASS_ROLELOGINCHECK flag

Hi, On 10/10/23 7:58 AM, Michael Paquier wrote: On Tue, Oct 10, 2023 at 06:57:05AM +0200, Drouvot, Bertrand wrote: Please find attached v9 (v8 rebase due to f483b2090). I was looking at v8 just before you sent this v9, and still got annoyed by the extra boolean argument added to InitPostgres

Re: Add a new BGWORKER_BYPASS_ROLELOGINCHECK flag

Hi, On 10/6/23 8:48 AM, Drouvot, Bertrand wrote: Hi, On 10/5/23 6:23 PM, Bharath Rupireddy wrote: On Thu, Oct 5, 2023 at 9:32 PM Drouvot, Bertrand wrote: +  CREATE ROLE nologrole with nologin; +  GRANT CREATE ON DATABASE mydb TO nologrole; A few nit-picks: 1. s/with/WITH 2. s/nologin

Re: Synchronizing slots from primary to standby

Hi, On 10/6/23 6:48 PM, Amit Kapila wrote: On Wed, Oct 4, 2023 at 5:34 PM Drouvot, Bertrand wrote: On 10/4/23 1:50 PM, shveta malik wrote: On Wed, Oct 4, 2023 at 5:00 PM Amit Kapila wrote: On Wed, Oct 4, 2023 at 11:55 AM Drouvot, Bertrand wrote: On 10/4/23 6:26 AM, shveta malik wrote

Re: Add a new BGWORKER_BYPASS_ROLELOGINCHECK flag

Hi, On 10/5/23 6:23 PM, Bharath Rupireddy wrote: On Thu, Oct 5, 2023 at 9:32 PM Drouvot, Bertrand wrote: + CREATE ROLE nologrole with nologin; + GRANT CREATE ON DATABASE mydb TO nologrole; A few nit-picks: 1. s/with/WITH 2. s/nologin/NOLOGIN done in v8 attached. 3. + is specified

Re: Add a new BGWORKER_BYPASS_ROLELOGINCHECK flag

Hi, On 10/5/23 2:21 PM, Bharath Rupireddy wrote: On Thu, Oct 5, 2023 at 12:22 PM Drouvot, Bertrand wrote: A comment on v6-0002: 1. + CREATE ROLE nologrole with nologin; + ALTER ROLE nologrole with superuser; +]); We don't need superuser privileges here, do we? Or do we need

Re: Add a new BGWORKER_BYPASS_ROLELOGINCHECK flag

Hi, On 10/5/23 7:10 AM, Michael Paquier wrote: On Wed, Oct 04, 2023 at 12:54:24PM +0200, Drouvot, Bertrand wrote: Except the Nit that I mentioned in 0001, that looks all good to me (with the new wait in 001_worker_spi.pl). Thanks, I've applied the refactoring, including in it the stuff

Re: Synchronizing slots from primary to standby

Hi, On 10/4/23 1:50 PM, shveta malik wrote: On Wed, Oct 4, 2023 at 5:00 PM Amit Kapila wrote: On Wed, Oct 4, 2023 at 11:55 AM Drouvot, Bertrand wrote: On 10/4/23 6:26 AM, shveta malik wrote: On Wed, Oct 4, 2023 at 5:36 AM Amit Kapila wrote: How about an alternate scheme where we

Re: Add a new BGWORKER_BYPASS_ROLELOGINCHECK flag

Hi, On 10/4/23 8:20 AM, Michael Paquier wrote: On Mon, Oct 02, 2023 at 10:53:22AM +0200, Drouvot, Bertrand wrote: On 10/2/23 10:17 AM, Michael Paquier wrote: On Mon, Oct 02, 2023 at 10:01:04AM +0200, Drouvot, Bertrand wrote: I think that would make sense to have more flexibility

Re: Synchronizing slots from primary to standby

Hi, On 10/4/23 7:00 AM, shveta malik wrote: On Wed, Oct 4, 2023 at 9:56 AM shveta malik wrote: The most simplistic approach would be: 1) maintain standby_slot_names GUC on primary 2) maintain synchronize_slot_names GUC on physical standby alone. On primary, let all logical-walsenders wait

Re: Synchronizing slots from primary to standby

Hi, On 10/4/23 6:26 AM, shveta malik wrote: On Wed, Oct 4, 2023 at 5:36 AM Amit Kapila wrote: On Tue, Oct 3, 2023 at 9:27 PM shveta malik wrote: On Tue, Oct 3, 2023 at 7:56 PM Drouvot, Bertrand wrote: Hi, On 10/3/23 12:54 PM, Amit Kapila wrote: On Mon, Oct 2, 2023 at 11:39 AM Drouvot

Re: Synchronizing slots from primary to standby

Hi, On 10/3/23 12:54 PM, Amit Kapila wrote: On Mon, Oct 2, 2023 at 11:39 AM Drouvot, Bertrand wrote: On 9/29/23 1:33 PM, Amit Kapila wrote: On Thu, Sep 28, 2023 at 6:31 PM Drouvot, Bertrand wrote: - probably open corner cases like: what if a standby is down? would that mean

Re: Add a new BGWORKER_BYPASS_ROLELOGINCHECK flag

Hi, On 10/3/23 11:21 AM, Bharath Rupireddy wrote: On Mon, Oct 2, 2023 at 4:58 PM Drouvot, Bertrand wrote: On 9/29/23 8:19 AM, Michael Paquier wrote: On Thu, Sep 28, 2023 at 02:37:02PM +0200, Drouvot, Bertrand wrote: This patch allows the role provided

Replace (stat())[7] in TAP tests with -s

Hi hackers, Please find attached a tiny patch to $SUBJECT. It: - provides more consistency to the way we get files size in TAP tests - seems more elegant that relying on a hardcoded result position Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon

Re: Add a new BGWORKER_BYPASS_ROLELOGINCHECK flag

Hi, On 10/2/23 10:17 AM, Michael Paquier wrote: On Mon, Oct 02, 2023 at 10:01:04AM +0200, Drouvot, Bertrand wrote: I think that would make sense to have more flexibility in the worker_spi module. I think that could be done in a dedicated patch though. I think it makes more sense to have

Re: Add a new BGWORKER_BYPASS_ROLELOGINCHECK flag

Hi, On 9/29/23 8:19 AM, Michael Paquier wrote: On Thu, Sep 28, 2023 at 02:37:02PM +0200, Drouvot, Bertrand wrote: This patch allows the role provided in BackgroundWorkerInitializeConnection() and BackgroundWorkerInitializeConnectionByOid() to lack login authorization. Interesting. Yes

Re: Synchronizing slots from primary to standby

Hi, On 9/29/23 1:33 PM, Amit Kapila wrote: On Thu, Sep 28, 2023 at 6:31 PM Drouvot, Bertrand wrote: I think that standby_slot_names could be used to do some filtering (means for which standby(s) we don't want the logical replication on the primary to go ahead and for which standby(s) one

Re: Synchronizing slots from primary to standby

Hi, On 9/25/23 6:10 AM, shveta malik wrote: On Fri, Sep 22, 2023 at 3:48 PM Amit Kapila wrote: On Thu, Sep 21, 2023 at 9:16 AM shveta malik wrote: On Tue, Sep 19, 2023 at 10:29 AM shveta malik wrote: Currently in patch001, synchronize_slot_names is a GUC on both primary and physical

Add a new BGWORKER_BYPASS_ROLELOGINCHECK flag

Hi hackers, Please find attached a patch proposal to $SUBJECT. This patch allows the role provided in BackgroundWorkerInitializeConnection() and BackgroundWorkerInitializeConnectionByOid() to lack login authorization. In InitPostgres(), in case of a background worker, authentication is not

Re: Synchronizing slots from primary to standby

Hi, On 9/19/23 6:50 AM, shveta malik wrote: On Wed, Sep 13, 2023 at 5:19 PM Amit Kapila wrote: On Wed, Sep 13, 2023 at 4:54 PM shveta malik wrote: PFA v17. It has below changes: @@ -2498,6 +2500,13 @@ ReorderBufferProcessTXN(ReorderBuffer *rb, ReorderBufferTXN *txn, } else {

Re: Synchronizing slots from primary to standby

Hi, On 9/25/23 10:44 AM, Drouvot, Bertrand wrote: Hi, On 9/23/23 3:38 AM, Amit Kapila wrote: On Fri, Sep 22, 2023 at 6:01 PM Drouvot, Bertrand wrote: There is a difference here that we also need to prevent removal of rows required by sync_slots. That could be achieved by physical slot

Re: Synchronizing slots from primary to standby

Hi, On 9/23/23 3:38 AM, Amit Kapila wrote: On Fri, Sep 22, 2023 at 6:01 PM Drouvot, Bertrand wrote: Thanks for all the work that has been done on this feature, and sorry to have been quiet on it for so long. On 9/18/23 12:22 PM, shveta malik wrote: On Wed, Sep 13, 2023 at 4:48 PM Hayato

Re: Synchronizing slots from primary to standby

Hi, Thanks for all the work that has been done on this feature, and sorry to have been quiet on it for so long. On 9/18/23 12:22 PM, shveta malik wrote: On Wed, Sep 13, 2023 at 4:48 PM Hayato Kuroda (Fujitsu) wrote: Right, but I wanted to know why it is needed. One motivation seemed to know

Re: Simple CustomScan example

Hi, On 9/6/23 9:32 PM, Chris Cleveland wrote: I'm trying to write a custom scan. It's pretty confusing. I've read the documentation at https://www.postgresql.org/docs/current/custom-scan.html , and I've scanned the code in Citus

  1   2   3   4   5   >