Re: [Patch] add new parameter to pg_replication_origin_session_setup

2025-09-03 Thread Doruk Yilmaz
Dear Hayato, > So, your python process establishes two connections, for publisher > (replication connection) > and subscriber (normal connection). It receives changes from the publisher, > constructs SQL statements from the received results, and sends to subscriber's > backend, is it right? Actu

RE: [Patch] add new parameter to pg_replication_origin_session_setup

2025-08-25 Thread Hayato Kuroda (Fujitsu)
Dear Doruk, > In our system the workers aren't background workers and we don't ship > a server-side extension; they're plain external processes (Python in > our case) talking over standard database connections. In many > deployments -especially managed Postgres- we can't load custom C code > even

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2025-08-22 Thread Doruk Yilmaz
Dear Hayato, > Can you explain more why we must extend the SQL interface? In our system the workers aren't background workers and we don't ship a server-side extension; they're plain external processes (Python in our case) talking over standard database connections. In many deployments -especiall

RE: [Patch] add new parameter to pg_replication_origin_session_setup

2025-08-21 Thread Hayato Kuroda (Fujitsu)
Dear Doruk, > That sounds reasonable. I’ve updated the patch and added more > information to the documentation covering the topics you mentioned. > I also added a Caution block so potential users won’t miss it. I hope > this patch meets your expectations. Can you explain more why we must extend t

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2025-08-18 Thread Doruk Yilmaz
> Your use looks good to me. So, maybe we can update the docs with the > dangers if the users of API doesn't follow commit order then it may > lead to data inconsistency should be sufficient. Additionally, we may > want to give an example as to how to use this API for parallel apply. That sounds r

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2025-08-12 Thread Amit Kapila
On Mon, Aug 11, 2025 at 10:41 PM Doruk Yilmaz wrote: > > On Mon, Aug 11, 2025 at 9:44 AM Amit Kapila wrote: > > How do you advance the origin? Did you use > > > pg_replication_origin_advance()? If so, you should be aware that it > > can be used for initial setup; see comment in that API code...

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2025-08-11 Thread Doruk Yilmaz
On Mon, Aug 11, 2025 at 9:44 AM Amit Kapila wrote: > How do you advance the origin? Did you use > pg_replication_origin_advance()? > If so, you should be aware that it > can be used for initial setup; see comment in that API code... No, we don't use pg_replication_origin_advance(). We use pg_rep

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2025-08-10 Thread Amit Kapila
On Wed, Jul 30, 2025 at 12:00 AM Doruk Yilmaz wrote: > > On Mon, Jul 29, 2025 at 8:13 AM Amit Kapila wrote: > > That is true but I still feel there has to be some mechanism where we > > can catch and give an ERROR to the user, if it doesn't follow the > > same. For example, pg_replication_origin_

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2025-07-29 Thread Doruk Yilmaz
On Mon, Jul 29, 2025 at 8:13 AM Amit Kapila wrote: > That is true but I still feel there has to be some mechanism where we > can catch and give an ERROR to the user, if it doesn't follow the > same. For example, pg_replication_origin_advance() always allows going > backwards in terms of LSN which

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2025-07-28 Thread Amit Kapila
On Tue, Jul 29, 2025 at 2:43 AM Doruk Yilmaz wrote: > > On Mon, Mar 3, 2025 at 6:39 AM Amit Kapila wrote: > > > > To use replication_origin by multiple processes, one must maintain the > > commit order as we do internally by allowing the leader process to > > wait for the parallel worker to finis

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2025-07-28 Thread Doruk Yilmaz
On Mon, Mar 3, 2025 at 6:39 AM Amit Kapila wrote: > > To use replication_origin by multiple processes, one must maintain the > commit order as we do internally by allowing the leader process to > wait for the parallel worker to finish the commit. See comments atop > replorigin_session_setup(). Now

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2025-03-02 Thread Amit Kapila
On Thu, Jan 9, 2025 at 3:26 AM Euler Taveira wrote: > > On Thu, Aug 15, 2024, at 5:53 PM, Doruk Yilmaz wrote: > > Hello again, > > On Tue, Aug 13, 2024 at 12:48 AM Euler Taveira wrote: > > I'm curious about your use case. Is it just because the internal function > > has a > > different signature

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2025-03-01 Thread Doruk Yilmaz
I noticed that the patch needs rebasing, so here is the rebased version. Hopefully it makes to the commitfest. Doruk Yılmaz From 74a74fd02bce786093c19a23bef9444d0b8ef41d Mon Sep 17 00:00:00 2001 From: Doruk Date: Thu, 15 Aug 2024 23:34:26 +0300 Subject: [PATCH v4] pg_replication_origin_session_se

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2025-01-08 Thread Euler Taveira
On Thu, Aug 15, 2024, at 5:53 PM, Doruk Yilmaz wrote: > Hello again, > > On Tue, Aug 13, 2024 at 12:48 AM Euler Taveira wrote: > > I'm curious about your use case. Is it just because the internal function > > has a > > different signature or your tool is capable of apply logical replication > >

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2024-08-15 Thread Doruk Yilmaz
is covered by CI [2][3]. Sadly I still can't log in to the Commitfest due to the cool-off period. I will create an entry as soon as this period ends. Thanks for all the feedback, Doruk Yılmaz From b9c54f3d217f67c24ce74ffa7c1f2812d784333e Mon Sep 17 00:00:00 2001 From: Doruk Date: Thu, 15 Aug

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2024-08-12 Thread Euler Taveira
On Mon, Aug 12, 2024, at 3:43 PM, Doruk Yilmaz wrote: > Hello all, Hi! > While working on our internal tools that utilise replication, we > realised that a new parameter was added to the internal C function > corresponding to pg_replication_origin_session_setup. > However this parameter wasn't in

[Patch] add new parameter to pg_replication_origin_session_setup

2024-08-12 Thread Doruk Yilmaz
Hello all, While working on our internal tools that utilise replication, we realised that a new parameter was added to the internal C function corresponding to pg_replication_origin_session_setup. However this parameter wasn't included in the user-facing API [1]. In 'src/backend/replication/logica