Re: Parallel Apply

2025-08-29 Thread Nisha Moond
Hi, I ran tests to compare the performance of logical synchronous replication with parallel-apply against physical synchronous replication. Highlights === On pgHead:(current behavior) - With synchronous physical replication set to remote_apply, the Primary’s TPS drops by ~60% (≈2.5x

Re: Parallel Apply

2025-08-18 Thread Amit Kapila
On Mon, Aug 18, 2025 at 8:20 PM Konstantin Knizhnik wrote: > > On 18/08/2025 9:56 AM, Nisha Moond wrote: > > On Wed, Aug 13, 2025 at 4:17 PM Zhijie Hou (Fujitsu) > > wrote: > >> Here is the initial POC patch for this idea. > >> > > Thank you Hou-san for the patch. > > > > I did some performance b

Re: Parallel Apply

2025-08-18 Thread Konstantin Knizhnik
On 18/08/2025 9:56 AM, Nisha Moond wrote: On Wed, Aug 13, 2025 at 4:17 PM Zhijie Hou (Fujitsu) wrote: Here is the initial POC patch for this idea. Thank you Hou-san for the patch. I did some performance benchmarking for the patch and overall, the results show substantial performance improv

Re: Parallel Apply

2025-08-17 Thread Nisha Moond
On Wed, Aug 13, 2025 at 4:17 PM Zhijie Hou (Fujitsu) wrote: > > Here is the initial POC patch for this idea. > Thank you Hou-san for the patch. I did some performance benchmarking for the patch and overall, the results show substantial performance improvements. Please find the details as follows

Re: Parallel Apply

2025-08-14 Thread Amit Kapila
On Wed, Aug 13, 2025 at 8:57 PM Bruce Momjian wrote: > > On Wed, Aug 13, 2025 at 09:50:27AM +0530, Amit Kapila wrote: > > On Tue, Aug 12, 2025 at 10:40 PM Bruce Momjian wrote: > > > > Currently, PostgreSQL supports parallel apply only for large streaming > > > > transactions (streaming=parallel).

Re: Parallel Apply

2025-08-13 Thread Bruce Momjian
On Wed, Aug 13, 2025 at 09:50:27AM +0530, Amit Kapila wrote: > On Tue, Aug 12, 2025 at 10:40 PM Bruce Momjian wrote: > > > Currently, PostgreSQL supports parallel apply only for large streaming > > > transactions (streaming=parallel). This proposal aims to extend > > > parallelism to non-streaming

RE: Parallel Apply

2025-08-13 Thread Zhijie Hou (Fujitsu)
On Monday, August 11, 2025 12:46 PM Amit Kapila wrote: > Background and Motivation > - > In high-throughput systems, where hundreds of sessions generate data > on the publisher, the subscriber's apply process often becomes a > bottleneck due to the single apply

Re: Parallel Apply

2025-08-12 Thread Amit Kapila
On Tue, Aug 12, 2025 at 9:22 PM Константин Книжник wrote: > > Hi, > This is something similar to what I have in mind when starting my experiments > with LR apply speed improvements. I think that maintaining a full > (RelationId, ReplicaIdentity) hash may be too expensive - there can be > hundr

Re: Parallel Apply

2025-08-12 Thread Amit Kapila
On Tue, Aug 12, 2025 at 10:40 PM Bruce Momjian wrote: > > On Mon, Aug 11, 2025 at 10:15:41AM +0530, Amit Kapila wrote: > > Hi, > > > > Background and Motivation > > - > > In high-throughput systems, where hundreds of sessions generate data > > on the publisher,

Re: Parallel Apply

2025-08-12 Thread Bruce Momjian
On Mon, Aug 11, 2025 at 10:15:41AM +0530, Amit Kapila wrote: > Hi, > > Background and Motivation > - > In high-throughput systems, where hundreds of sessions generate data > on the publisher, the subscriber's apply process often becomes a > bottleneck due to the

Re: Parallel Apply

2025-08-12 Thread Константин Книжник
On 11.08.2025 7:45 AM, Amit Kapila wrote: Hi, Background and Motivation - In high-throughput systems, where hundreds of sessions generate data on the publisher, the subscriber's apply process often becomes a bottleneck due to the single apply worker model. Whil

Re: Parallel Apply

2025-08-12 Thread Amit Kapila
On Mon, Aug 11, 2025 at 3:00 PM Kirill Reshke wrote: > > On Mon, 11 Aug 2025 at 13:45, Amit Kapila wrote: > > > > I am not sure if that is directly applicable because this work > > proposes to track dependencies based on logical WAL contents. However, > > if you can point me to README on the over

Re: Parallel Apply

2025-08-11 Thread Amit Kapila
On Tue, Aug 12, 2025 at 12:04 PM Andrei Lepikhov wrote: > > On 11/8/2025 06:45, Amit Kapila wrote: > > The core idea is that the leader apply worker ensures the following: > > a. Identifies dependencies between transactions. b. Coordinates > > parallel workers to apply independent transactions con

Re: Parallel Apply

2025-08-11 Thread Andrei Lepikhov
On 11/8/2025 06:45, Amit Kapila wrote: The core idea is that the leader apply worker ensures the following: a. Identifies dependencies between transactions. b. Coordinates parallel workers to apply independent transactions concurrently. c. Ensures correct ordering for dependent transactions. Depe

Re: Parallel Apply

2025-08-11 Thread Kirill Reshke
On Mon, 11 Aug 2025 at 13:45, Amit Kapila wrote: > > > I am not sure if that is directly applicable because this work > proposes to track dependencies based on logical WAL contents. However, > if you can point me to README on the overall design of the work you > are pointing to then I can check i

Re: Parallel Apply

2025-08-11 Thread Amit Kapila
On Mon, Aug 11, 2025 at 1:39 PM Kirill Reshke wrote: > > > > Design Overview > > > > To safely parallelize non-streaming transactions, we must ensure that > > transaction dependencies are respected to avoid failures and > > deadlocks. Consider the following scenarios to un

Re: Parallel Apply

2025-08-11 Thread Kirill Reshke
Hi! On Mon, 11 Aug 2025 at 09:46, Amit Kapila wrote: > > Hi, > > Background and Motivation > - > In high-throughput systems, where hundreds of sessions generate data > on the publisher, the subscriber's apply process often becomes a > bottleneck due to the sing