Re: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-04-15 Thread Давыдов Виталий
Dear All, On Wednesday, April 10, 2024 17:16 MSK, Давыдов Виталий wrote:  Hi Amit, Ajin, All Thank you for the patch and the responses. I apologize for my delayed answer due to some curcumstances. On Wednesday, April 10, 2024 14:18 MSK, Amit Kapila wrote: Vitaly, does the minimal solution

Re: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-04-10 Thread Давыдов Виталий
Hi Amit, Ajin, All Thank you for the patch and the responses. I apologize for my delayed answer due to some curcumstances. On Wednesday, April 10, 2024 14:18 MSK, Amit Kapila wrote: Vitaly, does the minimal solution provided by the proposed patch (Allow to alter two_phase option of a

Re: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-03-05 Thread Давыдов Виталий
Hi Heikki, Thank you for the reply. On Tuesday, March 05, 2024 12:05 MSK, Heikki Linnakangas wrote:  In a nutshell, this changes PREPARE TRANSACTION so that if synchronous_commit is 'off', the PREPARE TRANSACTION is not fsync'd to disk. So if you crash after the PREPARE TRANSACTION has

Re: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-02-29 Thread Давыдов Виталий
Dear All, Consider, please, my patch for async commit for twophase transactions. It can be applicable when catchup performance is not enought with publication parameter twophase = on. The key changes are: * Use XLogSetAsyncXactLSN instead of XLogFlush as it is for usual transactions. * In

Re: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-02-27 Thread Давыдов Виталий
I store 2pc state in the local memory in my patch. It would be good if you could link those threads.Sure, I will find and add some links to the discussions from past. Thank you! With best regards, Vitaly  On Tue, Feb 27, 2024 at 4:49 PM Давыдов Виталий wrote: > > Thank you for your in

Re: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-02-27 Thread Давыдов Виталий
Hi Amit, Thank you for your interest in the discussion! On Monday, February 26, 2024 16:24 MSK, Amit Kapila wrote:   I think the reason is probably that when the WAL record for prepared is already flushed then what will be the idea of async commit here?I think, the idea of async commit

Re: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-02-23 Thread Давыдов Виталий
Hi Amit, Amit Kapila wrote: I don't see we do anything specific for 2PC transactions to make them behave differently than regular transactions with respect to synchronous_commit setting. What makes you think so? Can you pin point the code you are referring to?Yes, sure. The function

Re: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-02-23 Thread Давыдов Виталий
Hi Ajin, Thank you for your feedback. Could you please try to increase the number of clients (-c pgbench option) up to 20 or more? It seems, I forgot to specify it. With best regards, Vitaly Davydov On Fri, Feb 23, 2024 at 12:29 AM Давыдов Виталий wrote: Dear All, I'd like to present

Slow catchup of 2PC (twophase) transactions on replica in LR

2024-02-22 Thread Давыдов Виталий
Dear All, I'd like to present and talk about a problem when 2PC transactions are applied quite slowly on a replica during logical replication. There is a master and a replica with established logical replication from the master to the replica with twophase = true. With some load level on the

Re: How to accurately determine when a relation should use local buffers?

2023-11-27 Thread Давыдов Виталий
Hi Aleksander, Well even assuming this patch will make it to the upstream some day, which I seriously doubt, it will take somewhere between 2 and 5 years. Personally I would recommend reconsidering this design. I understand what you are saying. I have no plans to create a patch for this issue. I

Re: How to accurately determine when a relation should use local buffers?

2023-11-23 Thread Давыдов Виталий
Hi Aleksander, I sort of suspect that you are working on a very specific extension and/or feature for PG fork. Any chance you could give us more details about the case?I'm trying to adapt a multimaster solution to some changes in pg16. We replicate temp table DDL due to some reasons.

Re: How to accurately determine when a relation should use local buffers?

2023-11-22 Thread Давыдов Виталий
Hi Aleksander, Thank you for your answers. It seems, local buffers are used for temporary relations unconditionally. In this case, we may check either relpersistence or backend id, or both of them. I didn't do a deep investigation of the code in this particular aspect but that could be a fair

How to accurately determine when a relation should use local buffers?

2023-11-20 Thread Давыдов Виталий
Dear Hackers, I would like to clarify, what the correct way is to determine that a given relation is using local buffers. Local buffers, as far as I know, are used for temporary tables in backends. There are two functions/macros (bufmgr.c): SmgrIsTemp, RelationUsesLocalBuffers. The first