Re: Add progressive backoff to XactLockTableWait functions

Hi, After studying proarray and lmgr more closely, I have found several critical issues in the two patches and underestimated their complexity and subtlety. Sorry for posting immature patches that may have created noise. I now realize that making lock acquisition— (void) LockAcquire(&tag, ShareLo

Re: Add progressive backoff to XactLockTableWait functions

Hi all, I spent some extra time walking the code to see where XactLockTableWait() actually fires. A condensed recap: 1) Current call-paths A. Logical walsender (XLogSendLogical → … → SnapBuildWaitSnapshot) in cascading standby B. SQL slot functions (pg_logical_slot_get_changes[_peek]) create_lo

Re: Add progressive backoff to XactLockTableWait functions

Hi, I spent some time digging deeper into the KnownAssignedXids logic in procarray.c and made a few refinements. Below are some observations and questions I’d like to discuss. 1) Present usage (AFAICS) It appears that logical decoding is the only place that waits for an XID on a standby. The si

Re: Add progressive backoff to XactLockTableWait functions

Hi Andres, hi all, Thanks a lot for the advice. > My position is basically: > > 1) We should *never* add new long-duration polling loops to postgres. We've >regretted it every time. It just ends up masking bugs and biting us in >scenarios we didn't predict (increased wakeups increasing po

Re: Add progressive backoff to XactLockTableWait functions

On 2025-07-05 01:14:45 +0900, Fujii Masao wrote: > > > On 2025/07/04 17:57, Xuneng Zhou wrote: > > Hi, > > > > On Thu, Jul 3, 2025 at 9:30 AM Xuneng Zhou > > wrote: > > > > Hi, > > > > > > >>> On 2025-07-02 22:55:16 +0900, Fujii Masao wrote: > >

Re: Add progressive backoff to XactLockTableWait functions

On 2025/07/04 17:57, Xuneng Zhou wrote: Hi, On Thu, Jul 3, 2025 at 9:30 AM Xuneng Zhou mailto:xunengz...@gmail.com>> wrote: Hi, >>> On 2025-07-02 22:55:16 +0900, Fujii Masao wrote: On 2025/06/24 1:32, Xuneng Zhou wrote: > 3. The proposed solution

Re: Add progressive backoff to XactLockTableWait functions

Hi, On Thu, Jul 3, 2025 at 9:30 AM Xuneng Zhou wrote: > Hi, > >> >> >>> On 2025-07-02 22:55:16 +0900, Fujii Masao wrote: >> On 2025/06/24 1:32, Xuneng Zhou wrote: >> > 3. The proposed solution >> > >> > If the above analysis is sound, one potential fix would be to add >> > s

Re: Add progressive backoff to XactLockTableWait functions

> > Thanks for the feedbacks! To be honest, I'm fine with v3, since it only increases the sleep time > after 5000 loop iterations, which has negligible performance impact. > But if these functions aren't intended to be used during recovery and > the loop shouldn't reach that many iterations, I'm a

Re: Add progressive backoff to XactLockTableWait functions

Hi, On Wed, Jul 2, 2025 at 10:56 PM Fujii Masao wrote: > > > On 2025/07/02 23:19, Andres Freund wrote: > > Hi, > > > > On July 2, 2025 10:15:09 AM EDT, Fujii Masao < > masao.fu...@oss.nttdata.com> wrote: > >> > >> > >> On 2025/07/02 23:04, Andres Freund wrote: > >>> Hi, > >>> > >>> On 2025-07-02

Re: Add progressive backoff to XactLockTableWait functions

On 2025/07/02 23:19, Andres Freund wrote: Hi, On July 2, 2025 10:15:09 AM EDT, Fujii Masao wrote: On 2025/07/02 23:04, Andres Freund wrote: Hi, On 2025-07-02 22:55:16 +0900, Fujii Masao wrote: On 2025/06/24 1:32, Xuneng Zhou wrote: 3. The proposed solution If the above analysis is s

Re: Add progressive backoff to XactLockTableWait functions

Hi, On July 2, 2025 10:15:09 AM EDT, Fujii Masao wrote: > > >On 2025/07/02 23:04, Andres Freund wrote: >> Hi, >> >> On 2025-07-02 22:55:16 +0900, Fujii Masao wrote: >>> On 2025/06/24 1:32, Xuneng Zhou wrote: 3. The proposed solution If the above analysis is sound, one potential

Re: Add progressive backoff to XactLockTableWait functions

On 2025/07/02 23:04, Andres Freund wrote: Hi, On 2025-07-02 22:55:16 +0900, Fujii Masao wrote: On 2025/06/24 1:32, Xuneng Zhou wrote: 3. The proposed solution If the above analysis is sound, one potential fix would be to add separate branching for standby in XactLockTableWait. However, thi

Re: Add progressive backoff to XactLockTableWait functions

Hi, On 2025-07-02 22:55:16 +0900, Fujii Masao wrote: > On 2025/06/24 1:32, Xuneng Zhou wrote: > > 3. The proposed solution > > > > If the above analysis is sound, one potential fix would be to add > > separate branching for standby in XactLockTableWait. However, this seems > > inconsistent with t

Re: Add progressive backoff to XactLockTableWait functions

On 2025/06/24 1:32, Xuneng Zhou wrote: Hi, Here's patch version 4. 1. The problem I conducted further investigation on this issue. The 1ms sleep in XactLockTableWait that falls back to polling was not problematic in certain scenarios prior to v16. It became an potential issue after the

Re: Add progressive backoff to XactLockTableWait functions

Hi, Here's patch version 4. 1. The problem I conducted further investigation on this issue. The 1ms sleep in XactLockTableWait that falls back to polling was not problematic in certain scenarios prior to v16. It became an potential issue after the "Allow logical decoding on standbys" feature w

Re: Add progressive backoff to XactLockTableWait functions

Hi Andres, Thanks for looking into this! On Tue, Jun 17, 2025 at 10:17 PM Andres Freund wrote: > Hi, > > On 2025-06-08 22:33:39 +0800, Xuneng Zhou wrote: > > This patch implements progressive backoff in XactLockTableWait() and > > ConditionalXactLockTableWait(). > > > > As Kevin reported in thi

Re: Add progressive backoff to XactLockTableWait functions

Hi, On 2025-06-08 22:33:39 +0800, Xuneng Zhou wrote: > This patch implements progressive backoff in XactLockTableWait() and > ConditionalXactLockTableWait(). > > As Kevin reported in this thread [1], XactLockTableWait() can enter a > tight polling loop during logical replication slot creation on

Re: Add progressive backoff to XactLockTableWait functions

Hi, On Sun, Jun 15, 2025 at 4:01 PM Xuneng Zhou wrote: > Hi, > > Attached is v2 of the patch to add threshold-based sleep to > XactLockTableWait functions. > > Changes from v1: > - Simplified approach based on Fujii's feedback [1]: instead of > exponential backoff, > we now sleep 1ms for the f

Re: Add progressive backoff to XactLockTableWait functions

Hi, Attached is v2 of the patch to add threshold-based sleep to XactLockTableWait functions. Changes from v1: - Simplified approach based on Fujii's feedback [1]: instead of exponential backoff, we now sleep 1ms for the first 5 seconds, then switch directly to 1s sleeps - Reduced the threshold

Re: Add progressive backoff to XactLockTableWait functions

Hi, Thanks for the feedback! On Thu, Jun 12, 2025 at 10:02 PM Fujii Masao wrote: > > > When I first suggested this idea, I used 10s as an example for > the maximum sleep time. But thinking more about it now, 10s might > be too long. Even if the target transaction has already finished, > XactLoc

Re: Add progressive backoff to XactLockTableWait functions

On 2025/06/08 23:33, Xuneng Zhou wrote: Hi hackers, This patch implements progressive backoff in XactLockTableWait() and ConditionalXactLockTableWait(). As Kevin reported in this thread [1], XactLockTableWait() can enter a tight polling loop during logical replication slot creation on standb

Add progressive backoff to XactLockTableWait functions

Hi hackers, This patch implements progressive backoff in XactLockTableWait() and ConditionalXactLockTableWait(). As Kevin reported in this thread [1], XactLockTableWait() can enter a tight polling loop during logical replication slot creation on standby servers, sleeping for fixed 1ms intervals t