From 62b487a0a4ff6425b30d5ba76d3cb85fd04a047b Mon Sep 17 00:00:00 2001 From: Bharath Rupireddy Date: Sun, 13 Sep 2026 23:43:15 +0000 Subject: [PATCH v9 2/5] Use WALReadFromBuffers() for logical replication walsenders. Commit 91f2cae7a4 introduced WALReadFromBuffers() but used it only for physical replication walsenders. This commit uses it for logical replication walsenders as well, so that logical decoding can also read WAL from the WAL buffers instead of always going to a file. When a logical replication consumer keeps up with WAL generation, the requested WAL is often still in the WAL buffers, so it can be read from there instead of from a file. The gain is largest with WAL direct I/O, where a file read is a physical disk read. Without direct I/O it still saves a syscall and does not regress. The benefit depends on the workload and how closely the consumer follows the insertion point. A read that finds all of the requested WAL in the WAL buffers skips the file read path, which is also where the reader closes and reopens its segment file as it crosses a segment boundary. So such a read never notices a segment change. The segment file stays open on the old segment while the reader's segment number advances to the new one. For example, when the first page of segment 2 comes from the WAL buffers, the reader's segment number becomes 2 but its file is still open on segment 1. A later read of segment 2 that falls back to the file reuses that stale open file and returns segment 1's data, seen during decoding as an "unexpected pageaddr" error. Fix this by closing the open segment when a read that comes entirely from the WAL buffers is not in the open segment, so that the next file read reopens the correct one. Reads that stay within the open segment leave it alone, as the reader's segment number does not change. A test that waits for a logical walsender to report WAL read activity now waits for the sum of reads and hits in pg_stat_io, because a read from the WAL buffers is reported as a hit and not as a read. Author: Bharath Rupireddy Reviewed-by: Jingtang Zhang Reviewed-by: Nitin Jadhav Reviewed-by: Michael Paquier Reviewed-by: Kirill Reshke Discussion: https://www.postgresql.org/message-id/CALj2ACVfF2Uj9NoFy-5m98HNtjHpuD17EDE9twVeJng-jTAe7A%40mail.gmail.com --- src/backend/replication/walsender.c | 34 +++++++++++++++++----- src/test/subscription/t/001_rep_changes.pl | 5 ++-- 2 files changed, 29 insertions(+), 10 deletions(-) diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index e9331de3df5..422c128628c 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1098,6 +1098,7 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req WALReadError errinfo; XLogSegNo segno; TimeLineID currTLI; + Size rbytes; /* * Make sure we have enough WAL available before retrieving the current @@ -1157,16 +1158,33 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req else count = flushptr - targetPagePtr; /* part of the page available */ - /* now actually read the data, we know it's there */ - if (!WALRead(state, - cur_page, - targetPagePtr, - count, - currTLI, /* Pass the current TLI because only + /* attempt to read WAL from WAL buffers first */ + rbytes = WALReadFromBuffers(cur_page, targetPagePtr, count, currTLI); + + /* now read the remaining WAL from WAL file */ + if (rbytes < count) + { + if (!WALRead(state, + cur_page + rbytes, + targetPagePtr + rbytes, + count - rbytes, + currTLI, /* Pass the current TLI because only * WalSndSegmentOpen controls whether new TLI * is needed. */ - &errinfo)) - WALReadRaiseError(&errinfo); + &errinfo)) + WALReadRaiseError(&errinfo); + } + else if (state->seg.ws_file >= 0 && + !XLByteInSeg(targetPagePtr, state->seg.ws_segno, + state->segcxt.ws_segsize)) + { + /* + * Close the segment when a read that comes entirely from the WAL + * buffers is not in the open segment, so that the next file read + * reopens the correct one. + */ + state->routine.segment_close(state); + } /* * After reading into the buffer, check that what we read was valid. We do diff --git a/src/test/subscription/t/001_rep_changes.pl b/src/test/subscription/t/001_rep_changes.pl index 7d41715ed81..fb89f5e6583 100644 --- a/src/test/subscription/t/001_rep_changes.pl +++ b/src/test/subscription/t/001_rep_changes.pl @@ -189,10 +189,11 @@ is($result, qq(2), 'check replicated changes for table having no columns'); # Wait for the logical WAL sender to update its IO statistics. This is # done before the next restart, which would force a flush of its stats, and -# far enough from the reset done above to not impact the run time. +# far enough from the reset done above to not impact the run time. A WAL sender +# reads WAL either from a file or from the WAL buffers, so count both. $node_publisher->poll_query_until( 'postgres', - qq[SELECT sum(reads) > 0 + qq[SELECT sum(reads) + sum(hits) > 0 FROM pg_catalog.pg_stat_io WHERE backend_type = 'walsender' AND object = 'wal'] -- 2.47.3