The cfbot's Linux 32-bit task failed on v3: the TAP test's padding loop raised "could not align the insert position". The loop computed the exact-fill payload as (gap - base), with base measured from a message with an empty payload. Once the payload exceeds the short data header's range, the record switches to the long data header, which is 3 bytes larger. With 8-byte MAXALIGN those extra bytes disappear into alignment padding and the fill still lands exactly on the target, but with 4-byte MAXALIGN they round up to a 4-byte overshoot, so on 32-bit builds every attempt missed the window and the loop gave up.
Here is v4, which approaches the target in small steps once the gap falls below base + 200 bytes, so the final exact-fill record always keeps the short data header. While at it, the window-hit check no longer hardcodes SizeOfXLogLongPHD as 40 bytes (36 on 32-bit): a normal switch reports the segment boundary itself and only the overridden EndPos lies past it, so any nonzero offset into the new segment marks the hit. No changes to the fix; 0001 is identical to v3. Regards, Paul Kim
From 2c036042833d439b0e8c026b015c17f6a8529fc8 Mon Sep 17 00:00:00 2001 From: Paul Kim <[email protected]> Date: Sat, 12 Sep 2026 21:00:48 +0900 Subject: [PATCH v4 1/2] Honor the WAL insertion adjustment in XLogBackgroundFlush WaitXLogInsertionsToFinish() adjusts a request that is past the end of reserved WAL down to the current reserved position and returns it. XLogBackgroundFlush() ignored that return value and passed its original request to XLogWrite(). In a non-assert build, a bogus asyncXactLSN just after a segment boundary can consequently advance the advertised write and flush positions through the new page header. A walsender can send that header alone, after which a standby can interpret stale contents of a recycled segment as a record. Assert builds instead fail the Insert >= Write invariant. Use the returned position when it is smaller than the request, and adjust both the write and flush targets. Do not assign it unconditionally, because the normal return value can be beyond the requested position. Also update the header comment of WaitXLogInsertionsToFinish(), which claimed that the return value is always >= 'upto', contradicting the adjustment documented in the function body. --- src/backend/access/transam/xlog.c | 28 +++++++++++++++++++++++++--- 1 file changed, 25 insertions(+), 3 deletions(-) diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 0bd4ae12420..05185a5e411 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -1540,7 +1540,10 @@ WALInsertLockUpdateInsertingAt(XLogRecPtr insertingAt) * Returns the location of the oldest insertion that is still in-progress. * Any WAL prior to that point has been fully copied into WAL buffers, and * can be flushed out to disk. Because this waits for any insertions older - * than 'upto' to finish, the return value is always >= 'upto'. + * than 'upto' to finish, the return value is normally >= 'upto'. However, + * if 'upto' is past the end of reserved WAL, the request is adjusted down + * to the current reserved position, and the return value can be smaller + * than 'upto'. Callers must not write or flush past the returned position. * * Note: When you are about to write out WAL, you must call this function * *before* acquiring WALWriteLock, to avoid deadlocks. This function might @@ -3008,6 +3011,7 @@ bool XLogBackgroundFlush(void) { XLogwrtRqst WriteRqst; + XLogRecPtr insertpos; bool flexible = true; static TimestampTz lastflush; TimestampTz now; @@ -3111,8 +3115,26 @@ XLogBackgroundFlush(void) START_CRIT_SECTION(); - /* now wait for any in-progress insertions to finish and get write lock */ - WaitXLogInsertionsToFinish(WriteRqst.Write); + /* now wait for any in-progress insertions to finish */ + insertpos = WaitXLogInsertionsToFinish(WriteRqst.Write); + + /* + * Honor the adjustment if the request was past the end of reserved + * WAL. Note that we must not assign the return value unconditionally + * the way XLogFlush() does: it is normally beyond the requested + * position, and the targets chosen above are deliberately conservative + * (Write backed off to a page boundary, Flush governed by + * wal_writer_flush_after, possibly a write-only cycle). So only ever + * lower them, keeping Flush <= Write. + */ + if (insertpos < WriteRqst.Write) + { + WriteRqst.Write = insertpos; + if (WriteRqst.Flush > insertpos) + WriteRqst.Flush = insertpos; + } + + /* get write lock */ LWLockAcquire(WALWriteLock, LW_EXCLUSIVE); RefreshXLogWriteResult(LogwrtResult); if (WriteRqst.Write > LogwrtResult.Write || -- 2.50.1 (Apple Git-155)
From 7d8f4b250c5c9abd278ab355c549af6a74058d03 Mon Sep 17 00:00:00 2001 From: Paul Kim <[email protected]> Date: Sat, 12 Sep 2026 21:02:26 +0900 Subject: [PATCH v4 2/2] Add a TAP test for the WAL insertion adjustment in XLogBackgroundFlush The failure reproduces without any fault injection: pad WAL so that an xlog-switch record starts exactly SizeOfXLogRecord bytes before a segment boundary. The switch's overridden EndPos then points past the next segment's long page header, and since pg_switch_wal() allocates no XID, the surrounding commit hands that position to XLogSetAsyncXactLSN() even with synchronous_commit = on. The walwriter's next cycle requests a flush past the end of reserved WAL. The test verifies that the walwriter survives the request and that the advertised flush position does not advance past the end of reserved WAL until real WAL is generated. --- src/test/modules/test_misc/meson.build | 1 + .../test_misc/t/016_walwriter_flush_adjust.pl | 144 ++++++++++++++++++ 2 files changed, 145 insertions(+) create mode 100644 src/test/modules/test_misc/t/016_walwriter_flush_adjust.pl diff --git a/src/test/modules/test_misc/meson.build b/src/test/modules/test_misc/meson.build index 5d81f5b13be..3b083025a4c 100644 --- a/src/test/modules/test_misc/meson.build +++ b/src/test/modules/test_misc/meson.build @@ -24,6 +24,7 @@ tests += { 't/013_temp_obj_multisession.pl', 't/014_log_statement_max_length.pl', 't/015_temp_schema_exit_deferrable.pl', + 't/016_walwriter_flush_adjust.pl', ], # The injection points are cluster-wide, so disable installcheck 'runningcheck': false, diff --git a/src/test/modules/test_misc/t/016_walwriter_flush_adjust.pl b/src/test/modules/test_misc/t/016_walwriter_flush_adjust.pl new file mode 100644 index 00000000000..5db265efcf2 --- /dev/null +++ b/src/test/modules/test_misc/t/016_walwriter_flush_adjust.pl @@ -0,0 +1,144 @@ +# Copyright (c) 2026, PostgreSQL Global Development Group + +# Test that XLogBackgroundFlush() honors the adjustment applied by +# WaitXLogInsertionsToFinish() when the flush request is past the end +# of generated WAL. +# +# Such a request arises naturally when an xlog-switch record starts +# exactly SizeOfXLogRecord bytes before a segment boundary: the +# overridden EndPos (segment boundary + SizeOfXLogLongPHD) reaches +# XLogSetAsyncXactLSN() via XactLastRecEnd, because the transaction +# around pg_switch_wal() has no XID and therefore commits +# asynchronously. Without the fix, the walwriter fails on one of +# XLogWrite()'s sanity checks (which one fires first depends on the +# WAL buffer state) and takes the server down, in both assert and +# production builds. + +use strict; +use warnings FATAL => 'all'; +use PostgreSQL::Test::Cluster; +use PostgreSQL::Test::Utils; +use Test::More; + +my $node = PostgreSQL::Test::Cluster->new('primary'); + +# Small segments keep the padding cheap. +$node->init(extra => ['--wal-segsize', '1']); +$node->append_conf( + 'postgresql.conf', qq( +autovacuum = off +wal_writer_delay = 10ms +wal_writer_flush_after = 0 +)); +$node->start; + +# Create a table for the later INSERT. +$node->safe_psql('postgres', 'CREATE TABLE t AS SELECT 1 AS i'); + +# Pad WAL with logical messages until the insert position is exactly +# SizeOfXLogRecord (24) bytes before a segment boundary, then switch. +# The record sizes involved are all MAXALIGNed, so filling the exact +# gap is possible; a record that crosses a page boundary picks up an +# extra page header and overshoots, in which case the loop moves on to +# the next segment boundary and tries again. +my $pad_and_switch = q{ +DO $$ +DECLARE + segsz numeric := (SELECT setting::numeric FROM pg_settings + WHERE name = 'wal_segment_size'); + zero pg_lsn := '0/0'; + base numeric; + cur numeric; + target numeric; + gap numeric; + seg0 numeric; +BEGIN + -- size of a logical message record with an empty payload + cur := pg_current_wal_insert_lsn() - zero; + PERFORM pg_logical_emit_message(false, 'x', ''); + base := (pg_current_wal_insert_lsn() - zero) - cur; + + seg0 := floor((pg_current_wal_insert_lsn() - zero) / segsz); + LOOP + cur := pg_current_wal_insert_lsn() - zero; + target := (floor(cur / segsz) + 1) * segsz - 24; + gap := target - cur; + EXIT WHEN gap = 0; + IF floor(cur / segsz) - seg0 > 3 THEN + RAISE EXCEPTION 'could not align the insert position'; + END IF; + IF gap >= base + 8192 THEN + PERFORM pg_logical_emit_message(false, 'x', repeat('a', 4096)); + ELSIF gap >= base + 200 THEN + -- approach the target in small steps, so that the final + -- record below stays under 256 bytes of main data and + -- keeps its short data header; a larger record would + -- switch to the long data header and its extra bytes + -- would spoil the exact fill unless MAXALIGN swallows + -- them + PERFORM pg_logical_emit_message(false, 'x', repeat('a', 64)); + ELSIF gap >= base THEN + PERFORM pg_logical_emit_message(false, 'x', + repeat('a', (gap - base)::int)); + ELSE + -- too close to the boundary, step over it and retry + PERFORM pg_logical_emit_message(false, 'x', ''); + END IF; + END LOOP; +END +$$; +SELECT pg_switch_wal() - '0/0'::pg_lsn; +}; + +my $segsz = 1024 * 1024; +my $bogus; +my $log_offset; + +# A concurrent WAL record (e.g. a bgwriter snapshot) between the +# padding and the switch can spoil the alignment; the switch then does +# not report the overridden position and we simply try again. +foreach my $attempt (1 .. 10) +{ + $log_offset = -s $node->logfile; + my $off = $node->safe_psql('postgres', $pad_and_switch); + + # A normal switch reports the segment boundary itself; only the + # overridden EndPos lies past it, at segment boundary + + # SizeOfXLogLongPHD (36 or 40 bytes, depending on MAXALIGN). Any + # nonzero offset into the new segment therefore marks the hit. + if ($off % $segsz != 0) + { + $bogus = $off; + last; + } +} +die "could not hit the segment-boundary switch window" + unless defined $bogus; + +# The walwriter's next cycle picks up the bogus request and logs the +# adjustment. +$node->wait_for_log(qr/request to flush past end of generated WAL/, + $log_offset); + +# The advertised flush position must not include the bogus request. +my $result = $node->safe_psql('postgres', + qq{SELECT pg_current_wal_flush_lsn() - '0/0'::pg_lsn < $bogus}); +is($result, 't', 'flush position stays below the bogus request'); + +# The walwriter must not have failed one of XLogWrite()'s sanity +# checks: no child process may have been terminated. +my $log = slurp_file($node->logfile, $log_offset); +unlike( + $log, + qr/terminating any other active server processes/, + 'no crash after the bogus flush request'); + +# Normal WAL activity gets past the bogus position. +$node->safe_psql('postgres', 'INSERT INTO t VALUES (2)'); +$node->safe_psql('postgres', 'SELECT pg_switch_wal()'); +$result = $node->safe_psql('postgres', + qq{SELECT pg_current_wal_flush_lsn() - '0/0'::pg_lsn > $bogus}); +is($result, 't', 'flush position advances past the bogus request'); + +$node->stop; +done_testing(); -- 2.50.1 (Apple Git-155)
