Hi Fujii,
On Thu, Oct 22, 2026 at 10:38 AM Fujii Masao <[email protected]> wrote:
> I haven't investigated this yet, but wanted to report it in advance.
I can reproduce it with your steps on master, every time once the
three conditions hold, and also on REL_19, REL_18 and REL_17 with
cassert, so it goes back to when the summarizer was added.
Before promotion the summarizer has already gone past the recovery
target. In recovery GetLatestLSN() returns the walreceiver flush
position, and the walreceiver keeps streaming after replay pauses.
With your steps on master, replay pauses at the recovery target
0/03001B08, the walreceiver has flushed to 0/03001D98, and the
summarizer has summarized up to 0/03001CB0 and is waiting for WAL at
0/03001D98.
After promotion TLI 1 ends at 0/03001B08 ("1 0/03001B08 before LSN
0/03001B08" in 00000002.history), behind the summarizer. With gdb
attached to the summarizer:
1. summarizer_read_local_xlog_page() notices TLI 2 while waiting and
sets read_upto = switchpoint: 0/03001D98 -> 0/03001B08. The comment
there expects this ("this will cause read_upto to move backwards,
because we might have been promoted before reaching the end of the
previous timeline").
2. The next read is past read_upto on a timeline that is now historic,
so end_of_wal is set and SummarizeWAL() returns
summary_end_lsn = private_data->read_upto = 0/03001B08, although it
started at 0/03001CB0.
3. WalSummarizerMain(): current_lsn = 0/03001CB0,
end_of_summary_lsn = 0/03001B08 -> the assertion at line 460.
The main loop already handles "we passed the switch point": at the top
of the loop, if current_lsn >= switch_lsn it backs up to the switch
point on the new timeline. But that path only runs when the timeline
change is seen there. When it is seen inside SummarizeWAL(), the
backwards LSN comes back as the return value instead.
Without assertions nothing bad seems to happen. On a non-cassert build
of master with the same steps, the summarizer takes 0/03001B08 as its
position, sees TLI 2 on the next iteration, switches at 0/03001B08 and
goes on. It is the same process, and you end up with the same
summaries and state as with the patch below. So as far as I can tell
this only affects assert-enabled builds, but there the summarizer
crash makes the postmaster restart the whole cluster right after the
promotion.
The attached patch makes SummarizeWAL() report no progress when
read_upto ended up before the start of the summary, so the existing
code in the main loop does the rewind on the next iteration. With it,
your steps no longer trip the assertion, and summarization resumes on
TLI 2 exactly at the target, on master, 19, 18 and 17 (the patch
applies to all four). It also adds
src/bin/pg_walsummary/t/004_promote_past_target.pl, which follows your
steps: it fails with the TRAP on unpatched master and passes with the
fix. make check, pg_walsummary and pg_combinebackup pass.
Another option is to relax the assertion and keep the current
behavior, since that is already what a non-assert build does. I went
with the clamp because then SummarizeWAL() never returns an LSN before
its start, and summarized_lsn doesn't go backwards on the old timeline
for an iteration before the rewind. No strong opinion, though.
Regards,
Manu
From 055fcc39ee748c5771b8907bd7609ac5c27411ef Mon Sep 17 00:00:00 2001
From: Manu <[email protected]>
Date: Wed, 21 Oct 2026 23:17:12 -0300
Subject: [PATCH v1] Fix assertion failure in WAL summarizer after promotion
A standby's WAL summarizer summarizes up to what the walreceiver has
flushed, which can be well past the point where replay stops at a
recovery target. If the standby is then promoted, the old timeline
ends at a switch point that the summarizer has already passed.
When the summarizer notices the new timeline while waiting for WAL
inside SummarizeWAL(), summarizer_read_local_xlog_page() moves
read_upto backwards to the switch point, as its comments anticipate,
and the next read hits end of WAL. SummarizeWAL() then reported
read_upto as the end of the summary, an LSN before the one it started
from, and WalSummarizerMain() failed
Assert(end_of_summary_lsn >= current_lsn)
Builds without assertions were not affected in practice: the main
loop treated the bogus end LSN as its new position, noticed the
timeline switch on its next iteration and resumed from the switch
point on the new timeline.
Fix by having SummarizeWAL() report no progress in that case. The
main loop then backs up to the switch point through the code that
already exists for that purpose.
Add a test reproducing the scenario reported by Fujii Masao.
Reported-by: Fujii Masao <[email protected]>
Discussion: https://postgr.es/m/cahgqgwfbpzr2wkrku9j8w6e+-rzd6ovazs8n_kxvzfccfnj...@mail.gmail.com
Backpatch-through: 17
---
src/backend/postmaster/walsummarizer.c | 12 +++
src/bin/pg_walsummary/meson.build | 1 +
.../t/004_promote_past_target.pl | 85 +++++++++++++++++++
3 files changed, 98 insertions(+)
create mode 100644 src/bin/pg_walsummary/t/004_promote_past_target.pl
diff --git a/src/backend/postmaster/walsummarizer.c b/src/backend/postmaster/walsummarizer.c
index 8333a4eaaa..d88f694ba6 100644
--- a/src/backend/postmaster/walsummarizer.c
+++ b/src/backend/postmaster/walsummarizer.c
@@ -1287,6 +1287,18 @@ SummarizeWAL(TimeLineID tli, XLogRecPtr start_lsn, bool exact,
break;
}
+ /*
+ * If the timeline we were reading became historic while we were waiting
+ * for more WAL, summarizer_read_local_xlog_page() may have moved read_upto
+ * backwards to a switch point that precedes the point where this summary
+ * started (for example, when a standby that has summarized WAL beyond its
+ * recovery target is promoted). In that case we have summarized nothing,
+ * so report that we made no progress. The caller will notice the timeline
+ * switch on its next iteration and back up to the switch point.
+ */
+ if (summary_end_lsn < summary_start_lsn)
+ summary_end_lsn = summary_start_lsn;
+
/* Destroy xlogreader. */
pfree(xlogreader->private_data);
XLogReaderFree(xlogreader);
diff --git a/src/bin/pg_walsummary/meson.build b/src/bin/pg_walsummary/meson.build
index 839b644ef2..46e02070cb 100644
--- a/src/bin/pg_walsummary/meson.build
+++ b/src/bin/pg_walsummary/meson.build
@@ -26,6 +26,7 @@ tests += {
't/001_basic.pl',
't/002_blocks.pl',
't/003_tli_switch.pl',
+ 't/004_promote_past_target.pl',
],
}
}
diff --git a/src/bin/pg_walsummary/t/004_promote_past_target.pl b/src/bin/pg_walsummary/t/004_promote_past_target.pl
new file mode 100644
index 0000000000..036502e406
--- /dev/null
+++ b/src/bin/pg_walsummary/t/004_promote_past_target.pl
@@ -0,0 +1,85 @@
+# Copyright (c) 2021-2026, PostgreSQL Global Development Group
+#
+# A standby that stops at a recovery target can have summarized WAL beyond
+# that target, because the WAL summarizer reads whatever the walreceiver has
+# flushed. When such a standby is promoted, the old timeline ends at a point
+# the summarizer has already passed. The summarizer must back up to the switch
+# point and continue on the new timeline, rather than report a summary that
+# ends before it started (which used to trip an assertion).
+
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+my $primary = PostgreSQL::Test::Cluster->new('primary');
+$primary->init(allows_streaming => 1);
+$primary->append_conf('postgresql.conf', <<EOM);
+autovacuum = off
+summarize_wal = on
+EOM
+$primary->start;
+
+$primary->safe_psql('postgres', 'CREATE TABLE t (i int)');
+$primary->backup('backup');
+
+# The recovery target is the redo pointer of a checkpoint, so that the
+# standby's summarizer completes a summary that starts exactly there before
+# promotion; a second checkpoint lets it summarize past the target.
+$primary->safe_psql('postgres', 'INSERT INTO t VALUES (1); CHECKPOINT;');
+my $target_lsn = $primary->safe_psql('postgres',
+ 'SELECT redo_lsn FROM pg_control_checkpoint()');
+$primary->safe_psql('postgres', 'INSERT INTO t VALUES (2); CHECKPOINT;');
+
+my $standby = PostgreSQL::Test::Cluster->new('standby');
+$standby->init_from_backup($primary, 'backup', has_streaming => 1);
+$standby->append_conf('postgresql.conf', <<EOM);
+recovery_target_lsn = '$target_lsn'
+recovery_target_inclusive = false
+recovery_target_action = 'pause'
+log_min_messages = debug1
+EOM
+$standby->start;
+
+# Wait until replay is paused at the target, the summarizer has summarized
+# past it on timeline 1, and the summarizer is waiting for more WAL.
+$standby->poll_query_until('postgres',
+ "SELECT pg_get_wal_replay_pause_state() = 'paused'")
+ or die "timed out waiting for recovery to pause";
+$standby->poll_query_until('postgres', <<EOM)
+SELECT EXISTS (SELECT FROM pg_available_wal_summaries()
+ WHERE tli = 1 AND start_lsn = '$target_lsn')
+EOM
+ or die "timed out waiting for summary starting at $target_lsn";
+$standby->poll_query_until('postgres', <<EOM)
+SELECT EXISTS (SELECT FROM pg_stat_activity
+ WHERE backend_type = 'walsummarizer'
+ AND wait_event = 'WalSummarizerWal')
+EOM
+ or die "timed out waiting for summarizer to wait for WAL";
+my $summarizer_pid = $standby->safe_psql('postgres',
+ "SELECT pid FROM pg_stat_activity WHERE backend_type = 'walsummarizer'");
+
+my $log_offset = -s $standby->logfile;
+$standby->promote;
+$standby->safe_psql('postgres', 'CHECKPOINT');
+
+# Summarization must resume on timeline 2 exactly at the switch point.
+my $result = $standby->poll_query_until('postgres', <<EOM);
+SELECT EXISTS (SELECT FROM pg_available_wal_summaries()
+ WHERE tli = 2 AND start_lsn = '$target_lsn')
+EOM
+ok($result, "summarization resumed on timeline 2 at $target_lsn");
+
+is( $standby->safe_psql(
+ 'postgres',
+ "SELECT pid FROM pg_stat_activity WHERE backend_type = 'walsummarizer'"
+ ),
+ $summarizer_pid,
+ 'WAL summarizer survived promotion');
+ok( !$standby->log_contains(
+ qr/WAL summarizer process .* was terminated by signal/, $log_offset),
+ 'no WAL summarizer crash after promotion');
+
+done_testing();
--
2.56.0
