On Wednesday, November 18th, 2026 at 8:11 AM, Hayato Kuroda (Fujitsu) 
<[email protected]> wrote:

> Hi,

Hi Hayato-san,

> Thanks for updating the patch! I checked 0001 again and here is my comment.
> While reading the code again, I came up with another idea to clean up
> the code: how about making the code similar with the WAL generation
> side? Something like attached. Or is it matter of taste?

It reads better, thanks.  XLogRecHasBlockRef(record, 0) on the redo side
is exactly mod_wbuf on the generation side (nitups > 0 ||
is_prev_bucket_same_wrt), which is when _hash_freeovflpage() registers
block 0, so folding the two branches into one writebuf_registered flag
is equivalent and mirrors the writer.  I took it in v15.

The one thing I kept from the original is reading the primary bucket
page's cleanup lock before the write buffer in the non-same-wrt case,
which your version preserves: hash_xlog_lock_primary_bucket() still runs
first when the write page is not the primary bucket page.

v15 attached, rebased onto b8cbc63dcc7.
  0001: fold your simplification into hash_xlog_squeeze_page().

0001 passes the recovery suite under the CI UBSan flags and 061 passes,
as before.  I think 0001 and 0002 are ready; does one of you want to
take them, or is there more you want to see first?

best.

-greg
From 6581bb9556eab4393664e7dc3af2fd16e5e78d74 Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Thu, 29 Oct 2026 10:30:14 -0400
Subject: [PATCH v15 1/2] Don't register pin-only primary bucket pages in hash
 WAL records

Replay of XLOG_HASH_DELETE, XLOG_HASH_MOVE_PAGE_CONTENTS and
XLOG_HASH_SQUEEZE_PAGE must take a cleanup lock on the primary bucket
page so that no scan on a hot standby is in progress in the bucket, and
none can start, while tuples are removed or moved.  That lock was
obtained by registering the page (pin-only) with REGBUF_NO_CHANGE; redo
never reads data from the block, the registration existed only for the
lock.  That broke the rule that a registered buffer is one the caller
holds exclusively locked and dirty, and under wal_consistency_checking
it forced a full-page image of a page a concurrent inserter could still
modify, so the record CRC was computed over shifting data.

Stop registering the primary bucket page when the record does not change
it.  When the page is not one of the pages the record changes, the
operation holds only a pin on it, so the record carries the page's block
number in its main data and replay reads that block and takes a cleanup
lock on it directly, at the same point and in the same order as before.
The block number is appended only in that case, so the records are no
larger than before in the common case where the primary bucket page is
registered.  Replay takes the cleanup lock in a new helper,
hash_xlog_lock_primary_bucket(), shared by the three redo routines.

After this change hash no longer uses REGBUF_NO_CHANGE; a follow-up
removes the flag itself.

The three record formats change, so bump XLOG_PAGE_MAGIC; master only.

Author: Greg Burd <[email protected]>
Co-authored-by: Vignesh C <[email protected]>
Suggested-by: Amit Kapila <[email protected]>
Reviewed-by: Amit Kapila <[email protected]>
Reviewed-by: Hayato Kuroda <[email protected]>
Discussion: https://postgr.es/m/auNs8b_vS2OPdTL8@floki
---
 src/backend/access/hash/README      |   9 ++
 src/backend/access/hash/hash.c      |  18 ++--
 src/backend/access/hash/hash_xlog.c | 146 ++++++++++++++--------------
 src/backend/access/hash/hashovfl.c  |  99 ++++++++-----------
 src/include/access/hash_xlog.h      |  37 ++++---
 src/include/access/xlog_internal.h  |   2 +-
 6 files changed, 156 insertions(+), 155 deletions(-)

diff --git a/src/backend/access/hash/README b/src/backend/access/hash/README
index fc9031117c9..7774e1d2fe0 100644
--- a/src/backend/access/hash/README
+++ b/src/backend/access/hash/README
@@ -641,6 +641,15 @@ entire bucket.  After recovery, the operations will work correctly, but
 the index will remain bloated and this can impact performance of read and
 insert operations until the next vacuum squeeze the bucket completely.
 
+Replay of the delete, move-page-contents and squeeze-page records takes a
+cleanup lock on the primary bucket page before touching any other page, so
+that no scan on a hot standby is in progress in the bucket or can start in
+it while tuples are removed or moved; such a scan could otherwise miss
+tuples or return them twice.  When the primary bucket page is not one of
+the pages the record changes, the operation holds only a pin on it, so it
+is not registered in the record.  The record then carries its block number
+instead, and replay locks that block directly.
+
 
 Other Notes
 -----------
diff --git a/src/backend/access/hash/hash.c b/src/backend/access/hash/hash.c
index 63846c10840..9a919654ce5 100644
--- a/src/backend/access/hash/hash.c
+++ b/src/backend/access/hash/hash.c
@@ -909,19 +909,17 @@ hashbucketcleanup(Relation rel, Bucket cur_bucket, Buffer bucket_buf,
 				XLogRegisterData(&xlrec, SizeOfHashDelete);
 
 				/*
-				 * bucket buffer was not changed, but still needs to be
-				 * registered to ensure that we can acquire a cleanup lock on
-				 * it during replay.
+				 * Replay must take a cleanup lock on the primary bucket page
+				 * to serialize with scans on a hot standby.  If buf is an
+				 * overflow page, the unchanged primary page is held only by a
+				 * pin, so append its block number instead of registering it.
+				 * Otherwise block 0 identifies the primary page.
 				 */
 				if (!xlrec.is_primary_bucket_page)
-				{
-					uint8		flags = REGBUF_STANDARD | REGBUF_NO_IMAGE | REGBUF_NO_CHANGE;
-
-					XLogRegisterBuffer(0, bucket_buf, flags);
-				}
+					XLogRegisterData(&bucket_blkno, sizeof(BlockNumber));
 
-				XLogRegisterBuffer(1, buf, REGBUF_STANDARD);
-				XLogRegisterBufData(1, deletable,
+				XLogRegisterBuffer(0, buf, REGBUF_STANDARD);
+				XLogRegisterBufData(0, deletable,
 									ndeletable * sizeof(OffsetNumber));
 
 				recptr = XLogInsert(RM_HASH_ID, XLOG_HASH_DELETE);
diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c
index 46047c6ea27..f4816486618 100644
--- a/src/backend/access/hash/hash_xlog.c
+++ b/src/backend/access/hash/hash_xlog.c
@@ -470,6 +470,42 @@ hash_xlog_split_complete(XLogReaderState *record)
 		UnlockReleaseBuffer(newbuf);
 }
 
+/*
+ * Take a cleanup lock on an unregistered primary bucket page, whose block
+ * number is stored in the record's main data.  This serializes replay against
+ * scans on a hot standby.  block_id must identify a registered block of the
+ * same index.
+ */
+static Buffer
+hash_xlog_lock_primary_bucket(XLogReaderState *record, uint8 block_id, Size rec_size)
+{
+	RelFileLocator rlocator;
+	ForkNumber	forknum;
+	BlockNumber bucket_blkno;
+	Buffer		buf;
+
+	Assert(XLogRecGetDataLen(record) == rec_size + sizeof(BlockNumber));
+	memcpy(&bucket_blkno, XLogRecGetData(record) + rec_size, sizeof(BlockNumber));
+
+	/*
+	 * The cleanup lock is needed only when hot standby is active, because
+	 * scans cannot run before that.  Skipping the read also avoids a torn
+	 * copy of the primary page.  This record does not modify that page, so it
+	 * has no full-page image of it.  In crash recovery, or on a standby
+	 * started from a base backup, the page on disk can be torn and the image
+	 * that repairs it can come later in the WAL.
+	 */
+	if (!HotStandbyActiveInReplay())
+		return InvalidBuffer;
+
+	XLogRecGetBlockTag(record, block_id, &rlocator, &forknum, NULL);
+	buf = XLogReadBufferExtended(rlocator, forknum, bucket_blkno,
+								 RBM_NORMAL, InvalidBuffer);
+	if (BufferIsValid(buf))
+		LockBufferForCleanup(buf);
+	return buf;
+}
+
 /*
  * replay move of page contents for squeeze operation of hash index
  */
@@ -491,27 +527,12 @@ hash_xlog_move_page_contents(XLogReaderState *record)
 	 * can miss some records or show the same record multiple times.
 	 */
 	if (xldata->is_prim_bucket_same_wrt)
-		action = XLogReadBufferForRedoExtended(record, 1, RBM_NORMAL, true, &writebuf);
+		action = XLogReadBufferForRedoExtended(record, 0, RBM_NORMAL, true, &writebuf);
 	else
 	{
-		/*
-		 * The cleanup lock is needed only when hot standby is active, because
-		 * scans cannot run before that. Skipping the read also avoids a torn
-		 * copy of the primary page. This record does not modify that page, so
-		 * it has no full-page image of it. In crash recovery, or on a standby
-		 * started from a base backup, the page on disk can be torn and the
-		 * image that repairs it can come later in the WAL.
-		 */
-		if (HotStandbyActiveInReplay())
-		{
-			/*
-			 * we don't care for return value as the purpose of reading
-			 * bucketbuf is to ensure a cleanup lock on primary bucket page.
-			 */
-			(void) XLogReadBufferForRedoExtended(record, 0, RBM_NORMAL, true, &bucketbuf);
-		}
+		bucketbuf = hash_xlog_lock_primary_bucket(record, 0, SizeOfHashMovePageContents);
 
-		action = XLogReadBufferForRedo(record, 1, &writebuf);
+		action = XLogReadBufferForRedo(record, 0, &writebuf);
 	}
 
 	/* replay the record for adding entries in overflow buffer */
@@ -523,7 +544,7 @@ hash_xlog_move_page_contents(XLogReaderState *record)
 		Size		datalen;
 		uint16		ninserted = 0;
 
-		data = begin = XLogRecGetBlockData(record, 1, &datalen);
+		data = begin = XLogRecGetBlockData(record, 0, &datalen);
 
 		writepage = BufferGetPage(writebuf);
 
@@ -562,13 +583,13 @@ hash_xlog_move_page_contents(XLogReaderState *record)
 	}
 
 	/* replay the record for deleting entries from overflow buffer */
-	if (XLogReadBufferForRedo(record, 2, &deletebuf) == BLK_NEEDS_REDO)
+	if (XLogReadBufferForRedo(record, 1, &deletebuf) == BLK_NEEDS_REDO)
 	{
 		Page		page;
 		char	   *ptr;
 		Size		len;
 
-		ptr = XLogRecGetBlockData(record, 2, &len);
+		ptr = XLogRecGetBlockData(record, 1, &len);
 
 		page = BufferGetPage(deletebuf);
 
@@ -620,6 +641,7 @@ hash_xlog_squeeze_page(XLogReaderState *record)
 	Buffer		prevbuf = InvalidBuffer;
 	Buffer		mapbuf;
 	XLogRedoAction action;
+	bool		writebuf_registered;
 
 	/*
 	 * Ensure we have a cleanup lock on primary bucket page before we start
@@ -627,33 +649,22 @@ hash_xlog_squeeze_page(XLogReaderState *record)
 	 * scan can start nor a scan can be already-in-progress during the replay
 	 * of this operation.  If we allow scans during this operation, then they
 	 * can miss some records or show the same record multiple times.
+	 *
+	 * A modified write page is registered as block 0.  If it is also the
+	 * primary bucket page, its cleanup lock protects scans; otherwise, lock
+	 * the primary bucket page separately.
 	 */
-	if (xldata->is_prim_bucket_same_wrt)
-		action = XLogReadBufferForRedoExtended(record, 1, RBM_NORMAL, true, &writebuf);
-	else
-	{
-		/*
-		 * The cleanup lock is needed only when hot standby is active, because
-		 * scans cannot run before that. Skipping the read also avoids a torn
-		 * copy of the primary page. This record does not modify that page, so
-		 * it has no full-page image of it. In crash recovery, or on a standby
-		 * started from a base backup, the page on disk can be torn and the
-		 * image that repairs it can come later in the WAL.
-		 */
-		if (HotStandbyActiveInReplay())
-		{
-			/*
-			 * we don't care for return value as the purpose of reading
-			 * bucketbuf is to ensure a cleanup lock on primary bucket page.
-			 */
-			(void) XLogReadBufferForRedoExtended(record, 0, RBM_NORMAL, true, &bucketbuf);
-		}
+	writebuf_registered = XLogRecHasBlockRef(record, 0);
 
-		if (xldata->ntups > 0 || xldata->is_prev_bucket_same_wrt)
-			action = XLogReadBufferForRedo(record, 1, &writebuf);
-		else
-			action = BLK_NOTFOUND;
-	}
+	if (!(xldata->is_prim_bucket_same_wrt && writebuf_registered))
+		bucketbuf = hash_xlog_lock_primary_bucket(record, 1, SizeOfHashSqueezePage);
+
+	if (writebuf_registered)
+		action = XLogReadBufferForRedoExtended(record, 0, RBM_NORMAL,
+											   xldata->is_prim_bucket_same_wrt,
+											   &writebuf);
+	else
+		action = BLK_NOTFOUND;
 
 	/* replay the record for adding entries in overflow buffer */
 	if (action == BLK_NEEDS_REDO)
@@ -665,7 +676,7 @@ hash_xlog_squeeze_page(XLogReaderState *record)
 		uint16		ninserted = 0;
 		bool		mod_wbuf = false;
 
-		data = begin = XLogRecGetBlockData(record, 1, &datalen);
+		data = begin = XLogRecGetBlockData(record, 0, &datalen);
 
 		writepage = BufferGetPage(writebuf);
 
@@ -731,7 +742,7 @@ hash_xlog_squeeze_page(XLogReaderState *record)
 	}
 
 	/* replay the record for initializing overflow buffer */
-	if (XLogReadBufferForRedo(record, 2, &ovflbuf) == BLK_NEEDS_REDO)
+	if (XLogReadBufferForRedo(record, 1, &ovflbuf) == BLK_NEEDS_REDO)
 	{
 		Page		ovflpage;
 		HashPageOpaque ovflopaque;
@@ -756,7 +767,7 @@ hash_xlog_squeeze_page(XLogReaderState *record)
 
 	/* replay the record for page previous to the freed overflow page */
 	if (!xldata->is_prev_bucket_same_wrt &&
-		XLogReadBufferForRedo(record, 3, &prevbuf) == BLK_NEEDS_REDO)
+		XLogReadBufferForRedo(record, 2, &prevbuf) == BLK_NEEDS_REDO)
 	{
 		Page		prevpage = BufferGetPage(prevbuf);
 		HashPageOpaque prevopaque = HashPageGetOpaque(prevpage);
@@ -770,11 +781,11 @@ hash_xlog_squeeze_page(XLogReaderState *record)
 		UnlockReleaseBuffer(prevbuf);
 
 	/* replay the record for page next to the freed overflow page */
-	if (XLogRecHasBlockRef(record, 4))
+	if (XLogRecHasBlockRef(record, 3))
 	{
 		Buffer		nextbuf;
 
-		if (XLogReadBufferForRedo(record, 4, &nextbuf) == BLK_NEEDS_REDO)
+		if (XLogReadBufferForRedo(record, 3, &nextbuf) == BLK_NEEDS_REDO)
 		{
 			Page		nextpage = BufferGetPage(nextbuf);
 			HashPageOpaque nextopaque = HashPageGetOpaque(nextpage);
@@ -801,7 +812,7 @@ hash_xlog_squeeze_page(XLogReaderState *record)
 	 * index updates can be happening concurrently.
 	 */
 	/* replay the record for bitmap page */
-	if (XLogReadBufferForRedo(record, 5, &mapbuf) == BLK_NEEDS_REDO)
+	if (XLogReadBufferForRedo(record, 4, &mapbuf) == BLK_NEEDS_REDO)
 	{
 		Page		mappage = BufferGetPage(mapbuf);
 		uint32	   *freep = NULL;
@@ -811,7 +822,7 @@ hash_xlog_squeeze_page(XLogReaderState *record)
 
 		freep = HashPageGetBitmap(mappage);
 
-		data = XLogRecGetBlockData(record, 5, &datalen);
+		data = XLogRecGetBlockData(record, 4, &datalen);
 		bitmap_page_bit = (uint32 *) data;
 
 		CLRBIT(freep, *bitmap_page_bit);
@@ -823,11 +834,11 @@ hash_xlog_squeeze_page(XLogReaderState *record)
 		UnlockReleaseBuffer(mapbuf);
 
 	/* replay the record for meta page */
-	if (XLogRecHasBlockRef(record, 6))
+	if (XLogRecHasBlockRef(record, 5))
 	{
 		Buffer		metabuf;
 
-		if (XLogReadBufferForRedo(record, 6, &metabuf) == BLK_NEEDS_REDO)
+		if (XLogReadBufferForRedo(record, 5, &metabuf) == BLK_NEEDS_REDO)
 		{
 			HashMetaPage metap;
 			Page		page;
@@ -835,7 +846,7 @@ hash_xlog_squeeze_page(XLogReaderState *record)
 			uint32	   *firstfree_ovflpage;
 			Size		datalen;
 
-			data = XLogRecGetBlockData(record, 6, &datalen);
+			data = XLogRecGetBlockData(record, 5, &datalen);
 			firstfree_ovflpage = (uint32 *) data;
 
 			page = BufferGetPage(metabuf);
@@ -871,27 +882,12 @@ hash_xlog_delete(XLogReaderState *record)
 	 * can miss some records or show the same record multiple times.
 	 */
 	if (xldata->is_primary_bucket_page)
-		action = XLogReadBufferForRedoExtended(record, 1, RBM_NORMAL, true, &deletebuf);
+		action = XLogReadBufferForRedoExtended(record, 0, RBM_NORMAL, true, &deletebuf);
 	else
 	{
-		/*
-		 * The cleanup lock is needed only when hot standby is active, because
-		 * scans cannot run before that. Skipping the read also avoids a torn
-		 * copy of the primary page. This record does not modify that page, so
-		 * it has no full-page image of it. In crash recovery, or on a standby
-		 * started from a base backup, the page on disk can be torn and the
-		 * image that repairs it can come later in the WAL.
-		 */
-		if (HotStandbyActiveInReplay())
-		{
-			/*
-			 * we don't care for return value as the purpose of reading
-			 * bucketbuf is to ensure a cleanup lock on primary bucket page.
-			 */
-			(void) XLogReadBufferForRedoExtended(record, 0, RBM_NORMAL, true, &bucketbuf);
-		}
+		bucketbuf = hash_xlog_lock_primary_bucket(record, 0, SizeOfHashDelete);
 
-		action = XLogReadBufferForRedo(record, 1, &deletebuf);
+		action = XLogReadBufferForRedo(record, 0, &deletebuf);
 	}
 
 	/* replay the record for deleting entries in bucket page */
@@ -900,7 +896,7 @@ hash_xlog_delete(XLogReaderState *record)
 		char	   *ptr;
 		Size		len;
 
-		ptr = XLogRecGetBlockData(record, 1, &len);
+		ptr = XLogRecGetBlockData(record, 0, &len);
 
 		page = BufferGetPage(deletebuf);
 
diff --git a/src/backend/access/hash/hashovfl.c b/src/backend/access/hash/hashovfl.c
index dbc57ef958c..35f214767bd 100644
--- a/src/backend/access/hash/hashovfl.c
+++ b/src/backend/access/hash/hashovfl.c
@@ -580,9 +580,13 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	/* Get write-lock on metapage to update firstfree */
 	LockBuffer(metabuf, BUFFER_LOCK_EXCLUSIVE);
 
-	/* This operation needs to log multiple tuples, prepare WAL for that */
+	/*
+	 * This operation needs to log multiple tuples, prepare WAL for that.  One
+	 * extra data segment accounts for the primary bucket page's block number,
+	 * appended to the record when that page is pin-only (not registered).
+	 */
 	if (RelationNeedsWAL(rel))
-		XLogEnsureRecordSpace(HASH_XLOG_FREE_OVFL_BUFS, 4 + nitups);
+		XLogEnsureRecordSpace(HASH_XLOG_FREE_OVFL_BUFS, 5 + nitups);
 
 	START_CRIT_SECTION();
 
@@ -656,6 +660,7 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 	if (RelationNeedsWAL(rel))
 	{
 		xl_hash_squeeze_page xlrec;
+		BlockNumber bucket_blkno = BufferGetBlockNumber(bucketbuf);
 
 		xlrec.prevblkno = prevblkno;
 		xlrec.nextblkno = nextblkno;
@@ -667,45 +672,26 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 		XLogRegisterData(&xlrec, SizeOfHashSqueezePage);
 
 		/*
-		 * bucket buffer was not changed, but still needs to be registered to
-		 * ensure that we can acquire a cleanup lock on it during replay.
+		 * Replay needs the primary bucket block number to acquire a cleanup
+		 * lock.  Register it as block 0 when it is modified; otherwise append
+		 * its block number to the record.
 		 */
-		if (!is_prim_bucket_same_wrt)
-		{
-			uint8		flags = REGBUF_STANDARD | REGBUF_NO_IMAGE | REGBUF_NO_CHANGE;
-
-			XLogRegisterBuffer(0, bucketbuf, flags);
-		}
+		if (!(is_prim_bucket_same_wrt && mod_wbuf))
+			XLogRegisterData(&bucket_blkno, sizeof(BlockNumber));
 
-		if (nitups > 0)
-		{
-			XLogRegisterBuffer(1, wbuf, REGBUF_STANDARD);
-			XLogRegisterBufData(1, itup_offsets,
-								nitups * sizeof(OffsetNumber));
-			for (int i = 0; i < nitups; i++)
-				XLogRegisterBufData(1, itups[i], tups_size[i]);
-		}
-		else if (is_prim_bucket_same_wrt || is_prev_bucket_same_wrt)
+		if (mod_wbuf)
 		{
-			uint8		wbuf_flags;
-
-			/*
-			 * A write buffer needs to be registered even if no tuples are
-			 * added to it to ensure that we can acquire a cleanup lock on it
-			 * if it is the same as primary bucket buffer or update the
-			 * nextblkno if it is same as the previous bucket buffer.
-			 */
-			Assert(nitups == 0);
-
-			wbuf_flags = REGBUF_STANDARD;
-			if (!is_prev_bucket_same_wrt)
-				wbuf_flags |= REGBUF_NO_CHANGE;
-			else
-				Assert(mod_wbuf);
-			XLogRegisterBuffer(1, wbuf, wbuf_flags);
+			XLogRegisterBuffer(0, wbuf, REGBUF_STANDARD);
+			if (nitups > 0)
+			{
+				XLogRegisterBufData(0, itup_offsets,
+									nitups * sizeof(OffsetNumber));
+				for (int i = 0; i < nitups; i++)
+					XLogRegisterBufData(0, itups[i], tups_size[i]);
+			}
 		}
 
-		XLogRegisterBuffer(2, ovflbuf, REGBUF_STANDARD);
+		XLogRegisterBuffer(1, ovflbuf, REGBUF_STANDARD);
 
 		/*
 		 * If prevpage and the writepage (block in which we are moving tuples
@@ -714,18 +700,18 @@ _hash_freeovflpage(Relation rel, Buffer bucketbuf, Buffer ovflbuf,
 		 * writepage.
 		 */
 		if (BufferIsValid(prevbuf) && !is_prev_bucket_same_wrt)
-			XLogRegisterBuffer(3, prevbuf, REGBUF_STANDARD);
+			XLogRegisterBuffer(2, prevbuf, REGBUF_STANDARD);
 
 		if (BufferIsValid(nextbuf))
-			XLogRegisterBuffer(4, nextbuf, REGBUF_STANDARD);
+			XLogRegisterBuffer(3, nextbuf, REGBUF_STANDARD);
 
-		XLogRegisterBuffer(5, mapbuf, REGBUF_STANDARD);
-		XLogRegisterBufData(5, &bitmapbit, sizeof(uint32));
+		XLogRegisterBuffer(4, mapbuf, REGBUF_STANDARD);
+		XLogRegisterBufData(4, &bitmapbit, sizeof(uint32));
 
 		if (update_metap)
 		{
-			XLogRegisterBuffer(6, metabuf, REGBUF_STANDARD);
-			XLogRegisterBufData(6, &metap->hashm_firstfree, sizeof(uint32));
+			XLogRegisterBuffer(5, metabuf, REGBUF_STANDARD);
+			XLogRegisterBufData(5, &metap->hashm_firstfree, sizeof(uint32));
 		}
 
 		recptr = XLogInsert(RM_HASH_ID, XLOG_HASH_SQUEEZE_PAGE);
@@ -966,10 +952,12 @@ readpage:
 
 					/*
 					 * This operation needs to log multiple tuples, prepare
-					 * WAL for that.
+					 * WAL for that.  One extra data segment accounts for the
+					 * primary bucket page's block number, appended to the
+					 * record when that page is pin-only (not registered).
 					 */
 					if (RelationNeedsWAL(rel))
-						XLogEnsureRecordSpace(0, 3 + nitups);
+						XLogEnsureRecordSpace(0, 4 + nitups);
 
 					START_CRIT_SECTION();
 
@@ -998,25 +986,22 @@ readpage:
 						XLogRegisterData(&xlrec, SizeOfHashMovePageContents);
 
 						/*
-						 * bucket buffer was not changed, but still needs to
-						 * be registered to ensure that we can acquire a
-						 * cleanup lock on it during replay.
+						 * If wbuf is an overflow page, the primary bucket
+						 * page is not changed and we hold only a pin on it,
+						 * so it is not registered.  Append its block number
+						 * instead, and replay takes a cleanup lock on it.
 						 */
 						if (!xlrec.is_prim_bucket_same_wrt)
-						{
-							int			flags = REGBUF_STANDARD | REGBUF_NO_IMAGE | REGBUF_NO_CHANGE;
-
-							XLogRegisterBuffer(0, bucket_buf, flags);
-						}
+							XLogRegisterData(&bucket_blkno, sizeof(BlockNumber));
 
-						XLogRegisterBuffer(1, wbuf, REGBUF_STANDARD);
-						XLogRegisterBufData(1, itup_offsets,
+						XLogRegisterBuffer(0, wbuf, REGBUF_STANDARD);
+						XLogRegisterBufData(0, itup_offsets,
 											nitups * sizeof(OffsetNumber));
 						for (i = 0; i < nitups; i++)
-							XLogRegisterBufData(1, itups[i], tups_size[i]);
+							XLogRegisterBufData(0, itups[i], tups_size[i]);
 
-						XLogRegisterBuffer(2, rbuf, REGBUF_STANDARD);
-						XLogRegisterBufData(2, deletable,
+						XLogRegisterBuffer(1, rbuf, REGBUF_STANDARD);
+						XLogRegisterBufData(1, deletable,
 											ndeletable * sizeof(OffsetNumber));
 
 						recptr = XLogInsert(RM_HASH_ID, XLOG_HASH_MOVE_PAGE_CONTENTS);
diff --git a/src/include/access/hash_xlog.h b/src/include/access/hash_xlog.h
index 149aed69dc5..0bb0d0590f5 100644
--- a/src/include/access/hash_xlog.h
+++ b/src/include/access/hash_xlog.h
@@ -129,9 +129,13 @@ typedef struct xl_hash_split_complete
  *
  * This data record is used for XLOG_HASH_MOVE_PAGE_CONTENTS
  *
- * Backup Blk 0: primary bucket page
- * Backup Blk 1: page containing moved tuples
- * Backup Blk 2: page from which tuples will be removed
+ * Backup Blk 0: page containing moved tuples
+ * Backup Blk 1: page from which tuples will be removed
+ *
+ * The primary bucket page is not registered.  If it is not the page moved to
+ * (is_prim_bucket_same_wrt is false), the block number of the primary bucket
+ * page follows the struct as a BlockNumber, and replay takes a cleanup lock
+ * on that block.
  */
 typedef struct xl_hash_move_page_contents
 {
@@ -149,13 +153,18 @@ typedef struct xl_hash_move_page_contents
  *
  * This data record is used for XLOG_HASH_SQUEEZE_PAGE
  *
- * Backup Blk 0: primary bucket page
- * Backup Blk 1: page containing tuples moved from freed overflow page
- * Backup Blk 2: freed overflow page
- * Backup Blk 3: page previous to the freed overflow page
- * Backup Blk 4: page next to the freed overflow page
- * Backup Blk 5: bitmap page containing info of freed overflow page
- * Backup Blk 6: meta page
+ * Backup Blk 0: page containing tuples moved from freed overflow page, if
+ *				 it is modified
+ * Backup Blk 1: freed overflow page
+ * Backup Blk 2: page previous to the freed overflow page
+ * Backup Blk 3: page next to the freed overflow page
+ * Backup Blk 4: bitmap page containing info of freed overflow page
+ * Backup Blk 5: meta page
+ *
+ * The primary bucket page is registered only as block 0, when it is the
+ * modified write page.  Otherwise the block number of the primary bucket
+ * page follows the struct as a BlockNumber, and replay takes a cleanup lock
+ * on that block.
  */
 typedef struct xl_hash_squeeze_page
 {
@@ -179,8 +188,12 @@ typedef struct xl_hash_squeeze_page
  *
  * This data record is used for XLOG_HASH_DELETE
  *
- * Backup Blk 0: primary bucket page
- * Backup Blk 1: page from which tuples are deleted
+ * Backup Blk 0: page from which tuples are deleted
+ *
+ * If block 0 is not the primary bucket page (is_primary_bucket_page is
+ * false), the block number of the primary bucket page follows the struct as a
+ * BlockNumber, and replay takes a cleanup lock on that block.  The primary
+ * bucket page is not registered.
  */
 typedef struct xl_hash_delete
 {
diff --git a/src/include/access/xlog_internal.h b/src/include/access/xlog_internal.h
index bf609c3c703..ed8f488dda5 100644
--- a/src/include/access/xlog_internal.h
+++ b/src/include/access/xlog_internal.h
@@ -32,7 +32,7 @@
 /*
  * Each page of XLOG file has a header like this:
  */
-#define XLOG_PAGE_MAGIC 0xD122	/* can be used as WAL version indicator */
+#define XLOG_PAGE_MAGIC 0xD123	/* can be used as WAL version indicator */
 
 typedef struct XLogPageHeaderData
 {

base-commit: b8cbc63dcc77cd2b9cbee2bc7ca64e471a2118ee
-- 
2.54.0

From 959edebc3ac8002e1312098c76572f39377e91cf Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Fri, 30 Oct 2026 10:10:03 -0400
Subject: [PATCH v15 2/2] Add a recovery test for the hash WAL pin-only bucket
 page fix

Add recovery/t/061_hash_vacuum_insert_crc.pl, which exercises the
interleaving the previous commits make safe: a hash index VACUUM logging
an overflow-page delete while a concurrent INSERT changes the primary
bucket page it holds pin-only, with wal_consistency_checking='hash'.

The test adds a general, test-only injection point in
XLogRecordAssemble(), right after the record CRC is computed and before
the record is copied into the WAL buffers.  Its argument names the
record as "rmid/info/mainlen" so a test can wait on exactly one kind of
record; here the test waits on the overflow-page XLOG_HASH_DELETE (whose
main data carries the appended primary bucket block number) while VACUUM
holds the primary bucket page pin-only, then lets an INSERT change that
page, then crashes and checks that pg_waldump and crash recovery both
accept the WAL with no "incorrect resource manager data checksum".

Before 27caf4ca5ae5 and this rework, the record's CRC was computed over
the concurrently-modified primary bucket page and replay failed; the
injection point makes that window deterministic so the case is covered
from now on.

Author: Greg Burd <[email protected]>
Co-authored-by: Vignesh C <[email protected]>
Reviewed-by: Amit Kapila <[email protected]>
Reviewed-by: Joao Rito <[email protected]>
Discussion: https://postgr.es/m/CALDaNm14vRb2TZWyhjSHx=SyT7R=e7+ackrmxuojdwwee40...@mail.gmail.com
---
 src/backend/access/transam/xloginsert.c       |  22 ++
 src/test/recovery/meson.build                 |   1 +
 .../recovery/t/061_hash_vacuum_insert_crc.pl  | 191 ++++++++++++++++++
 3 files changed, 214 insertions(+)
 create mode 100644 src/test/recovery/t/061_hash_vacuum_insert_crc.pl

diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c
index 341deb7b80d..cd3359434e1 100644
--- a/src/backend/access/transam/xloginsert.c
+++ b/src/backend/access/transam/xloginsert.c
@@ -39,6 +39,7 @@
 #include "replication/origin.h"
 #include "storage/bufmgr.h"
 #include "storage/proc.h"
+#include "utils/injection_point.h"
 #include "utils/memutils.h"
 #include "utils/pgstat_internal.h"
 #include "utils/rel.h"
@@ -991,6 +992,27 @@ XLogRecordAssemble(RmgrId rmid, uint8 info,
 	for (rdt = hdr_rdt.next; rdt != NULL; rdt = rdt->next)
 		COMP_CRC32C(rdata_crc, rdt->data, rdt->len);
 
+	/*
+	 * Test hook: let a test modify a buffer after the record's CRC has been
+	 * computed but before the record is copied into the WAL buffers, to
+	 * reproduce the window in which a concurrently modified, pin-only
+	 * registered buffer could make the on-disk CRC not match the record.  The
+	 * argument names the record as "rmid/info/mainlen" (resource manager, its
+	 * info bits, and the length of the main data) so a test can wait on
+	 * exactly one kind of record.  This runs inside a critical section, so it
+	 * must not allocate; it fires only in a backend that loaded the point
+	 * beforehand.
+	 */
+#ifdef USE_INJECTION_POINTS
+	{
+		char		pointarg[24];
+
+		snprintf(pointarg, sizeof(pointarg), "%d/%02X/" UINT64_FORMAT,
+				 rmid, info, mainrdata_len);
+		INJECTION_POINT_CACHED("xlog-assemble-after-crc", pointarg);
+	}
+#endif
+
 	/*
 	 * Ensure that the XLogRecord is not too large.
 	 *
diff --git a/src/test/recovery/meson.build b/src/test/recovery/meson.build
index b6ac9dd07a7..5051031df56 100644
--- a/src/test/recovery/meson.build
+++ b/src/test/recovery/meson.build
@@ -69,6 +69,7 @@ tests += {
       't/058_shutdown_crash_restart.pl',
       't/059_remote_apply_status_interval.pl',
       't/060_multixact_crashed_updater.pl',
+      't/061_hash_vacuum_insert_crc.pl',
     ],
   },
 }
diff --git a/src/test/recovery/t/061_hash_vacuum_insert_crc.pl b/src/test/recovery/t/061_hash_vacuum_insert_crc.pl
new file mode 100644
index 00000000000..89f7611af12
--- /dev/null
+++ b/src/test/recovery/t/061_hash_vacuum_insert_crc.pl
@@ -0,0 +1,191 @@
+# Copyright (c) 2026, PostgreSQL Global Development Group
+
+# Test that the WAL record written by hash index VACUUM for an overflow page is
+# intact when a concurrent INSERT changes the primary bucket page.
+#
+# hashbucketcleanup() keeps only a pin on the primary bucket page while it
+# walks the overflow chain.  Before the hash WAL rework it registered that page
+# (REGBUF_STANDARD | REGBUF_NO_IMAGE | REGBUF_NO_CHANGE) with the
+# XLOG_HASH_DELETE record it writes for each overflow page.  With
+# wal_consistency_checking enabled for the hash resource manager,
+# XLogRecordAssemble() included a full image of every registered block, so it
+# read the unlocked primary bucket page.  If an INSERT added a tuple to that
+# page after the record's CRC was computed but before the page was copied into
+# the WAL buffers, the record was written with a CRC that did not match its
+# contents, and replay stopped at it with "incorrect resource manager data
+# checksum".
+#
+# The window is a few microseconds wide, so the test widens it with an
+# injection point in XLogRecordAssemble(), placed right after the CRC is
+# computed.  It is attached to the VACUUM session only and matches just the
+# overflow-page delete, identified by "rmgr id/xl_info/main-data length".  The
+# point still fires after the fix, so the test exercises the same interleaving
+# on a correct server.
+#
+# The assertions state the correct behavior (all WAL is valid and replays), so
+# the test fails while the bug exists.
+
+use strict;
+use warnings FATAL => 'all';
+
+use IPC::Run;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+if ($ENV{enable_injection_points} ne 'yes')
+{
+	plan skip_all => 'Injection points not supported by this build';
+}
+
+my $node = PostgreSQL::Test::Cluster->new('primary');
+$node->init;
+$node->append_conf(
+	'postgresql.conf', qq{
+wal_consistency_checking = 'hash'
+autovacuum = off
+fsync = off
+checkpoint_timeout = 1h
+});
+$node->start;
+
+if (!$node->check_extension('injection_points'))
+{
+	plan skip_all => 'Extension injection_points not installed';
+}
+$node->safe_psql('postgres', 'CREATE EXTENSION injection_points');
+
+my $hash_rmid = $node->safe_psql('postgres',
+	q{SELECT rm_id FROM pg_get_wal_resource_managers() WHERE rm_name = 'Hash'}
+);
+
+# Build a single bucket whose primary page stays live and whose overflow page
+# is all dead, so VACUUM logs an XLOG_HASH_DELETE for the overflow page only:
+# the primary page has no dead index entries, so no delete record is written
+# for it, and the only consistency-checked hash DELETE in the window is the
+# overflow one, logged while the primary page is held pin-only.  All rows share
+# the key, so they land in one bucket; the first batch fills most of the
+# primary page and stays live, the second spills onto the overflow page and is
+# deleted.  Free slots are left on the primary page so the concurrent INSERT
+# below lands there.
+#
+# hashbuild() sizes the index from estimate_rel_size() on the heap, which for a
+# table that has never been vacuumed applies a floor of 10 pages.  Vacuuming
+# first sets reltuples to 0, so the estimate is believed and the index starts
+# with the minimum of 2 buckets.
+$node->safe_psql(
+	'postgres', q{
+CREATE TABLE hash_t (k int, tag int) WITH (autovacuum_enabled = false);
+VACUUM hash_t;
+CREATE INDEX hash_i ON hash_t USING hash (k);
+INSERT INTO hash_t SELECT 1, 0 FROM generate_series(1, 390);
+INSERT INTO hash_t SELECT 1, 1 FROM generate_series(1, 210);
+DELETE FROM hash_t WHERE tag = 1;
+});
+
+# Premise: metapage, two bucket pages, a bitmap page and one overflow page.
+is( $node->safe_psql(
+		'postgres',
+		q{SELECT pg_relation_size('hash_i') / current_setting('block_size')::int}
+	),
+	'5',
+	'premise: the index has a primary bucket page and one overflow page');
+
+# Start from a checkpoint, so crash recovery replays everything below and
+# pg_waldump has a record boundary to start from.
+$node->safe_psql('postgres', 'CHECKPOINT');
+my $start_lsn = $node->safe_psql('postgres', 'SELECT pg_current_wal_lsn()');
+
+# The VACUUM session.  The point is attached to this session only and must be
+# loaded before the critical section in which it runs.
+#
+# Wait on the XLOG_HASH_DELETE record for the overflow page, not the one for
+# the primary page.  Both are hash records (rm id $hash_rmid) with info 0x92
+# (XLOG_HASH_DELETE 0x90 | XLR_CHECK_CONSISTENCY 0x02).  They differ in main
+# data length: the primary-page delete carries just xl_hash_delete
+# (SizeOfHashDelete, two bools = 2 bytes), while the overflow-page delete also
+# appends the primary bucket page's block number (a 4-byte BlockNumber), for 6
+# bytes.  Waiting on the 6-byte one parks VACUUM while it holds the primary
+# bucket page pin-only, which is the page the INSERT below changes.
+my $overflow_delete = "$hash_rmid/92/6";
+my $vacuum = $node->background_psql('postgres');
+my $vacuum_pid = $vacuum->query_safe('SELECT pg_backend_pid()');
+$vacuum->query_safe(q{SELECT injection_points_set_local()});
+$vacuum->query_safe(
+	qq{SELECT injection_points_attach('xlog-assemble-after-crc', 'wait', '$overflow_delete')}
+);
+$vacuum->query_safe(
+	q{SELECT injection_points_load('xlog-assemble-after-crc')});
+$vacuum->query_until(
+	qr/starting_vacuum/,
+	q(\echo starting_vacuum
+VACUUM (INDEX_CLEANUP on) hash_t;
+));
+
+# VACUUM has removed the dead entries from the primary page, released its lock
+# (keeping the pin), and is now logging the removal from the overflow page: the
+# CRC of that record is computed and the unlocked primary page is not yet
+# copied.
+$node->poll_query_until('postgres',
+	"SELECT wait_event = 'xlog-assemble-after-crc' FROM pg_stat_activity WHERE pid = $vacuum_pid"
+) or die "VACUUM did not reach the injection point";
+note('VACUUM is between computing the CRC and copying the record');
+
+# Now there is room in the primary page, so this INSERT changes it while VACUUM
+# holds it pin-only during the overflow delete.
+$node->safe_psql('postgres', q{INSERT INTO hash_t VALUES (1, 0)});
+
+$node->safe_psql('postgres',
+	q{SELECT injection_points_wakeup('xlog-assemble-after-crc')});
+$vacuum->quit;
+
+my $end_lsn =
+  $node->safe_psql('postgres', 'SELECT pg_current_wal_insert_lsn()');
+$node->poll_query_until('postgres',
+	"SELECT pg_current_wal_flush_lsn() >= '$end_lsn'")
+  or die "timed out waiting for WAL flush";
+
+# The first and the last WAL segment of the range.  With only --path, pg_waldump
+# takes the WAL segment size from the first file readdir() returns, which can be
+# a zero-filled, unused segment, and then fails with "invalid WAL segment size".
+# Naming the segments makes it read the size from the first of them.
+my $start_seg =
+  $node->safe_psql('postgres', "SELECT pg_walfile_name('$start_lsn')");
+my $end_seg =
+  $node->safe_psql('postgres', "SELECT pg_walfile_name('$end_lsn')");
+
+# Crash, so that start-up has to replay (and CRC-check) the workload.
+$node->stop('immediate');
+
+# 1. Every record in the range must have a valid CRC.  pg_waldump verifies it
+# for every record it reads, whatever the --rmgr filter.
+my $stderr;
+my $ok = IPC::Run::run(
+	[
+		'pg_waldump',
+		'--path' => $node->data_dir . '/pg_wal',
+		'--start' => $start_lsn,
+		'--end' => $end_lsn,
+		'--rmgr' => 'Hash',
+		$start_seg, $end_seg,
+	],
+	'>' => '/dev/null',
+	'2>' => \$stderr);
+
+ok($ok, 'pg_waldump reads the whole workload without errors');
+unlike(
+	$stderr,
+	qr/incorrect resource manager data checksum/,
+	'no WAL record has a mismatching CRC');
+
+# 2. Crash recovery must replay it without complaint.
+my $log_offset = -s $node->logfile;
+$node->start;
+my $log = slurp_file($node->logfile, $log_offset);
+unlike(
+	$log,
+	qr/incorrect resource manager data checksum|inconsistent page found/,
+	'crash recovery replays the workload without CRC or consistency errors');
+
+$node->stop;
+done_testing();
-- 
2.54.0

Reply via email to