Hi, Shutdown may be indefinitely stuck under the following circumstances: - Data checksum is enabled (needed to generate FPI_FOR_HINT record) - A logical replication walsender is running - A select in an explicit ongoing transaction pruned a heap page and logged a FPI_FOR_HINT record. This record is likely going to be a contrecord and start a new page.
Starting the shutdown will kill this ongoing transaction. Since the transaction doesn't have an allocated xid, the FPI_FOR_HINT record will be left unflushed. When the checkpointer calls ShutdownXLOG(), all walsenders will be notified to stop. However, the logical replication walsender will be stuck in an infinite loop, trying to read this unflushed record and never reaching the stop state, blocking the whole shutdown sequence. This can be reproduced with the following script (this assumes `pgbench -i` was run to create pgbench_accounts and a running logical replication walsender): TRUNCATE pgbench_accounts; -- Completely fill the first heap page INSERT INTO pgbench_accounts SELECT *, *, *, '' FROM generate_series(0, 62); -- This should tag the page's metadata as full BEGIN; UPDATE pgbench_accounts SET bid=4 where aid=1; ROLLBACK; -- Force checkpoint so next change will be a FPW CHECKPOINT; -- Open an explicit transaction BEGIN; -- Select will do an opportunistic pruning, find nothing to prune but will still unset the page full flag, writing a FPI_FOR_HINT SELECT ctid, * FROM pgbench_accounts WHERE aid=2; Then shutdown the database with 'pg_ctl stop' with the transaction left opened. The shutdown will be stuck and the logical replication walsender will be stuck at 100% CPU. I've managed to reproduce this issue on 14 and the current HEAD. The attached (tentative) patch fixes the issue by flushing all records before signaling walsenders to stop. At that point, all backends should have been killed, so flushing leftover records felt like a correct approach. Regards, Anthonin Bonnefoy
v1-0001-Fix-stuck-shutdown-due-to-unflushed-records.patch
Description: Binary data
