Hi Andres, Hi all,

We found an event in the cluster I somehow missed.  The 6th of May, a sysadmin
operation moved the VM's storage to another datastore using svmotion.  The
storages were moved without shutting down the VMs nor PostgreSQL themselves.  In
this three node Patroni cluster, instances were all moved as standbys.  Which
means a swichover occured. Let's call the nodes: n21, n22, n23.

Here is a sum up of the events:

0. Node n23 is the current primary (since March)
1. 2026-05-06 11h30: n21 storage moved using svmotion without shutting it down
2. 2026-05-06 22h30: switchover.  n21 becomes primary, n23 a standby
3. 2026-05-07 11h30: n22 storage moved using svmotion without shutting it down
4. 2026-05-11 11h30: n23 storage moved using svmotion without shutting it down
5. 2026-05-13 19h00: a daily batch issues a lot of INSERT+INIT continuous
   blocks, including the 2547 affected ones.  Each block is filled with about 27
   xacts inserting only one row and being **all** immediately aborted.  There's
   at least 500 blocks before and after the affected zone having the same exact
   workload.  Only inserted data differ.  Xact id increase monotonically.
6. 2026-05-14 02h00: daily vacuum starts on the affected table.  Among usual
   other tasks, all blocks filled with only dead rows are "CLEAN remxid 0" and
   set "VISIBLE cutoff xid 0".  All? no.  Not the 2547 affected ones. For some
   reason, VACUUM ignored them.
7. between 2026-06-02 21h00 and 2026-06-03 21h00: an old row has been cleaned
   somewhere, moving the xid horizon from 3,498,165,236 to 3,596,946,949 and
   removing all pg_xact segments from 0D08 to 0D65
8. 2026-06-04 02h20: daily vacuum analyze raise the first error about missing
   file "pg_xact/0D44".  The xact status of the 2547 affected blocks were all in
   this file.
9. 2026-06-12: the customer contact us about this issue
10. 2026-06-15: pg_check_frozen() exposed all the affected blocks

Note that step 8 was repeating every day until we fixed the production (see
first email of the thread).  The error was always raised during the ANALYZE
phase. So it _seems_ (but I can't prove it) VACUUM never cleaned these pages at
all between May 13th and June 12th.

I can accept the n21 datastore hot move hadn't been really atomic, seeding
some incoherence here and there.  We already discover a corruption after a
vmotion (not svmotion) with another customer.  In short, standby n21 might been
promoted with a silent logical corruption.  However, I can not be 100% sure
it would be the root cause of the problem here.  Especially since I fail to
find what would be the corruption.

Indeed, the affected data have been written **after** this switchover.  What
kind of corruption would have trigger this behavior? The Visibility Map should
have been updated during the writes… shouldn't they? But according to
pg_check_frozen(), it wasn't!

What could forbid the vacuum to clean this specific region for such a long time
if it's not the visibility map?

Looking at the affected data we collected in june, they all have flags:

  HEAP_HASNULL, HEAP_HASVARWIDTH, HEAP_XMAX_INVALID

Some have an additional HEAP_XMIN_INVALID that could have been set when some
readings or a vacuum analyze land here.  But nothing would keep vacuum away
from them...

Does this story ring any bell to someone?

I build some graphical map of the relation to visualize various actions on each
blocks, but "decent" png are about 5MB and I don't want to spam everyone on this
list with them.  If you're interested, I can share them offlist.

Regards,


Reply via email to