Hi, Tomas provided me with a machine where the problem was reproducible (Thanks again!). I first started to make sure a54e1f158 is unrelated - and indeed, the problem appears independently.
I've a few leads that I'm currently testing out. One observation I think might be crucial is that the problem, in Tomas' testcase with just VACUUM FULL of pg_class and INSERTs into another rel, appears to only happen if autovacuum is enabled. The crashes are always immediately after autovacuum did an analyze of the relation. What I see on the apply side, I think, that invalidation records for contents of pg_class and pg_class itself are intermixed. I'll continue looking into this. Greetings, Andres Freund