On 22.02.21 05:22, Andres Freund wrote:
Hi,
On 2021-02-19 15:53:32 +0100, Markus Wanner wrote:
However, more generally speaking, I suspect you are overthinking this. All
of the complexity arises because of the assumption that an output plugin
receiving and confirming a PREPARE may not be able to persist that first
phase of transaction application. Instead, you are trying to somehow
resurrect the transactional changes and the prepare at COMMIT PREPARED time
and decode it in a deferred way.
The output plugin should never persist anything.
Sure, sorry, I was sloppy in formulation. I meant the replica or client
that receives the data from the output plugin. Given it asked for
two-phase commits in the output plugin, it clearly is interested in the
PREPARE.
That's the job of the
client, not the output plugin. The output plugin simply doesn't have the
information to know whether the client received data and successfully
applied data or not.
Exactly. Therefore, it should not randomly reshuffle or reorder
PREPAREs until after other COMMITs.
The output plugin doesn't set / influence start_decoding_at (unless you
want to count just ERRORing out).
Yeah, same sloppiness, sorry.
With that line of thinking, the point in time (or in WAL) of the COMMIT
PREPARED does not matter at all to reason about the decoding of the PREPARE
operation. Instead, there are only exactly two cases to consider:
a) the PREPARE happened before the start_decoding_at LSN and must not be
decoded. (But the effects of the PREPARE must then be included in the
initial synchronization. If that's not supported, the output plugin should
not enable two-phase commit.)
I don't think that can be made work without disproportionate
complexity. Especially not in cases where we start to be CONSISTENT
based on pre-existing on-disk snapshots.
Well, the PREPARE to happen before the start_decoding_at LSN is a case
the output plugin needs to deal with. I pointed out why the current way
of dealing with it clearly is wrong.
What issues do you see with the approach I proposed?
Regards
Markus
