On 2026-06-11 11:27, torikoshia wrote:
On this point, I have started discussing with the Debezium
community, where I originally encountered this issue. The main
question is whether a change in PostgreSQL's behavior would be
welcomed, or whether Debezium instead treats logical messages
differently from other decoded records.
After asking the Debezium community, I learned that they have no
particular preference as to how PostgreSQL addresses this issue:
-- https://github.com/debezium/dbz/issues/2058#issuecomment-4774845738
Based on the discussions so far, my understanding is that, regardless
of whether PostgreSQL implements this fix in a minor or major release,
Debezium will accommodate either approach and therefore does not have a
preference for one over the other, or for any alternative solution.
I think there would be roughly three ways to address the issue:
1) Preserve the current behavior, that is, continue returning endptr
only when decoding pg_logical_emit_message(), and update the
documentation accordingly.
2) Change the behavior so that decoding pg_logical_emit_message()
returns startptr, as is the case for DML records.
2.1) Make the change in minor releases.
2.2) Make the change in a major release, while updating the
documentation for earlier major versions to describe their existing
behavior.
If there are requirements from other consumers of the logical
decoding, I think those should of course be taken into account.
Personally, however, I do not think we should choose option 1.
As I wrote previously, the current behavior can be a pitfall
that may lead to data loss when implementing a logical
decoding consumer:
```
Regarding the third point, I believe this distinction is
important for consumers of logical decoding. Typically, a
consumer records the LSN up to which the decoded result has been
successfully processed and uses that LSN as the restart point
after a crash or other failure. From that perspective, whether
startptr or endptr is returned matters.
```
As for options 2.1 and 2.2, given the potential impact on consumers,
changing the behavior only in a major release, as in option 2.2,
initially seems like a good approach.
However, the discussion so far has not identified any particular
reason why endptr should be returned, and the current behavior
also appears to differ from the documentation. This therefore
seems to me more like a bug. If we regard it as such, fixing it
in minor releases, as in option 2.1, would seem to be the most
straightforward approach.
If there are consumers that are already aware of this issue and
have implemented special handling for pg_logical_emit_message(),
they would need to remove or adjust that handling, which could
be inconvenient.
That said, this behavior cannot readily be inferred from the
documentation. It is also a relatively niche issue that is likely
to be noticed only after it causes a problem such as data loss.
Moreover, once someone identifies the cause, it seems like the kind
of issue they would be inclined to report to the PostgreSQL
community, but it seems that we haven't received such reports.
For these reasons, I suspect there are few, if any, products that
currently implement such a workaround.
For consumers that are affected by this issue but are not yet aware
of it, fixing it in minor releases would have the advantage that
simply upgrading PostgreSQL to a newer minor release would resolve
the problem without adding special handling for
pg_logical_emit_message() to their products.
I imagine most consumers would gain this advantage.
For these reasons, I am beginning to think that fixing this in minor
releases may be the preferable approach.
What do you think?
--
Thanks,
--
Atsushi Torikoshi
Seconded from NTT DATA CORPORATION to SRA OSS K.K.
From 265830febb6e22b92dc181b97d773b0ab1b2e338 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi <[email protected]>
Date: Wed, 15 Jul 2026 18:34:21 +0900
Subject: [PATCH v2] Fix LSN of logical decoding message
Report startptr, rather than endptr, as the LSN of a decoded
logical message. The logical replication protocol documentation
describes this field as the LSN of the logical decoding message, but
the current implementation reports the end of its WAL record.
This distinction matters to logical decoding consumers. A consumer
typically records the LSN up to which decoded output has been
successfully processed and uses it as the restart point after a crash
or other failure. PostgreSQL reports startptr when decoding DML
records, but reports endptr only for decoding
pg_logical_emit_message(). Consumers that handle logical messages
in the same way as other decoded records can therefore store an LSN
past the message it has just processed and restart from that later
position after a failure, potentially resulting in data loss.
The regression test is adjusted accordingly. The test emits only a
single non-transactional logical message, which is decoded with
xid = 0, so filtering on xid = 0 is sufficient to identify the
message without matching its LSN.
---
src/backend/replication/logical/decode.c | 2 +-
src/test/subscription/t/020_messages.pl | 4 ++--
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index c944be4ac83..09efc15e454 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -668,7 +668,7 @@ logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
if (!message->transactional)
snapshot = SnapBuildGetOrBuildSnapshot(builder);
- ReorderBufferQueueMessage(ctx->reorder, xid, snapshot, buf->endptr,
+ ReorderBufferQueueMessage(ctx->reorder, xid, snapshot, buf->origptr,
message->transactional,
message->message, /* first part of message is
* prefix */
diff --git a/src/test/subscription/t/020_messages.pl b/src/test/subscription/t/020_messages.pl
index ac518849156..bdedd5ff812 100644
--- a/src/test/subscription/t/020_messages.pl
+++ b/src/test/subscription/t/020_messages.pl
@@ -94,7 +94,7 @@ is($result, qq(),
$node_publisher->safe_psql('postgres', "INSERT INTO tab_test VALUES (1)");
-my $message_lsn = $node_publisher->safe_psql('postgres',
+$node_publisher->safe_psql('postgres',
"SELECT pg_logical_emit_message(false, 'pgoutput', 'a non-transactional message')"
);
@@ -107,7 +107,7 @@ $result = $node_publisher->safe_psql(
'proto_version', '1',
'publication_names', 'tap_pub',
'messages', 'true')
- WHERE lsn = '$message_lsn' AND xid = 0
+ WHERE xid = 0
));
is($result, qq(77|0), 'non-transactional message on slot is M');
--
2.48.1
