On Sun, Sep 13, 2020 at 10:25:01PM +0200, Brar Piening wrote:
> While implementing streaming replication client functionality for Npgsql
> I stumbled upon a minor documentation error at
> https://www.postgresql.org/docs/current/protocol-replication.html
>
> The "content" return value for the TIMELINE_HISTORYcommand is advertised
> as bytea while it is in fact raw ASCII bytes.
>
> How did I find out?
> Since the value I get doesn't start with a "\x" and contains ascii text,
> although I've bytea_outputset to hex, I first thought that the streaming
> replication protocol simply doesn't honor bytea_output, but then I
> realized that I also get unencoded tabs and newlines which wouldn't be
> possible if the value woud be passed through byteaout.
>
> This is certainly a minor problem since the timeline history file only
> contains generated strings that are ASCII-only, so just using the
> unencoded bytes is actually easier than decoding bytea.
> OTOH it did cost me a few hours (writing a bytea decoder and figuring
> out why it doesn't work by looking at varlena.c and proving the docs
> wrong) so I want to point this out here since it is possibly an
> unintended behavior or at least a documentation error.
> Also I'm wary of taking dependency on an undocumented implementation
> detail that could possibly change at any point.
I have looked at this. It seems SendTimeLineHistory() is sending raw
bytes from the history file, with no encoding conversion, and
ReceiveXlogStream() is receiving it, again assuming it is just plain
text. I am not sure we really have an SQL data type where we do this.
BYTEA doesn't do encoding conversion, but does backslash procesing, and
TEXT does encoding conversion.
I suppose we either have to document this as BYTEA with no backslash
processing, or TEXT with no encoding conversion --- I think I prefer the
later.
Attached is a patch to update the docs, and the data type passed by
SendTimeLineHistory(). Does this look safe to everyone?
--
Bruce Momjian <br...@momjian.us> https://momjian.us
EnterpriseDB https://enterprisedb.com
The usefulness of a cup is in its emptiness, Bruce Lee
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index f5e3318106..3cc30a480b 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -1877,11 +1877,11 @@ The commands accepted in replication mode are:
<varlistentry>
<term>
- <literal>content</literal> (<type>bytea</type>)
+ <literal>content</literal> (<type>text</type>)
</term>
<listitem>
<para>
- Contents of the timeline history file.
+ Contents of the timeline history file; no encoding conversion is performed.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index 7c9d1b67df..7db8975065 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -496,7 +496,7 @@ SendTimeLineHistory(TimeLineHistoryCmd *cmd)
pq_sendstring(&buf, "content"); /* col name */
pq_sendint32(&buf, 0); /* table oid */
pq_sendint16(&buf, 0); /* attnum */
- pq_sendint32(&buf, BYTEAOID); /* type oid */
+ pq_sendint32(&buf, TEXTOID); /* type oid */
pq_sendint16(&buf, -1); /* typlen */
pq_sendint32(&buf, 0); /* typmod */
pq_sendint16(&buf, 0); /* format code */
