Brendan Jurd wrote:
Hi hackers,
In the process of trying to unify the various text/cstring conversions
in the backend, I came across some stuff that seemed weird in
pg_convert().
>From src/backend/utils/mb/mbutils.c:345:
Datum
pg_convert(PG_FUNCTION_ARGS)
{
bytea *string = PG_GETARG_TEXT_P(0);
Is this a typo? Seems this should be PG_GETARG_BYTEA_P.
Moving on from that, the function takes the bytea argument and
converts it into a C string (using the exact the same technique as
textout, which is why I noticed it).
The documentation is very clear that bytea values "specifically allow
storing octets of value zero and other "non-printable" octets". That
being the case, is it sane to convert a bytea to a cstring at all?
The possibility of having valid nulls in the value renders the whole
point of a null-terminated character array ... well, null.
Before putting it into a cstring, the function does put the bytea
value through pg_verify_mbstr(), so basically the issue goes away if
we accept the premise that we will never allow a character encoding
where the null byte is valid. However, if we reject that premise
there's a problem.
pg_convert() does pass the length of the bytea along to
pg_do_encoding_conversion(), so either
a) the encoding functions properly respect length and ignore nulls in
the string, in which case the null at the end is worthless and we may
as well just operate on the VARDATA of the bytea, or
b) the encoding functions treat a null byte as the end of the string,
in which case they are broken w.r.t. to bytea input.
Please read the recent discussions about encoding issues. convert() now
returns a bytea precisely because we cannot be sure that the data
returned will be valid in the database encoding. The behaviour here is
entirely intentional. We have just closed every hole we are aware of
whereby data that isn't valid in the database encoding can enter the
database. We're not about to reopen them.
And yes, we will not accept an encoding with a null byte. We don't even
accept nulls in Unicode. If we do accept such an encoding then this
would be among the least of our problems, I suspect.
We can and possibly should change the GETARG call, but the varlena types
are structurally equivalent, so it's not a mortal sin being committed here.
cheers
andrew
---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?
http://archives.postgresql.org
