Re: [BUGS] COPY fails on 8.1 with invalid byte sequences in text
On 27/10/06, Thomas H. [EMAIL PROTECTED] wrote: FYI, prior to 8.2, there is another source of bad UTF8 byte sequences: when using tsearch2 on utf8 content in 8.2, tsearch2 was generating bad utf8 sequences. as tsearch2 does lowercase each char in the text its indexing, it did also do so with multibyte-characters... unfortunately taking each byte separately, so it seems. the unicode-representation of german umlauts (äöü) are some examples of charcodes, that where turned into invalid sequences. this data could be successfully pg_dump'ed, but not pg_restore'd. in 8.2, this looks fixed. to upgrade from 8.1.5 to 8.2b1 we had to remove all tsearch2 index data, dump the db, restore the db in 8.2 and recreate the indices. You need to initdb with utf8 and then install tsearch2 with utf8. Both need utf8. I had a similar problem. Perhaps your 8.1 postgres cluster wasn't utf8? - thomas - Original Message - From: Jeff Davis [EMAIL PROTECTED] To: pgsql-bugs@postgresql.org Sent: Saturday, October 28, 2006 12:38 AM Subject: Re: [BUGS] COPY fails on 8.1 with invalid byte sequences in text On Fri, 2006-10-27 at 14:42 -0700, Jeff Davis wrote: It seems to be essentially a data corruption issue if applications insert binary data in text fields using escape sequences. Shouldn't PostgreSQL reject an invalid UTF8 sequence in any text type? Another note: PostgreSQL rejects invalid UTF8 sequences in other contexts. For instance, if you use PQexecParams() and insert using type text and any format (text or binary), it will reject invalid sequences. It will of course allow anything to be sent when the type is bytea. Also, I thought I'd publish the workaround that I'm using. I created a function that seems to work for validating text data as being valid UTF8. CREATE OR REPLACE FUNCTION valid_utf8(TEXT) returns BOOLEAN LANGUAGE plperlu AS $valid_utf8$ use utf8; return utf8::decode($_[0]) ? 1 : 0; $valid_utf8$; I just add a check constraint on all of my text attributes in all of my tables. Not fun, but it works. Regards, Jeff Davis ---(end of broadcast)--- TIP 6: explain analyze is your friend ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org ---(end of broadcast)--- TIP 6: explain analyze is your friend
[BUGS] Drive ODBC
Preciso de um Drive ODBC para windows 98 Att,Henrique LavnisSetor de Informática - Visual Tintas[EMAIL PROTECTED](31) 3385-1365 (31) 8832-6601
Re: [BUGS] BUG #2712: could not fsync segment: Permission
I haven't reproduced this on my box. But if you can give me a patch to try I can build binaries for Thomas to test, if he can do testing but not building. Utterly untested ... BTW, why does pgrename have an #if to check either GetLastError() or errno, but pgunlink doesn't? Ok, I've built a .EXE with this patch. It's compiled without pretty much all other options, hope that still works :-) (Meaning no NLS, no kerberos, no SSL etc) Grab the exe from http://www.hagander.net/download/postgres_renamepatch.zip. Sorry about the delay. //Magnus ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [BUGS] BUG #2694: Memory allocation error when selecting array
Tom Lane wrote:
Joe Conway [EMAIL PROTECTED] writes:
Sorry for the slow response -- I'm at the airport just heading home from
a marathon 30 day business trip.
Yow. Hope you get some time off...
Yeah, I just took a week. Next week I'm back to work and the week after
that I'm back to Germany for a few...
On looking at the code, I notice that this somewhat-related case works:
regression=# select array[null::text[], null::text[]];
array
---
{}
(1 row)
The reason is that null inputs are just ignored in ExecEvalArray. So
one pretty simple patch would be to ignore zero-dimensional inputs too.
This would have implications for mixed inputs though: instead of
regression=# select array['{}'::text[], '{a,b,c}'::text[]];
ERROR: multidimensional arrays must have array expressions with matching
dimensions
you'd get behavior like
regression=# select array[null::text[], '{a,b,c}'::text[]];
array
---
{{a,b,c}}
(1 row)
Which of these seems more sane?
I'm not sure I love either. I would think both NULL and empty array
expressions should be disallowed in this scenario, i.e.:
regression=# select array['{}'::text[], '{a,b,c}'::text[]];
ERROR: multidimensional arrays must have array expressions with
matching dimensions
regression=# select array[NULL::text[], '{a,b,c}'::text[]];
ERROR: multidimensional arrays must have array expressions with
matching dimensions
In both cases you are trying to construct a multidimensional array with
inconsistent dimensions.
On the other hand, building an N-dimension array from entirely empty
array expressions should just produce an empty array, while using all
NULL expressions should produce an N-dim array full of NULLs.
But as I've opined before, all of this seems to me to be much cleaner if
arrays were always one-dimensional, and array elements could also be
nested arrays (per SQL 2003). If we said that the cardinality of the
nested array is an integral part of the datatype, then I think you would
have:
regression=# select array['{}'::text[], '{a,b,c}'::text[]];
ERROR: nested arrays must have array expressions with matching
dimensions
regression=# select array[NULL::text[], '{a,b,c}'::text[]];
array
---
{NULL, {a,b,c}}
(1 row)
So maybe this is the behavior we should shoot for now?
Joe
---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
Re: [BUGS] BUG #2712: could not fsync segment: Permission
thanks alot. using only standard features, so the missing bits shouldn't be a problem. will try to test later tonight or tomorrow. Grab the exe from http://www.hagander.net/download/postgres_renamepatch.zip. report will follow. regards, thomas - Original Message - From: Magnus Hagander [EMAIL PROTECTED] To: Tom Lane [EMAIL PROTECTED] Cc: Peter Brant [EMAIL PROTECTED]; Thomas H. [EMAIL PROTECTED]; pgsql-bugs@postgresql.org; Bruce Momjian [EMAIL PROTECTED] Sent: Sunday, October 29, 2006 6:10 PM Subject: RE: [BUGS] BUG #2712: could not fsync segment: Permission I haven't reproduced this on my box. But if you can give me a patch to try I can build binaries for Thomas to test, if he can do testing but not building. Utterly untested ... BTW, why does pgrename have an #if to check either GetLastError() or errno, but pgunlink doesn't? Ok, I've built a .EXE with this patch. It's compiled without pretty much all other options, hope that still works :-) (Meaning no NLS, no kerberos, no SSL etc) Grab the exe from http://www.hagander.net/download/postgres_renamepatch.zip. Sorry about the delay. //Magnus ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
