Re: [BUGS] COPY fails on 8.1 with invalid byte sequences in text

2006-11-03 Thread Dennis Bjorklund
Tom Lane skrev: Is this not a bug? I don't actually see that it is. The documentation is perfectly clear on the point: (It is your responsibility that the byte sequences you create are valid characters in the server character set encoding.) (This is in 4.1.2.1. String

Re: [BUGS] COPY fails on 8.1 with invalid byte sequences in text

2006-11-03 Thread Jeff Davis
On Tue, 2006-10-31 at 23:18 -0500, Tom Lane wrote: Jeff Davis [EMAIL PROTECTED] writes: Is this not a bug? I don't actually see that it is. The documentation is perfectly clear on the point: (It is your responsibility that the byte sequences you create are valid characters

Re: [BUGS] COPY fails on 8.1 with invalid byte sequences in text

2006-10-31 Thread Jeff Davis
On Fri, 2006-10-27 at 14:42 -0700, Jeff Davis wrote: You can insert invalid UTF8 bytes sequences into a TEXT type on an 8.1 installation by doing something like: I created a patch that appears to fix the problem, and does not appear to break anything else. Is this acceptable? Regards,

Re: [BUGS] COPY fails on 8.1 with invalid byte sequences in text

2006-10-31 Thread Tom Lane
Jeff Davis [EMAIL PROTECTED] writes: I created a patch that appears to fix the problem, and does not appear to break anything else. ... except maybe bytea ... regards, tom lane ---(end of broadcast)--- TIP 9: In versions

Re: [BUGS] COPY fails on 8.1 with invalid byte sequences in text

2006-10-31 Thread Jeff Davis
On Tue, 2006-10-31 at 16:13 -0500, Tom Lane wrote: Jeff Davis [EMAIL PROTECTED] writes: I created a patch that appears to fix the problem, and does not appear to break anything else. ... except maybe bytea ... Ok. So then it seems that the only possible places to fix it are in textin

Re: [BUGS] COPY fails on 8.1 with invalid byte sequences in text

2006-10-31 Thread Tom Lane
Jeff Davis [EMAIL PROTECTED] writes: Is this not a bug? I don't actually see that it is. The documentation is perfectly clear on the point: (It is your responsibility that the byte sequences you create are valid characters in the server character set encoding.) (This is in

Re: [BUGS] COPY fails on 8.1 with invalid byte sequences in text

2006-10-29 Thread Birju Prajapati
On 27/10/06, Thomas H. [EMAIL PROTECTED] wrote: FYI, prior to 8.2, there is another source of bad UTF8 byte sequences: when using tsearch2 on utf8 content in 8.2, tsearch2 was generating bad utf8 sequences. as tsearch2 does lowercase each char in the text its indexing, it did also do so with

[BUGS] COPY fails on 8.1 with invalid byte sequences in text types

2006-10-27 Thread Jeff Davis
You can insert invalid UTF8 bytes sequences into a TEXT type on an 8.1 installation by doing something like: INSERT INTO foo(t) VALUES('\xFF'); Then, you can do a: COPY foo TO '/some/file'; but if you try to do a: COPY foo FROM '/some/file'; That will fail because /some/file contains invalid

Re: [BUGS] COPY fails on 8.1 with invalid byte sequences in text

2006-10-27 Thread Jeff Davis
On Fri, 2006-10-27 at 14:42 -0700, Jeff Davis wrote: It seems to be essentially a data corruption issue if applications insert binary data in text fields using escape sequences. Shouldn't PostgreSQL reject an invalid UTF8 sequence in any text type? Another note: PostgreSQL rejects invalid

Re: [BUGS] COPY fails on 8.1 with invalid byte sequences in text

2006-10-27 Thread Thomas H.
FYI, prior to 8.2, there is another source of bad UTF8 byte sequences: when using tsearch2 on utf8 content in 8.2, tsearch2 was generating bad utf8 sequences. as tsearch2 does lowercase each char in the text its indexing, it did also do so with multibyte-characters... unfortunately taking