[GENERAL] encoding confusion with \copy command

2014-09-17 Thread Martin Waite
Hi, I have a postgresql 7.4 server and client on Centos 6.4. The database server is using UTF-8 encoding. I have been exploring the use of the \copy command for importing CSV data generated by SQL Server 2008. SQL Server 2008 export tool does not escape quotes that are in the content of

Re: [GENERAL] encoding confusion with \copy command

2014-09-17 Thread Martin Waite
Hi Adrian, I apologise - I meant 9.4 regards, Martin On 17 September 2014 14:35, Adrian Klaver adrian.kla...@aklaver.com wrote: On 09/17/2014 03:03 AM, Martin Waite wrote: Hi, I have a postgresql 7.4 server and client on Centos 6.4. The database server is using UTF-8 encoding. First

Re: [GENERAL] encoding confusion with \copy command

2014-09-17 Thread Adrian Klaver
On 09/17/2014 03:03 AM, Martin Waite wrote: Hi, I have a postgresql 7.4 server and client on Centos 6.4. The database server is using UTF-8 encoding. First I think we need to establish what version of Postgres you using. Are you really using 7.4? I have been exploring the use of the

Re: [GENERAL] encoding confusion with \copy command

2014-09-17 Thread Adrian Klaver
On 09/17/2014 06:48 AM, Martin Waite wrote: Hi Adrian, I apologise - I meant 9.4 Looks like you will need an intermediate step. A quick search found the below which might offer a solution: http://www.excel-sql-server.com/sql-server-export-to-excel-using-bcp-sqlcmd-csv.htm FYI the good

Re: [GENERAL] encoding confusion with \copy command

2014-09-17 Thread Martin Waite
You are right that I need an intermediate step. I will probably use a CSV parser that is liberal in what it accepts, but writes out strict CSV data suitable for postgres. Thanks for the help. On 17 September 2014 15:40, Adrian Klaver adrian.kla...@aklaver.com wrote: On 09/17/2014 06:48 AM,

Re: [GENERAL] encoding confusion with \copy command

2014-09-17 Thread John DeSoi
On Sep 17, 2014, at 11:52 AM, Martin Waite waite@gmail.com wrote: You are right that I need an intermediate step. I will probably use a CSV parser that is liberal in what it accepts, but writes out strict CSV data suitable for postgres. If you find such a utility, please share. My

Re: [GENERAL] encoding confusion with \copy command

2014-09-17 Thread Adrian Klaver
On 09/17/2014 06:08 PM, John DeSoi wrote: On Sep 17, 2014, at 11:52 AM, Martin Waite waite@gmail.com wrote: You are right that I need an intermediate step. I will probably use a CSV parser that is liberal in what it accepts, but writes out strict CSV data suitable for postgres. If

Re: [GENERAL] encoding confusion with \copy command

2014-09-17 Thread John R Pierce
On 9/17/2014 6:08 PM, John DeSoi wrote: On Sep 17, 2014, at 11:52 AM, Martin Waitewaite@gmail.com wrote: You are right that I need an intermediate step. I will probably use a CSV parser that is liberal in what it accepts, but writes out strict CSV data suitable for postgres. If you

Re: [GENERAL] ENCODING = 'LATIN1' LC_CTYPE?

2014-08-08 Thread Albe Laurenz
Sylvia Preuß wrote: I’d like to create a database with ENCODING LATIN1 . CREATE DATABASE z_latin1 WITH OWNER = admin ENCODING = 'LATIN1' TABLESPACE = pg_default LC_COLLATE = 'German_Germany.1252' LC_CTYPE = 'German_Germany.1252' CONNECTION LIMIT = -1;

[GENERAL] ENCODING = 'LATIN1' LC_CTYPE?

2014-08-07 Thread Preuß , Sylvia
Dear list, I'd like to create a database with ENCODING LATIN1 . CREATE DATABASE z_latin1 WITH OWNER = admin ENCODING = 'LATIN1' TABLESPACE = pg_default LC_COLLATE = 'German_Germany.1252' LC_CTYPE = 'German_Germany.1252' CONNECTION LIMIT = -1; FEHLER:

Re: [GENERAL] encoding and LC_COLLATE

2011-11-15 Thread LPlateAndy
Hi Mark (and Adrian), As as update i've now found the same data fails on my postgres 8 which doesn't seem to have the LC_COLLATE etc setting and is just UTF-8 so i guess there is possibly just something about the way the data is getting passed in. This is the error message from postgres

Re: [GENERAL] encoding and LC_COLLATE

2011-11-15 Thread LPlateAndy
Hi Adrian/Mark Thanks again for your help, i have now got the load working by setting the encoding to WIN1252. I had been assuming i was setting it to UTF8 SET CLIENT_ENCODING TO 'WIN1252'; Andy -- View this message in context:

[GENERAL] encoding and LC_COLLATE

2011-11-14 Thread LPlateAndy
Hi, I set up my postgres 9.0 install 6 months ago and generally everything is fine but a recent data load with an e acute character failed which an unsupported message which surprised me as we're using UTF-8. However, i can now see that the listing for the database set up show a restriction

Re: [GENERAL] encoding and LC_COLLATE

2011-11-14 Thread LPlateAndy
Hi, In response to my own question i have now read the following: http://stackoverflow.com/questions/6579621/lc-collate-and-lc-ctype-suport-for-utf-8-in-postgresql It seems to show that apart from a complete reinstall i should just use a Locale of C (as i'm on Windows and POSIX wont work).

Re: [GENERAL] encoding and LC_COLLATE

2011-11-14 Thread Adrian Klaver
On Monday, November 14, 2011 3:03:32 am LPlateAndy wrote: Hi, I set up my postgres 9.0 install 6 months ago and generally everything is fine but a recent data load with an e acute character failed which an unsupported message which surprised me as we're using UTF-8. However, i can now

Re: [GENERAL] encoding and LC_COLLATE

2011-11-14 Thread LPlateAndy
Hi Adrian, You're right, i'm trying to get the copy command to put a load of data into a table. It's now working fine except for any instances with an e acute I tried putting SET CLIENT_ENCODING TO 'UTF-8'; but still got the error. I guess that just because i'm verifying what's incoming,

Re: [GENERAL] encoding and LC_COLLATE

2011-11-14 Thread Adrian Klaver
On Mon, Nov 14, 2011 at 8:25 AM, LPlateAndy a...@centremaps.co.uk wrote: Hi Adrian, ** ** You’re right, i’m trying to get the copy command to put a load of data into a table. It’s now working fine except for any instances with an e acute ** ** I tried putting “ SET

Re: [GENERAL] encoding and LC_COLLATE

2011-11-14 Thread Mark Watson
De : pgsql-general-ow...@postgresql.org [mailto:pgsql-general-ow...@postgresql.org] De la part de Adrian Klaver Envoyé : 14 novembre 2011 13:03 ... Second is the data coming in actually UTF8 or some other encoding? ...  Hi Andy, I have to agree with Adrian in that the data may be coming in

[GENERAL] Encoding Conversion

2011-02-03 Thread Wilton Wonrath
Hi, When I´m doing a conversion PostgreSQL returns SCAPES to me besides the desire caracter. Why ? Using Postgresql 8.2: SELECT CONVERT('Rejeição: Duplicidade de NF-e, com diferença na Chave de Acesso [35110100608804000178550010001009471840996034]','UTF8','LATIN1') Result: Rejeição:

Re: [GENERAL] Encoding Conversion

2011-02-03 Thread Tom Lane
Wilton Wonrath wrwonr...@yahoo.com.br writes: When I´m doing a conversion PostgreSQL returns SCAPES to me besides the desire caracter. Why ? Somewhere between 8.2 and 9.0 convert() was redefined to take and return bytea, not text, to reflect the fact that the data it deals in isn't

[GENERAL] Encoding change question...

2010-08-16 Thread Karl Denninger
So I have myself a nice pickle here. I've got a database which was originally created with SQL_ASCII for the encoding (anything goes text fields) Unfortunately, I have a bunch of data that was encoded in UTF-8 that's in an RSS feed that I need to load into said database. iconv barfs all

Re: [GENERAL] Encoding change question...

2010-08-16 Thread Peter C. Lai
The doublequotes isn't UTF8 it's people copying and pasting from Microsoft stuff, which is WIN-1252. So try to use that with iconv instead of utf8 On 2010-08-16 12:40:03PM -0500, Karl Denninger wrote: So I have myself a nice pickle here. I've got a database which was originally created

Re: [GENERAL] Encoding change question...

2010-08-16 Thread Karl Denninger
Peter C. Lai wrote: The doublequotes isn't UTF8 it's people copying and pasting from Microsoft stuff, which is WIN-1252. So try to use that with iconv instead of utf8 On 2010-08-16 12:40:03PM -0500, Karl Denninger wrote: So I have myself a nice pickle here. I've got a database which

[GENERAL] Encoding using the Frontend/Backend Protocol TCP/IP

2009-11-19 Thread Raimon Fernandez
Hello, I'm trying to send some strings that have chars outside from standar ascii, like çñàèó Once I'm connected, the client and server both uses UT8Encoding. And I'm sending all the strings encoded in UTF8. At least the received ones are working, as I get the text exactly as it is, with

Re: [GENERAL] Encoding using the Frontend/Backend Protocol TCP/IP

2009-11-19 Thread Kovalevski Andrei
Hi could it be that you have errors in your UTF8 string? For example you might use UTF16 encoding, it can explain why some characters force errors but others are not. Can you post here the string and its' encoded version? Raimon Fernandez wrote: Hello, I'm trying to send some strings

Re: [GENERAL] Encoding using the Frontend/Backend Protocol TCP/IP

2009-11-19 Thread Raimon Fernandez
On 19/11/2009, at 17:27, Kovalevski Andrei wrote: Hi could it be that you have errors in your UTF8 string? For example you might use UTF16 encoding, it can explain why some characters force errors but others are not. It only happens with values like àéïçñ I think UTF8 can handle this

Re: [GENERAL] Encoding using the Frontend/Backend Protocol TCP/IP

2009-11-19 Thread Raimon Fernandez
On 19/11/2009, at 18:13, Raimon Fernandez wrote: On 19/11/2009, at 17:27, Kovalevski Andrei wrote: Hi could it be that you have errors in your UTF8 string? For example you might use UTF16 encoding, it can explain why some characters force errors but others are not. It only

Re: [GENERAL] Encoding using the Frontend/Backend Protocol TCP/IP

2009-11-19 Thread Kovalevski Andrei
Hi, the string is ok, but the problem is inside the message. The length of the message is incorrect: your message: 5100*46*557064617465207472616E73616374696F6E7320736574206465736372697074696F6E3D27546573742056616C75657364C387272077686572652069643D313133 it should be:

Re: [GENERAL] Encoding using the Frontend/Backend Protocol TCP/IP

2009-11-19 Thread Raimon Fernandez
On 19/11/2009, at 21:21, Kovalevski Andrei wrote: Hi, the string is ok, but the problem is inside the message. The length of the message is incorrect: your message:

[GENERAL] Encoding question when dumping/restoring databases for upgrade

2009-08-13 Thread arsi
Hello, I am sitting on version 7.4.x and am going to upgrade to version 8.3.x. From all I can read I should have no problem with actual format of the pgdump file (for actual dumping and restoring purposes) but I am having problems with encoding (which I was fairly sure I would). I have

Re: [GENERAL] Encoding question when dumping/restoring databases for upgrade

2009-08-13 Thread Tom Lane
a...@archie.netg.se writes: I am sitting on version 7.4.x and am going to upgrade to version 8.3.x. From all I can read I should have no problem with actual format of the pgdump file (for actual dumping and restoring purposes) but I am having problems with encoding (which I was fairly sure

Re: [GENERAL] encoding of PostgreSQL messages

2009-03-05 Thread Hiroshi Saito
Hi. Karsten-san. Yeah, It was a problem unsolvable by the driver to relay. although perseverance keeping without giving up! -- arigatougozaimasu:-) Regards, Hiroshi Saito - Original Message - From: Karsten Hilbert karsten.hilb...@gmx.net On Tue, Mar 03, 2009 at 12:35:37AM

Re: [GENERAL] encoding of PostgreSQL messages

2009-03-04 Thread Karsten Hilbert
On Tue, Mar 03, 2009 at 12:35:37AM +0900, Hiroshi Saito wrote: Sorry very late reaction. I desire problem solution. So do I :-) Ganbatte ! Karsten -- GPG key ID E4071346 @ wwwkeys.pgp.net E167 67FD A291 2BEA 73BD 4537 78B9 A9F9 E407 1346 -- Sent via pgsql-general mailing list

Re: [GENERAL] encoding of PostgreSQL messages

2009-03-02 Thread Hiroshi Saito
Hi. Sorry very late reaction. I desire problem solution.Therefore, one evidence I tried jdbc program. http://winpg.jp/~saito/pg_work/LC_MESSAGE_CHECK/connect_problem/jdbctestx.java C:\home\HIROSHIjava jdbctestx org.postgresql.util.PSQLException: FATAL: ???[??postgres?? at

Re: [GENERAL] encoding of PostgreSQL messages

2009-02-12 Thread Hiroshi Inoue
Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: Tom Lane wrote: Reflecting on the bigger picture ... I would imagine that the vast majority of existing applications depend on client_encoding settings that come from postgresql.conf, ALTER USER SET, ALTER DATABASE SET, or just the

Re: [GENERAL] encoding of PostgreSQL messages

2009-02-12 Thread Karsten Hilbert
On Thu, Feb 12, 2009 at 10:28:38PM +0900, Hiroshi Inoue wrote: Reflecting on the bigger picture ... I would imagine that the vast majority of existing applications depend on client_encoding settings that come from postgresql.conf, ALTER USER SET, ALTER DATABASE SET, or just the default

Re: [GENERAL] encoding of PostgreSQL messages

2009-02-12 Thread Karsten Hilbert
On Wed, Feb 11, 2009 at 02:20:47PM +0900, Hiroshi Inoue wrote: Tom Lane wrote: Hiroshi Inoue in...@tpf.co.jp writes: I'm thinking of the following steps in the backend code. 1.Set LC_MESSAGES to C until the client_encoding is determined. I have tried that but it didn't work out for

Re: [GENERAL] encoding of PostgreSQL messages

2009-02-11 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: Tom Lane wrote: Reflecting on the bigger picture ... I would imagine that the vast majority of existing applications depend on client_encoding settings that come from postgresql.conf, ALTER USER SET, ALTER DATABASE SET, or just the default (==

Re: [GENERAL] encoding of PostgreSQL messages

2009-02-11 Thread Peter Eisentraut
On Wednesday 11 February 2009 18:00:31 Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: Tom Lane wrote: Reflecting on the bigger picture ... I would imagine that the vast majority of existing applications depend on client_encoding settings that come from postgresql.conf, ALTER

Re: [GENERAL] encoding of PostgreSQL messages

2009-02-10 Thread Hiroshi Inoue
Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: Bruce Momjian wrote: Can someone comment on this? I think we have discussed more proper solutions earlier in this thread. IMO the best approach would be for the client to include the client encoding in the startup package.

Re: [GENERAL] encoding of PostgreSQL messages

2009-02-10 Thread Tom Lane
Hiroshi Inoue in...@tpf.co.jp writes: I'm thinking of the following steps in the backend code. 1.Set LC_MESSAGES to C until the client_encoding is determined. 2.When a client_encoding is specifed in the startup message, bind the corrsponding codeset to the textdomain and set

Re: [GENERAL] encoding of PostgreSQL messages

2009-02-10 Thread Hiroshi Inoue
Tom Lane wrote: Hiroshi Inoue in...@tpf.co.jp writes: I'm thinking of the following steps in the backend code. 1.Set LC_MESSAGES to C until the client_encoding is determined. 2.When a client_encoding is specifed in the startup message, bind the corrsponding codeset to the textdomain

Re: [GENERAL] encoding of PostgreSQL messages

2009-02-10 Thread Peter Eisentraut
Tom Lane wrote: Reflecting on the bigger picture ... I would imagine that the vast majority of existing applications depend on client_encoding settings that come from postgresql.conf, ALTER USER SET, ALTER DATABASE SET, or just the default (== database encoding). I don't think a solution that

Re: [GENERAL] encoding of PostgreSQL messages

2009-02-10 Thread Peter Eisentraut
Tom Lane wrote: I believe the only real fix is to guarantee that messages are sent as untranslated ASCII until we have sent an encoding indicator at the end of the startup sequence. Which has its own pretty clear downside: no more translation of authorization failures. We should process the

Re: [GENERAL] encoding of PostgreSQL messages

2009-02-08 Thread Hiroshi Inoue
Peter Eisentraut wrote: Bruce Momjian wrote: Can someone comment on this? Looks like a horrible hack to me. Recoding stuff to the client encoding in the server outside the existing recoding mechanism looks pretty evil to me. Plus, it does not address the problem of what happens to

Re: [GENERAL] encoding of PostgreSQL messages

2009-02-08 Thread Peter Eisentraut
Bruce Momjian wrote: Can someone comment on this? Looks like a horrible hack to me. Recoding stuff to the client encoding in the server outside the existing recoding mechanism looks pretty evil to me. Plus, it does not address the problem of what happens to messages sent before this, it

Re: [GENERAL] encoding of PostgreSQL messages

2009-02-08 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: Bruce Momjian wrote: Can someone comment on this? Looks like a horrible hack to me. Recoding stuff to the client encoding in the server outside the existing recoding mechanism looks pretty evil to me. Plus, it does not address the problem of what

Re: [GENERAL] encoding of PostgreSQL messages

2009-02-08 Thread Karsten Hilbert
On Sun, Feb 08, 2009 at 10:38:16AM -0500, Tom Lane wrote: I believe the only real fix is to guarantee that messages are sent as untranslated ASCII until we have sent an encoding indicator at the end of the startup sequence. Which has its own pretty clear downside: no more translation of

Re: [GENERAL] encoding of PostgreSQL messages

2009-02-08 Thread Hiroshi Inoue
Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: Bruce Momjian wrote: Can someone comment on this? Looks like a horrible hack to me. Recoding stuff to the client encoding in the server outside the existing recoding mechanism looks pretty evil to me. Plus, it does not address

Re: [GENERAL] encoding of PostgreSQL messages

2009-02-07 Thread Bruce Momjian
Can someone comment on this? --- Hiroshi Inoue wrote: Hi, This topic seems to be related to the bug report [ODBC] Localized error messages, wrong charset . Bruce Momjian wrote: Added to TODO: Improve

Re: [GENERAL] encoding of PostgreSQL messages

2009-01-30 Thread Hiroshi Inoue
Hi, This topic seems to be related to the bug report [ODBC] Localized error messages, wrong charset . Bruce Momjian wrote: Added to TODO: Improve encoding of connection startup messages sent to the client Currently some authentication error messages are sent in

[GENERAL] Encoding problem using pg_dumpall

2009-01-29 Thread Moshe Ben-Shoham
Hi, I have a database with encoding UTF-8 installed on Windows, and I try to dump it using pg_dumpall, on the machine on which the database is installed. I get the following error message: C:\Program Files\PostgreSQL\8.3\binpg_dumpall -U admint c:\temp\dbdump.sql pg_dump: SQL command

Re: [GENERAL] Encoding problem using pg_dumpall

2009-01-29 Thread Tom Lane
Moshe Ben-Shoham mos...@nexperience.com writes: C:\Program Files\PostgreSQL\8.3\binpg_dumpall -U admint c:\temp\dbdump.sql pg_dump: SQL command failed pg_dump: Error message from server: ERROR: character 0xd595 of encoding UTF8 has no equivalent in WIN1252 Apparently you have WIN1252 set

Re: [GENERAL] Encoding problem using pg_dumpall

2009-01-29 Thread Magnus Hagander
Tom Lane wrote: Moshe Ben-Shoham mos...@nexperience.com writes: C:\Program Files\PostgreSQL\8.3\binpg_dumpall -U admint c:\temp\dbdump.sql pg_dump: SQL command failed pg_dump: Error message from server: ERROR: character 0xd595 of encoding UTF8 has no equivalent in WIN1252 Apparently you

Re: [GENERAL] Encoding problem using pg_dumpall

2009-01-29 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: Tom Lane wrote: (Hmm, actually it looks like pg_dumpall hasn't got a -E switch, which seems like an oversight. So you need to fix your locale, or else use pg_dump directly.) IIRC, you can't set the windows console to be UTF8. Ugh. That seems to

Re: [GENERAL] Encoding problem using pg_dumpall

2009-01-29 Thread Magnus Hagander
Tom Lane wrote: Magnus Hagander mag...@hagander.net writes: Tom Lane wrote: (Hmm, actually it looks like pg_dumpall hasn't got a -E switch, which seems like an oversight. So you need to fix your locale, or else use pg_dump directly.) IIRC, you can't set the windows console to be UTF8.

Re: [GENERAL] encoding of PostgreSQL messages

2009-01-21 Thread Bruce Momjian
On Thu, Jan 01, 2009 at 08:33:56PM +0200, Peter Eisentraut wrote: Subject: Re: [GENERAL] encoding of PostgreSQL messages User-Agent: KMail/1.9.9 On Wednesday 31 December 2008 20:23:47 Tom Lane wrote: The proper fix is probably to include the client encoding in the connection startup

Re: [GENERAL] encoding of PostgreSQL messages

2009-01-07 Thread Karsten Hilbert
English until server_encoding can be retrieved by the client by current means. Thanks, Karsten On Thu, Jan 01, 2009 at 08:33:56PM +0200, Peter Eisentraut wrote: Subject: Re: [GENERAL] encoding of PostgreSQL messages User-Agent: KMail/1.9.9 On Wednesday 31 December 2008 20:23:47 Tom Lane wrote

Re: [GENERAL] encoding of PostgreSQL messages

2009-01-01 Thread Peter Eisentraut
On Wednesday 31 December 2008 20:23:47 Tom Lane wrote: The proper fix is probably to include the client encoding in the connection startup message. What of errors occurring before such an option could be applied? Connection errors are handled by the client, which knows the client encoding.

Re: [GENERAL] encoding of PostgreSQL messages

2008-12-31 Thread Karsten Hilbert
On Mon, Dec 29, 2008 at 09:07:14AM -0300, Alvaro Herrera wrote: That would not quite be enough -- I am talking about messages reported *during* auth, say FATAL: password authentication failed for user postgres or fe_sendauth: no password supplied both of which, in

Re: [GENERAL] encoding of PostgreSQL messages

2008-12-31 Thread Tom Lane
Karsten Hilbert karsten.hilb...@gmx.net writes: On Mon, Dec 29, 2008 at 09:07:14AM -0300, Alvaro Herrera wrote: And I'm now wondering if we should delay initializing the translation stuff until after client_encoding has been reported. Or else - just don't pass those messages through gettext

Re: [GENERAL] encoding of PostgreSQL messages

2008-12-31 Thread Karsten Hilbert
Karsten Hilbert karsten.hilb...@gmx.net writes: On Mon, Dec 29, 2008 at 09:07:14AM -0300, Alvaro Herrera wrote: And I'm now wondering if we should delay initializing the translation stuff until after client_encoding has been reported. Or else - just don't pass those messages through

Re: [GENERAL] encoding of PostgreSQL messages

2008-12-31 Thread Peter Eisentraut
On Wednesday 31 December 2008 18:57:29 Karsten Hilbert wrote: The solution is to find the right layer to take control of the encoding but this is eventually only possible if the encoding is *known*. Thus the plea for 7-bit-ascii English by default until the encoding *can* be known. Going to

Re: [GENERAL] encoding of PostgreSQL messages

2008-12-31 Thread Karsten Hilbert
On Wednesday 31 December 2008 18:57:29 Karsten Hilbert wrote: The solution is to find the right layer to take control of the encoding but this is eventually only possible if the encoding is *known*. Thus the plea for 7-bit-ascii English by default until the encoding *can* be known.

Re: [GENERAL] encoding of PostgreSQL messages

2008-12-31 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On Wednesday 31 December 2008 18:57:29 Karsten Hilbert wrote: The solution is to find the right layer to take control of the encoding but this is eventually only possible if the encoding is *known*. Thus the plea for 7-bit-ascii English by default until

Re: [GENERAL] encoding of PostgreSQL messages

2008-12-31 Thread Karsten Hilbert
The proper fix is probably to include the client encoding in the connection startup message. What of errors occurring before such an option could be applied? I think that ultimately it's necessary to accept that there will be some window during connection startup where sending plain

Re: [GENERAL] encoding of PostgreSQL messages

2008-12-31 Thread Karsten Hilbert
Hm, so maybe both Peter and Alvaro are right: 1) Setting the translation wrapper to a NOOP as early as possible. Thus, the first messages are sent in 7-bit ASCII English. Despite being *marked* for translation and a translation to exist in the .po file, that is. Karsten --

Re: [GENERAL] encoding of PostgreSQL messages

2008-12-29 Thread Alvaro Herrera
Karsten Hilbert wrote: On Tue, Dec 23, 2008 at 06:45:17PM -0300, Alvaro Herrera wrote: Hmm, isn't client_encoding reported in the startup packet sent by the server, after auth? That would not quite be enough -- I am talking about messages reported *during* auth, say FATAL:

[GENERAL] encoding of PostgreSQL messages

2008-12-23 Thread Karsten Hilbert
Hi all ! How can I programmatically detect which encoding a PostgreSQL server I am trying to connect to sends back messages -- before I connect (so client_encoding and the pg_settings table are flat out). Thanks, Karsten -- GPG key ID E4071346 @ wwwkeys.pgp.net E167 67FD A291 2BEA 73BD 4537

Re: [GENERAL] encoding of PostgreSQL messages

2008-12-23 Thread Alvaro Herrera
Karsten Hilbert wrote: Hi all ! How can I programmatically detect which encoding a PostgreSQL server I am trying to connect to sends back messages -- before I connect (so client_encoding and the pg_settings table are flat out). Hmm, isn't client_encoding reported in the startup packet sent

Re: [GENERAL] encoding of PostgreSQL messages

2008-12-23 Thread Karsten Hilbert
On Tue, Dec 23, 2008 at 06:45:17PM -0300, Alvaro Herrera wrote: How can I programmatically detect which encoding a PostgreSQL server I am trying to connect to sends back messages -- before I connect (so client_encoding and the pg_settings table are flat out). Hmm, isn't client_encoding

Re: [GENERAL] encoding of PostgreSQL messages

2008-12-23 Thread John DeSoi
On Dec 23, 2008, at 4:24 PM, Karsten Hilbert wrote: How can I programmatically detect which encoding a PostgreSQL server I am trying to connect to sends back messages -- before I connect (so client_encoding and the pg_settings table are flat out). I don't think there is a way because you

[GENERAL] Encoding

2008-11-19 Thread Gustavo Rosso
This error give me to load a file into postgres: ERROR: invalid byte sequence for encoding UTF8: 0xa4 I must force load data. How must to do? -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription:

Re: [GENERAL] Encoding

2008-11-19 Thread Alvaro Herrera
Gustavo Rosso escribió: This error give me to load a file into postgres: ERROR: invalid byte sequence for encoding UTF8: 0xa4 I must force load data. How must to do? Set client_encoding to latin1 (or latin9). How you do this is client-dependent. Note that you can post questions in spanish

Re: [GENERAL] encoding confusion

2008-06-11 Thread Albe Laurenz
Sim Zacks wrote: We originally tested it on mysql and now we are migrating it to postgresql. The messages are stored in a longblob field on mysql and a bytea field in postgresql. I set the database up as UTF-8, even though we get emails that are not UTF encoded, mostly because I didn't

Re: [GENERAL] encoding confusion

2008-06-11 Thread Sim Zacks
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 The data in the longblob field might be text, which could be causing the confusion. For example, when I look at the data in the longblob field, I see /n for a newline and when I look at the bytea it is 012. I can only tell you what happened in the

Re: [GENERAL] encoding confusion

2008-06-11 Thread Richard Huxton
Sim Zacks wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 The data in the longblob field might be text, which could be causing the confusion. For example, when I look at the data in the longblob field, I see /n for a newline and when I look at the bytea it is 012. That's right - newline

Re: [GENERAL] encoding confusion

2008-06-11 Thread Alban Hertroys
On Jun 11, 2008, at 9:03 AM, Richard Huxton wrote: Sim Zacks wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 The data in the longblob field might be text, which could be causing the confusion. For example, when I look at the data in the longblob field, I see /n for a newline and

[GENERAL] encoding confusion

2008-06-10 Thread Sim Zacks
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 {BACKGROUND] I am testing dbmail for our corporate email solution. We originally tested it on mysql and now we are migrating it to postgresql. The messages are stored in a longblob field on mysql and a bytea field in postgresql. I set the database

Re: [GENERAL] encoding confusion

2008-06-10 Thread Richard Huxton
Sim Zacks wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 {BACKGROUND] I am testing dbmail for our corporate email solution. We originally tested it on mysql and now we are migrating it to postgresql. The messages are stored in a longblob field on mysql and a bytea field in postgresql.

Re: [GENERAL] encoding does not match server's locale

2008-02-04 Thread Tom Lane
mikie [EMAIL PROTECTED] writes: However, I used different encodings in previous 8.2 within a cluster and now I don't know how to restore my databases. initdb in C locale. regards, tom lane ---(end of broadcast)--- TIP 3:

[GENERAL] encoding does not match server's locale

2008-02-04 Thread mikie
Hi, I just wanted to upgrade from 8.2 to 8.3. So I performed pg_dump_all and now when I try to feed psql with the dump I have problems with some databases: psql:c:/Program Files/PostgreSQL/8.2/bin/dump.sql:32: ERROR: encoding LATIN2 does not match server's locale Polish_Poland.1250 OK, I have

[GENERAL] Encoding questions

2007-07-08 Thread Pat Maddox
My databases all have the default SQL_ASCII encoding. I read the docs and it says that you shouldn't use it if you're going to have any non-ASCII data. I need to start supporting UTF-8, so it seems I need to convert my database. Can I just convert it, or do I need to dump it, drop it, recreate

Re: [GENERAL] Encoding questions

2007-07-08 Thread Michael Glaesemann
On Jul 8, 2007, at 14:09 , Pat Maddox wrote: Can I just convert it, or do I need to dump it, drop it, recreate with UTF-8, and then load the data? It depends on what is currently in your (SQL_ASCII) database. If it's only ASCII data or all non-ASCII data is UTF-8, you should be okay just

[GENERAL] Encoding Sort order

2007-05-31 Thread Richard Broersma Jr
I understand that it is possible for tables from different databases that have the exact same data but different Encoding can sort rows in differet orders. Could this allow affect the order that triggers and rules are fired? Regards, Richard Broersma Jr. ---(end of

Re: [GENERAL] Encoding Sort order

2007-05-31 Thread Tom Lane
Richard Broersma Jr [EMAIL PROTECTED] writes: I understand that it is possible for tables from different databases that have the exact same data but different Encoding can sort rows in differet orders. Could this allow affect the order that triggers and rules are fired? AFAIK, no --- those

Re: [GENERAL] Encoding Sort order

2007-05-31 Thread CAJ CAJ
On 5/31/07, Richard Broersma Jr [EMAIL PROTECTED] wrote: I understand that it is possible for tables from different databases that have the exact same data but different Encoding can sort rows in differet orders. Could this allow affect the order that triggers and rules are fired? This

[GENERAL] encoding problem at restore

2007-02-18 Thread Bob Hunter
Hello, I have just updated to postgres8.1 and have the following problem. The first line of the PostgreSQL database dump says: SET client_encoding = 'SQL_ASCII'; which is correct. However, the restore says: ERROR: invalid byte sequence for encoding UTF8: 0xe02031 HINT: This error can also

Re: [GENERAL] encoding problem at restore

2007-02-18 Thread Michael Fuhr
On Sat, Feb 17, 2007 at 03:12:44AM -0800, Bob Hunter wrote: ERROR: invalid byte sequence for encoding UTF8: 0xe02031 HINT: This error can also happen if the byte sequence does not match the encoding expected by the server, which is controlled by client_encoding. CONTEXT: COPY tablename,

Re: [GENERAL] encoding advice requested

2006-11-14 Thread Martijn van Oosterhout
On Tue, Nov 14, 2006 at 12:01:44AM +0100, Daniel Verite wrote: Also, you'll find this extensively and better explained in this article, for example: http://ppewww.ph.gla.ac.uk/~flavell/charset/form-i18n.html This is a *really* good article about character sets and form submission. Especially

Re: [GENERAL] encoding advice requested

2006-11-13 Thread Albe Laurenz
My database locale is en_US, and by default my databases are UTF8. My application code allows the user to paste text into a box and submit it to the database. Sometimes the pasted text contains non UTF8 characters, typically the fancy forms of quotes and apostrophes. The database does

Re: [GENERAL] encoding advice requested

2006-11-13 Thread Rick Schumeyer
Albe Laurenz wrote: My database locale is en_US, and by default my databases are UTF8. My application code allows the user to paste text into a box and submit it to the database. Sometimes the pasted text contains non UTF8 characters, typically the fancy forms of quotes and

Re: [GENERAL] encoding advice requested

2006-11-13 Thread Marcus Engene
Rick Schumeyer skrev: I will have to try the WIN1252 encoding. On the client side, my application is a web browser. On the server side, it is php scripts on a linux box. The data comes from copying data from a browser window (pointing to another web site) and pasting it into an html

Re: [GENERAL] encoding advice requested

2006-11-13 Thread Daniel Verite
Rick Schumeyer wrote: I will have to try the WIN1252 encoding. On the client side, my application is a web browser. On the server side, it is php scripts on a linux box. The data comes from copying data from a browser window (pointing to another web site) and pasting it into

Re: [GENERAL] encoding advice requested

2006-11-12 Thread Shoaib Mir
Using the convert function might be of help here as well:convert(string using conversion_name)Change encoding using specified conversion name. Conversions can be defined by CREATE CONVERSION. Also there are some pre-defined conversion names (

Re: [GENERAL] encoding advice requested

2006-11-12 Thread Daniel Verite
Rick Schumeyer wrote: My database locale is en_US, and by default my databases are UTF8. My application code allows the user to paste text into a box and submit it to the database. Sometimes the pasted text contains non UTF8 characters, typically the fancy forms of quotes and

[GENERAL] encoding advice requested

2006-11-11 Thread Rick Schumeyer
My database locale is en_US, and by default my databases are UTF8. My application code allows the user to paste text into a box and submit it to the database. Sometimes the pasted text contains non UTF8 characters, typically the fancy forms of quotes and apostrophes. The database does not

Re: [GENERAL] encoding advice requested

2006-11-11 Thread Martijn van Oosterhout
On Sat, Nov 11, 2006 at 02:45:00PM -0500, Rick Schumeyer wrote: My database locale is en_US, and by default my databases are UTF8. My application code allows the user to paste text into a box and submit it to the database. Sometimes the pasted text contains non UTF8 characters, typically

Re: [GENERAL] Encoding, Unicode, locales, etc.

2006-11-01 Thread Karsten Hilbert
On Tue, Oct 31, 2006 at 11:47:56PM -0500, Tom Lane wrote: Because we depend on libc's locale support, which (on many platforms) isn't designed to switch between locales cheaply. The fact that we allow a per-database encoding spec at all was probably a bad idea in hindsight --- it's out front

Re: [GENERAL] Encoding, Unicode, locales, etc.

2006-11-01 Thread Carlos Moreno
Thanks Tom, for your reply. Tom Lane wrote: Carlos Moreno [EMAIL PROTECTED] writes: Why is it that the database cluster is resrticted to a single locale (or single set of locales) instead of being configurable on a per-database basis? Because we depend on libc's locale support,

  1   2   >