Re: [GENERAL] problem converting database to UTF-8

2009-02-03 Thread Schwaighofer Clemens
On Fri, Jan 23, 2009 at 02:18, David Goodenough wrote: > > Is there a definative HOWTO that I can follow, if not does someone > have a set of instructions that will work? > > If it matters I am running under Debian. I did it once for a very large db (large for me was 5GB) and converted it from EU

Re: [GENERAL] problem converting database to UTF-8

2009-01-24 Thread Jasen Betts
On 2009-01-22, David Goodenough wrote: > I have a database which was created as LATIN1 (the machine has the > wrong locales installed when I set up PG). It is running 8.3. > > So I found various places which said the way to do this was to do > a pg_dumpall -f dump_file, get rid of the entire dat

Re: [GENERAL] problem converting database to UTF-8

2009-01-22 Thread Vladimir Konrad
> > cat the-source-dump.sql | iconv -t utf8 - > my-converted.sql > > > > Size should not matter in this case... > > Yeah it does. iconv buffers everything in memory, as I recall. Just found an alternative - "uconv" command (part of ICU project): http://www.icu-project.org/userguide/intro.html

Re: [GENERAL] problem converting database to UTF-8

2009-01-22 Thread Daniel Verite
Alan Hodgson wrote: Yeah it does. iconv buffers everything in memory, as I recall. However, you can "split" the file into manageable pieces, run each through iconv, and recombine afterwards. Another way is to just use GNU recode for large files instead of iconv. It's slower but do

Re: [GENERAL] problem converting database to UTF-8

2009-01-22 Thread Alan Hodgson
On Thursday 22 January 2009, David Goodenough wrote: > > You have not understood what I said. I ran iconv, and it changes the > encoding of the data, but not the ENCODING= statements that are > embedded in the datastream. Yes I can change those with sed, but > I do not know what else I need to

Re: [GENERAL] problem converting database to UTF-8

2009-01-22 Thread Vladimir Konrad
> You have not understood what I said. I ran iconv, and it changes the > encoding of the data, but not the ENCODING= statements that are > embedded in the datastream. Yes I can change those with sed, but > I do not know what else I need to change. There must be an easier > way. Oops, please a

Re: [GENERAL] problem converting database to UTF-8

2009-01-22 Thread David Goodenough
On Thursday 22 January 2009, Vladimir Konrad wrote: > > iconv does not change the database encodings embedded in the file > > (and it is quite large). > > Have you read the manual? > >file A pathname of an input file. If no file operands are >specified, or if a file operand is '-'

Re: [GENERAL] problem converting database to UTF-8

2009-01-22 Thread Alan Hodgson
On Thursday 22 January 2009, Vladimir Konrad wrote: > > iconv does not change the database encodings embedded in the file > > (and it is quite large). > > Have you read the manual? > >file A pathname of an input file. If no file operands are >specified, or if a file operand is '-

Re: [GENERAL] problem converting database to UTF-8

2009-01-22 Thread Vladimir Konrad
> iconv does not change the database encodings embedded in the file > (and it is quite large). Have you read the manual? file A pathname of an input file. If no file operands are specified, or if a file operand is '-', the standard input shall be used. cat the-source-dump

Re: [GENERAL] problem converting database to UTF-8

2009-01-22 Thread David Goodenough
On Thursday 22 January 2009, Vladimir Konrad wrote: > > Is there a definative HOWTO that I can follow, if not does someone > > have a set of instructions that will work? > > What about running "iconv" command on the dumped .sql file and transform > it to the utf8? > > Vlad > > PS: man iconv for man

Re: [GENERAL] problem converting database to UTF-8

2009-01-22 Thread Vladimir Konrad
> Is there a definative HOWTO that I can follow, if not does someone > have a set of instructions that will work? What about running "iconv" command on the dumped .sql file and transform it to the utf8? Vlad PS: man iconv for manual -- Sent via pgsql-general mailing list (pgsql-general@postgre

[GENERAL] problem converting database to UTF-8

2009-01-22 Thread David Goodenough
I have a database which was created as LATIN1 (the machine has the wrong locales installed when I set up PG). It is running 8.3. So I found various places which said the way to do this was to do a pg_dumpall -f dump_file, get rid of the entire database, init_db -E UTF-8, and then psql -f dumpfi