Re: [SQL] request for help with COPY syntax

2007-10-26 Thread Chuck D.
On October 25, 2007 09:22:10 pm Tom Lane wrote: > > Did the sed actually do anything? (Hint: the file size of > geonames_fixed.txt would be larger than geonames.txt if it did. > Or you could diff the two files to confirm that something sensible > happened.) > > I suspect that your shell may be fou

Re: [SQL] request for help with COPY syntax

2007-10-25 Thread Chuck D.
On October 25, 2007 09:22:10 pm you wrote: > Did the sed actually do anything? (Hint: the file size of > geonames_fixed.txt would be larger than geonames.txt if it did. > Or you could diff the two files to confirm that something sensible > happened.) > > I suspect that your shell may be fouling th

Re: [SQL] request for help with COPY syntax

2007-10-25 Thread Chuck D.
On October 25, 2007 03:16:59 pm Fernando Hevia wrote: > > As I understand it when a line starts with $ you would like to merge it > with the previous line. > No, it appears the data file I am attempting to COPY has some records with fields that contain a CR/LF in the data of that field. Postgres

Re: [SQL] request for help with COPY syntax

2007-10-25 Thread Chuck D.
On October 25, 2007 10:57:49 am you wrote: > > If all you just want to do is strip out the ^M, you can run dos2unix on > it, assuming that you are running a *nix distro. Well, I guess I could strip the ^M but I'm still left with a $ in the middle of a field which in the same as the line terminato

Re: [SQL] request for help with COPY syntax

2007-10-25 Thread Chuck D.
On October 25, 2007 09:35:23 am Chuck D. wrote: > On October 24, 2007 01:10:59 am Paul Lambert wrote: > > I get around this problem with my data loads by specifying some other > > arbitrary character that I know won't appear in the data as the quote > > character. &

Re: [SQL] request for help with COPY syntax

2007-10-25 Thread Chuck D.
On October 24, 2007 01:10:59 am Paul Lambert wrote: > > I get around this problem with my data loads by specifying some other > arbitrary character that I know won't appear in the data as the quote > character. > > Eg QUOTE E'\f' will specify form feed as the quote character, ergo any > data with d

Re: [SQL] request for help with COPY syntax

2007-10-24 Thread Chuck D.
On October 23, 2007 10:44:51 am you wrote: > Hi Chuck, > Do you need those characters in your table? If not I think you will be > better off preprocessing the data before running copy. > > Replacing those " for ' or directly removing them is quite simple if you > are working in Unix, actually it sh

Re: [SQL] request for help with COPY syntax

2007-10-24 Thread Chuck D.
On October 23, 2007 08:51:18 pm you wrote: > > I got it to work with your sample data by using the COPY command as > follows: COPY geo.orig_city_maxmind > FROM '/home/www/geo/DATA/MAXMIND.com/cities_no_header.txt' > CSV quote as ; I see what you are after and you solved the syntax

[SQL] request for help with COPY syntax

2007-10-23 Thread Chuck D.
Greetings everyone, I'm having some trouble with COPY syntax. I'm importing the cities data from MaxMind, but I run into errors when the data adds a double quote inside a field. The data is CSV, comma delimited, no quotes around fields, ISO-8859-1. I'm using COPY with the defaults and setting