Re: [SQL] request for help with COPY syntax

2007-10-26 Thread Chuck D.
On October 25, 2007 09:22:10 pm Tom Lane wrote: > > Did the sed actually do anything? (Hint: the file size of > geonames_fixed.txt would be larger than geonames.txt if it did. > Or you could diff the two files to confirm that something sensible > happened.) > > I suspect that your shell may be fou

Re: [SQL] request for help with COPY syntax

2007-10-25 Thread Chuck D.
On October 25, 2007 09:22:10 pm you wrote: > Did the sed actually do anything? (Hint: the file size of > geonames_fixed.txt would be larger than geonames.txt if it did. > Or you could diff the two files to confirm that something sensible > happened.) > > I suspect that your shell may be fouling th

Re: [SQL] request for help with COPY syntax

2007-10-25 Thread Tom Lane
"Chuck D." <[EMAIL PROTECTED]> writes: > I tried this, which I found on the web from Tom Lane: > sed 's/^M/\\r/g' geonames.txt > geonames_fixed.txt > But still get the same error. I used ctrl-v ctrl-m to reproduce the > ^M. Not sure why it is kicking out those lines still. Did the sed actually

Re: [SQL] request for help with COPY syntax

2007-10-25 Thread Chuck D.
On October 25, 2007 03:16:59 pm Fernando Hevia wrote: > > As I understand it when a line starts with $ you would like to merge it > with the previous line. > No, it appears the data file I am attempting to COPY has some records with fields that contain a CR/LF in the data of that field. Postgres

Re: [SQL] request for help with COPY syntax

2007-10-25 Thread Fernando Hevia
> On October 25, 2007 10:57:49 am you wrote: > > > > If all you just want to do is strip out the ^M, you can run dos2unix on > > it, assuming that you are running a *nix distro. > > Well, I guess I could strip the ^M but I'm still left with a $ in the > middle > of a field which in the same as th

Re: [SQL] request for help with COPY syntax

2007-10-25 Thread Chuck D.
On October 25, 2007 10:57:49 am you wrote: > > If all you just want to do is strip out the ^M, you can run dos2unix on > it, assuming that you are running a *nix distro. Well, I guess I could strip the ^M but I'm still left with a $ in the middle of a field which in the same as the line terminato

Re: [SQL] request for help with COPY syntax

2007-10-25 Thread Bricklen Anderson
Chuck D. wrote: Pardon me on this, the cat -A report for the failed line (and subsequent lines) shows ^M$ within the field, not just $. I assume that is probably a \r\n and postgres wants \r for field data and \n to end a line. I've tried working this over with sed but can't get the syntax

Re: [SQL] request for help with COPY syntax

2007-10-25 Thread Chuck D.
On October 25, 2007 09:35:23 am Chuck D. wrote: > On October 24, 2007 01:10:59 am Paul Lambert wrote: > > I get around this problem with my data loads by specifying some other > > arbitrary character that I know won't appear in the data as the quote > > character. > > > > Eg QUOTE E'\f' will specif

Re: [SQL] request for help with COPY syntax

2007-10-25 Thread Chuck D.
On October 24, 2007 01:10:59 am Paul Lambert wrote: > > I get around this problem with my data loads by specifying some other > arbitrary character that I know won't appear in the data as the quote > character. > > Eg QUOTE E'\f' will specify form feed as the quote character, ergo any > data with d

Re: [SQL] request for help with COPY syntax

2007-10-24 Thread Fernando Hevia
> De: Chuck D. > > I'm not sure if they are needed because I've never seen a double quote in > a > place name before. I don't believe they are errors though because there > are > more records that contain them. As well, some records have single and > double > quotes allowed within a record and

Re: [SQL] request for help with COPY syntax

2007-10-24 Thread Harald Fuchs
In article <[EMAIL PROTECTED]>, "Chuck D." <[EMAIL PROTECTED]> writes: > On October 23, 2007 08:51:18 pm you wrote: >> >> I got it to work with your sample data by using the COPY command as >> follows: COPY geo.orig_city_maxmind >> FROM '/home/www/geo/DATA/MAXMIND.com/cities_no_header.txt' >> CSV

Re: [SQL] request for help with COPY syntax

2007-10-24 Thread Paul Lambert
Chuck D. wrote: Greetings everyone, I'm having some trouble with COPY syntax. I'm importing the cities data from MaxMind, but I run into errors when the data adds a double quote inside a field. The data is CSV, comma delimited, no quotes around fields, ISO-8859-1. I'm using COPY with the d

Re: [SQL] request for help with COPY syntax

2007-10-24 Thread Chuck D.
On October 23, 2007 10:44:51 am you wrote: > Hi Chuck, > Do you need those characters in your table? If not I think you will be > better off preprocessing the data before running copy. > > Replacing those " for ' or directly removing them is quite simple if you > are working in Unix, actually it sh

Re: [SQL] request for help with COPY syntax

2007-10-24 Thread Chuck D.
On October 23, 2007 08:51:18 pm you wrote: > > I got it to work with your sample data by using the COPY command as > follows: COPY geo.orig_city_maxmind > FROM '/home/www/geo/DATA/MAXMIND.com/cities_no_header.txt' > CSV quote as ; I see what you are after and you solved the syntax

Re: [SQL] request for help with COPY syntax

2007-10-23 Thread Adrian Klaver
On Tuesday 23 October 2007 9:19 am, Chuck D. wrote: > Greetings everyone, > > I'm having some trouble with COPY syntax. > > I'm importing the cities data from MaxMind, but I run into errors when the > data adds a double quote inside a field. > > The data is CSV, comma delimited, no quotes around fi

Re: [SQL] request for help with COPY syntax

2007-10-23 Thread Fernando Hevia
> -Mensaje original- > De: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] > En nombre de Chuck D. > > Anyone known how I can rewrite the COPY command to allow those " or ' > within > the data? After a couple days I wasn't able to find any examples to help. > Hi Chuck, Do you need those c

Re: [SQL] request for help with COPY syntax

2007-10-23 Thread Andrew Sullivan
On Tue, Oct 23, 2007 at 10:19:07AM -0600, Chuck D. wrote: > by,kruhavyetskalini"na,KruhavyetsKalini"na,02,52.1438889,31.6925 > > There are a couple " where I would expect to see ' instead. I see other > lines > in the data that use both in a field. Ugh. I think I would normalise the data befo

[SQL] request for help with COPY syntax

2007-10-23 Thread Chuck D.
Greetings everyone, I'm having some trouble with COPY syntax. I'm importing the cities data from MaxMind, but I run into errors when the data adds a double quote inside a field. The data is CSV, comma delimited, no quotes around fields, ISO-8859-1. I'm using COPY with the defaults and setting