Re: [GENERAL] Populating large DB from Perl script

2007-11-06 Thread Andy
Kynn Jones wrote: I have large database that needs to be built from scratch roughly once every month. I use a Perl script to do this. The tables are very large, so I avoid as much as possible using in-memory data structures, and instead I rely heavily on temporary flat files. I have solved th

Re: [GENERAL] Populating large DB from Perl script

2007-11-05 Thread Rodrigo De León
On 11/3/07, Mikko Partio <[EMAIL PROTECTED]> wrote: > On Nov 2, 2007 8:45 PM, Kynn Jones <[EMAIL PROTECTED]> wrote: > It would be great if there was a stored proc-archive somewhere in the > web where people could post their procedures. I know there are some > code examples in the official documenta

Re: [GENERAL] Populating large DB from Perl script

2007-11-03 Thread Shane Ambler
Mikko Partio wrote: On Nov 2, 2007 8:45 PM, Kynn Jones <[EMAIL PROTECTED]> wrote: PS: As an aside to the list, as a programmer, when I'm starting out in language, I learn more than I can say from reading source code written by the experts, but for some reason I have had a hard time coming acros

Re: [GENERAL] Populating large DB from Perl script

2007-11-02 Thread Mikko Partio
On Nov 2, 2007 8:45 PM, Kynn Jones <[EMAIL PROTECTED]> wrote: > PS: As an aside to the list, as a programmer, when I'm starting out in > language, I learn more than I can say from reading source code written > by the experts, but for some reason I have had a hard time coming > across expertly writ

Re: [GENERAL] Populating large DB from Perl script

2007-11-02 Thread andy
Kynn Jones wrote: So... not really an answer (other than I used a stored proc) Actually, I'm interested in your solution. Just to make sure I understood what you did: you bulk-populated (i.e. with $dbh->do('COPY...'), $dbh->pg_putline(...), $dbh->pg_endcopy) the referring tables, with their fk

Re: [GENERAL] Populating large DB from Perl script

2007-11-02 Thread Dawid Kuroczko
On 11/1/07, Kynn Jones <[EMAIL PROTECTED]> wrote: > Hi. This is a recurrent problem that I have not been able to find a > good solution for. I have large database that needs to be built from > scratch roughly once every month. I use a Perl script to do this. > > The tables are very large, so I

Re: [GENERAL] Populating large DB from Perl script

2007-11-02 Thread Kynn Jones
> So... not really an answer (other than I used a stored proc) Actually, I'm interested in your solution. Just to make sure I understood what you did: you bulk-populated (i.e. with $dbh->do('COPY...'), $dbh->pg_putline(...), $dbh->pg_endcopy) the referring tables, with their fkey constraints disa

Re: [GENERAL] Populating large DB from Perl script

2007-11-02 Thread andy
Kynn Jones wrote: Hi. This is a recurrent problem that I have not been able to find a good solution for. I have large database that needs to be built from scratch roughly once every month. I use a Perl script to do this. The tables are very large, so I avoid as much as possible using in-memo

Re: [GENERAL] Populating large DB from Perl script

2007-11-01 Thread Jorge Godoy
Em Thursday 01 November 2007 16:57:36 Kynn Jones escreveu: > > But it occurred to me that this is a generic enough problem, and that > I'm probably re-inventing a thoroughly invented wheel. Are there > standard techniques or resources or Pg capabilities to deal with this > sort of situation? You

[GENERAL] Populating large DB from Perl script

2007-11-01 Thread Kynn Jones
Hi. This is a recurrent problem that I have not been able to find a good solution for. I have large database that needs to be built from scratch roughly once every month. I use a Perl script to do this. The tables are very large, so I avoid as much as possible using in-memory data structures,