Re: [GENERAL] Populating large DB from Perl script

2007-11-06 Thread Andy
Kynn Jones wrote: I have large database that needs to be built from scratch roughly once every month. I use a Perl script to do this. The tables are very large, so I avoid as much as possible using in-memory data structures, and instead I rely heavily on temporary flat files. I have solved

Re: [GENERAL] Populating large DB from Perl script

2007-11-05 Thread Rodrigo De León
On 11/3/07, Mikko Partio [EMAIL PROTECTED] wrote: On Nov 2, 2007 8:45 PM, Kynn Jones [EMAIL PROTECTED] wrote: It would be great if there was a stored proc-archive somewhere in the web where people could post their procedures. I know there are some code examples in the official documentation

Re: [GENERAL] Populating large DB from Perl script

2007-11-03 Thread Mikko Partio
On Nov 2, 2007 8:45 PM, Kynn Jones [EMAIL PROTECTED] wrote: PS: As an aside to the list, as a programmer, when I'm starting out in language, I learn more than I can say from reading source code written by the experts, but for some reason I have had a hard time coming across expertly written

Re: [GENERAL] Populating large DB from Perl script

2007-11-03 Thread Shane Ambler
Mikko Partio wrote: On Nov 2, 2007 8:45 PM, Kynn Jones [EMAIL PROTECTED] wrote: PS: As an aside to the list, as a programmer, when I'm starting out in language, I learn more than I can say from reading source code written by the experts, but for some reason I have had a hard time coming across

Re: [GENERAL] Populating large DB from Perl script

2007-11-02 Thread andy
Kynn Jones wrote: Hi. This is a recurrent problem that I have not been able to find a good solution for. I have large database that needs to be built from scratch roughly once every month. I use a Perl script to do this. The tables are very large, so I avoid as much as possible using

Re: [GENERAL] Populating large DB from Perl script

2007-11-02 Thread Kynn Jones
So... not really an answer (other than I used a stored proc) Actually, I'm interested in your solution. Just to make sure I understood what you did: you bulk-populated (i.e. with $dbh-do('COPY...'), $dbh-pg_putline(...), $dbh-pg_endcopy) the referring tables, with their fkey constraints

Re: [GENERAL] Populating large DB from Perl script

2007-11-02 Thread Dawid Kuroczko
On 11/1/07, Kynn Jones [EMAIL PROTECTED] wrote: Hi. This is a recurrent problem that I have not been able to find a good solution for. I have large database that needs to be built from scratch roughly once every month. I use a Perl script to do this. The tables are very large, so I avoid

Re: [GENERAL] Populating large DB from Perl script

2007-11-02 Thread andy
Kynn Jones wrote: So... not really an answer (other than I used a stored proc) Actually, I'm interested in your solution. Just to make sure I understood what you did: you bulk-populated (i.e. with $dbh-do('COPY...'), $dbh-pg_putline(...), $dbh-pg_endcopy) the referring tables, with their fkey

[GENERAL] Populating large DB from Perl script

2007-11-01 Thread Kynn Jones
Hi. This is a recurrent problem that I have not been able to find a good solution for. I have large database that needs to be built from scratch roughly once every month. I use a Perl script to do this. The tables are very large, so I avoid as much as possible using in-memory data structures,

Re: [GENERAL] Populating large DB from Perl script

2007-11-01 Thread Jorge Godoy
Em Thursday 01 November 2007 16:57:36 Kynn Jones escreveu: But it occurred to me that this is a generic enough problem, and that I'm probably re-inventing a thoroughly invented wheel. Are there standard techniques or resources or Pg capabilities to deal with this sort of situation? You can