I am currently using a Perl data loader that was set up to load data to
three particular tables. The majority of this data is entered into one table.
The scripts obtain the data by parsing it out of an Excel spreadsheet.
My loads seem to take a very long time. Each file only has about 12,000
Jodi-
Have you tried turning autocommit off & doing a single commit after the
load?
-NickF
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]]On Behalf Of Jodi Kanter
Sent: Wednesday, March 20, 2002 2:19 PM
To: Postgres Admin List
Subject: [ADMIN] slow inserts
Hi Jodi,
> None of the data is actually committed to the database until the scripts
> complete so I believe that autocommit is turned off.
>
what if you try to write the output of your script into a separate file
and pipe it to a psql as input? What i mean is to strip of the processing
time for
DBI->connect("dbi:Pg:dbname=dbname;host=hostname",
'user', 'password', {AutoCommit => 0})
Thanks,Peter Darley
-Original Message-From:
[EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]]On Behalf Of Jodi
KanterSent: Wednesday, March 20, 2002 11:19 AMTo:
Post
"
<[EMAIL PROTECTED]>
Sent: Wednesday, March 20, 2002 2:42 PM
Subject: Re: [ADMIN] slow inserts
> Jodi-
>
> Have you tried turning autocommit off & doing a single commit after the
> load?
>
> -NickF
>
> -Original Message-
> From: [EMAIL PROTECTED]
&
Jodi Kanter wrote:
> I am currently using a Perl data loader that was set up to load data to
three particular
> tables.
(Snip)
> I have placed some debugging syntax in the code and it seems that the
extra time if
> related to postgres as I had originally thought it may have to do with
the pars
> my $dbh=DBI->connect(dbi:Pg ...);
>
> my $insh = $dbh->prepare("Insert into table values (?,?,?)";
>
$insh->begin_work;
> foreach ($excelrow){
> parse;
> $insh->execute($data1,$data2,$data3);
> }
$insh->commit;
>
> I have written a few script of that kind my self, and I was really
> surp
