Hi,
I must convert an old table into a new table. The conversion goes at ~
100 records per second. Given the fact that I must convert 40 million
records, it takes too much time: more hours than the 48 hour weekend I
have for the conversion;-).
The tables are rather simple: both tables only have
Hi Joost,
why do you convert programmatically? I would do something like
create sequence s_objectid;
insert into
prototype.orders(objectid,ordernumber,orderdate,customernumber)
select next_val('s_objectid'),ordernummer, orderdatum, klantnummer from
odbc.orders
Sounds a lot faster to me.
On Fri, Sep 23, 2005 at 08:49:27AM +0200, Joost Kraaijeveld wrote:
3. Can I anyhow improve the performance without replacing my hardware,
e.g. by tweaking the software?
It's not clear what your object id generator does. If it's just a
sequence, it's not clear that you need this program at
On Fri, 2005-09-23 at 05:55 -0400, Michael Stone wrote:
It's not clear what your object id generator does. If it's just a
sequence, it's not clear that you need this program at all--just use a
SELECT INTO and make the object id a SERIAL.
It generates a GUID (and no, I do not want to turn this
On Fri, Sep 23, 2005 at 12:21:15PM +0200, Joost Kraaijeveld wrote:
On Fri, 2005-09-23 at 05:55 -0400, Michael Stone wrote:
It's not clear what your object id generator does. If it's just a
sequence, it's not clear that you need this program at all--just use a
SELECT INTO and make the object id
On 23-9-2005 13:05, Michael Stone wrote:
On Fri, Sep 23, 2005 at 12:21:15PM +0200, Joost Kraaijeveld wrote:
Ok, that's great, but you didn't respond to the suggestion of using COPY
INTO instead of INSERT.
But I have no clue where to begin with determining the bottleneck (it
even may be a
On Fri, 2005-09-23 at 13:19 +0200, Arjen van der Meijden wrote:
Another suggestion:
How many indexes and constraints are on the new table?
As mentioned in the first mail: in this tables only primary key
constraints, no other indexes or constraints.
Drop all of them and recreate them once the
On Fri, 2005-09-23 at 07:05 -0400, Michael Stone wrote:
On Fri, Sep 23, 2005 at 12:21:15PM +0200, Joost Kraaijeveld wrote:
On Fri, 2005-09-23 at 05:55 -0400, Michael Stone wrote:
I didn't say it was, did I?
No, you did not. But only last week someon'es head was (luckely for him
only
On 23-9-2005 15:35, Joost Kraaijeveld wrote:
On Fri, 2005-09-23 at 13:19 +0200, Arjen van der Meijden wrote:
Drop all of them and recreate them once the table is filled. Of course
that only works if you know your data will be ok (which is normal for
imports of already conforming data like
Joost Kraaijeveld [EMAIL PROTECTED] writes:
I will test this a for perfomance improvement, but still, I wonder if ~
100 inserts/second is a reasonable performance for my software/hardware
combination.
Is the client code running on the same machine as the database server?
If not, what's the
On Fri, 2005-09-23 at 10:33 -0400, Tom Lane wrote:
Is the client code running on the same machine as the database server?
If not, what's the network delay and latency between them?
Yes, it is running on the same machine.
The major problem you're going to have here is at least one network
Joost Kraaijeveld [EMAIL PROTECTED] writes:
But do you maybe know a pointer to info, or tools that can measure, what
my machine is doing during all the time it is doing nothing? Something
like the performance monitor in Windows but than for Linux?
top, vmstat, iostat, sar, strace, oprofile,
On Fri, Sep 23, 2005 at 03:49:25PM +0200, Joost Kraaijeveld wrote:
On Fri, 2005-09-23 at 07:05 -0400, Michael Stone wrote:
Ok, that's great, but you didn't respond to the suggestion of using COPY
INTO instead of INSERT.
Part of the code I left out are some data conversions (e.g. from
13 matches
Mail list logo
