Re: [PERFORM] Slow updates, poor IO

2008-09-28 Thread Scott Marlowe
On Sat, Sep 27, 2008 at 4:33 PM, John Huttley [EMAIL PROTECTED] wrote: this is part of the trade-offs of MVCC. was... was a part of the trade-offs. You are thinking of HOT? I don't think it applies in the case of full table updates?? Sure, you just need a table with plenty of empty

Re: [PERFORM] Slow updates, poor IO

2008-09-28 Thread John Huttley
Scott Marlowe wrote: On Fri, Sep 26, 2008 at 5:03 PM, John Huttley [EMAIL PROTECTED] wrote: Hi Andrew, There are two problems. The first is the that if there is a table with a index and an update is performed on a non indexed field, the index is still re indexed. I assume you mean

Re: [PERFORM] Slow updates, poor IO

2008-09-28 Thread Tom Lane
John Huttley [EMAIL PROTECTED] writes: Scott Marlowe wrote: was... was a part of the trade-offs. You are thinking of HOT? I don't think it applies in the case of full table updates?? Sure, as long as there's enough free space on each page. If you wanted to make a table that was optimized

Re: [PERFORM] Slow updates, poor IO

2008-09-28 Thread Scott Carey
I have had great success using FILLFACTOR on certain tables where big updates like this occur and improving performance. It is still not as fast as I would like, but there are significant gains. A big disk array won't help you as much as it should -- yes it will be faster, but it will still be

Re: [PERFORM] Slow updates, poor IO

2008-09-28 Thread John Huttley
Ahh! I've not dealt with that before. I'll look it up. Thanks Tom. Tom Lane wrote: John Huttley [EMAIL PROTECTED] writes: You are thinking of HOT? I don't think it applies in the case of full table updates?? Sure, as long as there's enough free space on each page. If you wanted to

Re: [PERFORM] Slow updates, poor IO

2008-09-28 Thread John Huttley
Thanks to everyone that responded. I've done some benchmarking checkpoint _segments=16 is fine, going to 64 made no improvement. Using update file set size=99 as a statement, but changing 99 on each run.. With 32M shared memory, time in sec and leaving the system idle long enough between

Re: [PERFORM] Slow updates, poor IO

2008-09-28 Thread Greg Smith
On Mon, 29 Sep 2008, John Huttley wrote: checkpoint _segments=16 is fine, going to 64 made no improvement. You might find that it does *after* increasing shared_buffers. If the buffer cache is really small, the checkpoints can't have very much work to do, so their impact on performance is

Re: [PERFORM] Slow updates, poor IO

2008-09-28 Thread John Huttley
Greg Smith wrote: On Mon, 29 Sep 2008, John Huttley wrote: checkpoint _segments=16 is fine, going to 64 made no improvement. You might find that it does *after* increasing shared_buffers. If the buffer cache is really small, the checkpoints can't have very much work to do, so their

Re: [PERFORM] Slow updates, poor IO

2008-09-28 Thread Scott Marlowe
On Sun, Sep 28, 2008 at 8:01 PM, John Huttley [EMAIL PROTECTED] wrote: Ahh bugger, I've just trashed my test setup. I've settled on 64Mb shared memory since I've only got 1Gb or RAM and the system impact of 256M is severe. Also it uses FB-DIMMS which cost arm+leg+first born

Re: [PERFORM] Slow updates, poor IO

2008-09-28 Thread John Huttley
Ah yess... actually I can get the Kingston stuff locally. However at the moment I'm happily married and want to keep it that way! Everything is in pairs too. Actually its a lot cheaper than when it first came out, but still a lot more than your corner shop DDR-2 stuff. --John Scott

Re: [PERFORM] Slow updates, poor IO

2008-09-28 Thread Dan Langille
On Sep 28, 2008, at 10:01 PM, John Huttley wrote: Greg Smith wrote: On Mon, 29 Sep 2008, John Huttley wrote: checkpoint _segments=16 is fine, going to 64 made no improvement. You might find that it does *after* increasing shared_buffers. If the buffer cache is really small, the

Re: [PERFORM] Slow updates, poor IO

2008-09-28 Thread John Huttley
I've canned the db and got rid my of data. I'm in the midst of doing some other benchmarking for a possible change to the bacula database. Loading up 1M records into a table of 60M records complete with indexes. It's still going... --john Dan Langille wrote: On Sep 28, 2008, at 10:01 PM,

Re: [PERFORM] Slow updates, poor IO

2008-09-28 Thread Scott Marlowe
On Sun, Sep 28, 2008 at 9:08 PM, John Huttley [EMAIL PROTECTED] wrote: Ah yess... actually I can get the Kingston stuff locally. However at the moment I'm happily married and want to keep it that way! Everything is in pairs too. Actually its a lot cheaper than when it first came out, but