Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-20 Thread Simon Riggs
On Thu, 2004-12-16 at 17:54, Richard Huxton wrote: > Josh Berkus wrote: > >>Clearly, OSDL-DBT2 is not a real world test! That is its benefit, since > >>it is heavily instrumented and we are able to re-run it many times > >>without different parameter settings. The application is well known and > >>

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-18 Thread Simon Riggs
On Thu, 2004-12-16 at 17:54, Richard Huxton wrote: > Josh Berkus wrote: > >>Clearly, OSDL-DBT2 is not a real world test! That is its benefit, since > >>it is heavily instrumented and we are able to re-run it many times > >>without different parameter settings. The application is well known and > >>

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-17 Thread Greg Stark
Jan Wieck <[EMAIL PROTECTED]> writes: > Doesn't cranking up the bgwriter_percent to 100 effectively make the entire > shared memory a write-through cache? In other words, with 100% the bgwriter > will allways write all dirty blocks out and it becomes unlikely to avoid an IO > for subsequent modif

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-16 Thread Richard Huxton
Josh Berkus wrote: Simon, Clearly, OSDL-DBT2 is not a real world test! That is its benefit, since it is heavily instrumented and we are able to re-run it many times without different parameter settings. The application is well known and doesn't suffer that badly from factors that would allow certa

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-16 Thread Zeugswetter Andreas DAZ SD
> Hmmm, I've not seen this. For example, with people who are having trouble > with checkpoint spikes on Linux, I've taken to recommending that they call > sync() (via cron) every 5-10 seconds (thanks, Bruce, for suggestion!). > Believe it or not, this does help smooth out the spikes and give

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-15 Thread Josh Berkus
Simon, > Clearly, OSDL-DBT2 is not a real world test! That is its benefit, since > it is heavily instrumented and we are able to re-run it many times > without different parameter settings. The application is well known and > doesn't suffer that badly from factors that would allow certain effects

Re: Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-15 Thread simon
Josh Berkus <[EMAIL PROTECTED]> wrote on 15.12.2004, 18:36:53: > Hmmm, I've not seen this. For example, with people who are having trouble > with checkpoint spikes on Linux, I've taken to recommending that they call > sync() (via cron) every 5-10 seconds (thanks, Bruce, for suggestion!). > B

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-15 Thread Josh Berkus
Folks, > To allow DBT2 to be used for real bgwriter benchmarking, Mark would need to > change the following: > > 1) Randomize the timing of the commits, so that sometimes there is only 30 > writes/minute, and other times there is 300. A timing pattern that would > produce a "sine wave" with occa

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-15 Thread Josh Berkus
Jan, > I too don't think that this approach will retain the checkpoing smooting > effect, the current implementation has. > > The real problem is that the "cleaner" the buffer pool is, the longer > the scan for dirty buffers will take because the dirty blocks tend to be > at the very end of the sc

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-15 Thread Jan Wieck
On 12/12/2004 9:43 PM, Neil Conway wrote: On Sun, 2004-12-12 at 22:08 +, Simon Riggs wrote: > On Sun, 2004-12-12 at 05:46, Neil Conway wrote: > Is the plan to make bgwriter_percent = 100 the default setting? Hmm...must confess that my only plan is: i) discover dynamic behaviour of bgwriter ii)

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-15 Thread Jan Wieck
On 12/12/2004 5:08 PM, Simon Riggs wrote: On Sun, 2004-12-12 at 05:46, Neil Conway wrote: Simon Riggs wrote: > If the bgwriter_percent = 100, then we should actually do the sensible > thing and prepare the list that we need, i.e. limit > StrategyDirtyBufferList to finding at most bgwriter_maxpages.

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-14 Thread Simon Riggs
On Wed, 2004-12-15 at 00:00, Mark Wong wrote: > http://www.osdl.org/projects/dbt2dev/results/dev4-010/211 > Thanks Mark for turning that around so quickly. Looks good... Results performed to compare test 207 http://www.osdl.org/projects/dbt2dev/results/dev4-010/207 test 211 with bg3.patc

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-14 Thread Mark Wong
Sorry, wrong link, right one here: http://www.osdl.org/projects/dbt2dev/results/dev4-010/211 Mark ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-14 Thread Mark Wong
Sorry for the delay; here are results with the bg3.patch with database parameters that should match run 207. I haven't been able to take the time too look over the results myself, but I tried to make sure this run was the same as 207: http://www.osdl.org/projects/dbt2dev/results/dev4-010/2

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-13 Thread Simon Riggs
On Mon, 2004-12-13 at 02:43, Neil Conway wrote: > On Sun, 2004-12-12 at 22:08 +, Simon Riggs wrote: > > > On Sun, 2004-12-12 at 05:46, Neil Conway wrote: > > > Is the plan to make bgwriter_percent = 100 the default setting? > > > > Hmm...must confess that my only plan is: > > i) discover dynam

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-13 Thread Simon Riggs
On Mon, 2004-12-13 at 04:39, Mark Kirkwood wrote: > I am seeing a reasonably reproducible performance boost after applying > your patch (I'm not sure if that was one of the main objectives, but it > certainly is nice). > > I *was* seeing a noticeable decrease between 7.4.6 and 8.0.0RC1 running

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-12 Thread Mark Kirkwood
Simon, I am seeing a reasonably reproducible performance boost after applying your patch (I'm not sure if that was one of the main objectives, but it certainly is nice). I *was* seeing a noticeable decrease between 7.4.6 and 8.0.0RC1 running pgbench. However, after applying your patch, 8.0 is p

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-12 Thread Neil Conway
On Sun, 2004-12-12 at 22:08 +, Simon Riggs wrote: > > On Sun, 2004-12-12 at 05:46, Neil Conway wrote: > > Is the plan to make bgwriter_percent = 100 the default setting? > > Hmm...must confess that my only plan is: > i) discover dynamic behaviour of bgwriter > ii) fix any bugs or wierdness as

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-12 Thread Simon Riggs
> On Sun, 2004-12-12 at 05:46, Neil Conway wrote: > Simon Riggs wrote: > > If the bgwriter_percent = 100, then we should actually do the sensible > > thing and prepare the list that we need, i.e. limit > > StrategyDirtyBufferList to finding at most bgwriter_maxpages. > > Is the plan to make bgwrit

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-11 Thread Neil Conway
I wonder if we even need to retain the bgwriter_percent GUC var. Is there actually a situation in which the combination of bgwriter_maxpages and bgwriter_delay does not give the DBA sufficient flexibility in tuning bgwriter behavior? Simon Riggs wrote: If the bgwriter_percent = 100, then we sho