Re: [HACKERS] Synchronized Scan update

2007-03-14 Thread Zeugswetter Andreas ADI SD
> > The advantage of sync_scan_offset is that, in some situations, a > > second scan can actually finish faster than if it were the only query > > executing, because a previous scan has already caused some blocks to > > be cached. However, 16 is a small number because that benefit would > > on

Re: [HACKERS] Synchronized Scan update

2007-03-13 Thread Simon Riggs
On Tue, 2007-03-13 at 13:39 -0700, Jeff Davis wrote: > > > Do you have an opinion about sync_scan_threshold versus a simple > > > sync_scan_enable? > > > > enable_sync_scan? > > > > After looking at other GUC names, I suggest that it's either > "sync_scan" (for on/off) or "sync_scan_threshold"

Re: [HACKERS] Synchronized Scan update

2007-03-13 Thread Jeff Davis
On Tue, 2007-03-13 at 19:24 +, Simon Riggs wrote: > > The advantage of sync_scan_offset is that, in some situations, a second > > scan can actually finish faster than if it were the only query > > executing, because a previous scan has already caused some blocks to be > > cached. However, 16 is

Re: [HACKERS] Synchronized Scan update

2007-03-13 Thread Simon Riggs
On Tue, 2007-03-13 at 11:28 -0700, Jeff Davis wrote: > On Tue, 2007-03-13 at 17:11 +, Simon Riggs wrote: > > On Mon, 2007-03-12 at 17:46 -0700, Jeff Davis wrote: > > > On Mon, 2007-03-12 at 13:21 +, Simon Riggs wrote: > > > > So based on those thoughts, sync_scan_offset should be fixed at 1

Re: [HACKERS] Synchronized Scan update

2007-03-13 Thread Jeff Davis
On Tue, 2007-03-13 at 17:11 +, Simon Riggs wrote: > On Mon, 2007-03-12 at 17:46 -0700, Jeff Davis wrote: > > On Mon, 2007-03-12 at 13:21 +, Simon Riggs wrote: > > > So based on those thoughts, sync_scan_offset should be fixed at 16, > > > rather than being variable. In addition, ss_report_l

Re: [HACKERS] Synchronized Scan update

2007-03-13 Thread Hannu Krosing
Ühel kenal päeval, T, 2007-03-13 kell 12:53, kirjutas Tom Lane: > Jeff Davis <[EMAIL PROTECTED]> writes: > > I agree that ss_report_loc() doesn't need to report on every call. If > > there's any significant overhead I agree that it should report less > > often. Do you think that the overhead is sig

Re: [HACKERS] Synchronized Scan update

2007-03-13 Thread Jeff Davis
On Tue, 2007-03-13 at 17:17 +, Simon Riggs wrote: > On Tue, 2007-03-13 at 10:08 -0700, Jeff Davis wrote: > > On Tue, 2007-03-13 at 12:53 -0400, Tom Lane wrote: > > > Jeff Davis <[EMAIL PROTECTED]> writes: > > > > I agree that ss_report_loc() doesn't need to report on every call. If > > > > ther

Re: [HACKERS] Synchronized Scan update

2007-03-13 Thread Simon Riggs
On Tue, 2007-03-13 at 10:08 -0700, Jeff Davis wrote: > On Tue, 2007-03-13 at 12:53 -0400, Tom Lane wrote: > > Jeff Davis <[EMAIL PROTECTED]> writes: > > > I agree that ss_report_loc() doesn't need to report on every call. If > > > there's any significant overhead I agree that it should report less

Re: [HACKERS] Synchronized Scan update

2007-03-13 Thread Jeff Davis
On Tue, 2007-03-13 at 10:08 -0700, Jeff Davis wrote: > > One extra LWLock cycle per page processed definitely *is* a significant > > overhead ... can you say "context swap storm"? I'd think about doing it > > once every 100 or so pages. > > > > No lock is needed to store the hint. If somehow the

Re: [HACKERS] Synchronized Scan update

2007-03-13 Thread Simon Riggs
On Mon, 2007-03-12 at 17:46 -0700, Jeff Davis wrote: > On Mon, 2007-03-12 at 13:21 +, Simon Riggs wrote: > > So based on those thoughts, sync_scan_offset should be fixed at 16, > > rather than being variable. In addition, ss_report_loc() should only > > report its position every 16 blocks, rath

Re: [HACKERS] Synchronized Scan update

2007-03-13 Thread Jeff Davis
On Tue, 2007-03-13 at 12:53 -0400, Tom Lane wrote: > Jeff Davis <[EMAIL PROTECTED]> writes: > > I agree that ss_report_loc() doesn't need to report on every call. If > > there's any significant overhead I agree that it should report less > > often. Do you think that the overhead is significant on s

Re: [HACKERS] Synchronized Scan update

2007-03-13 Thread Tom Lane
Jeff Davis <[EMAIL PROTECTED]> writes: > I agree that ss_report_loc() doesn't need to report on every call. If > there's any significant overhead I agree that it should report less > often. Do you think that the overhead is significant on such a simple > function? One extra LWLock cycle per page p

Re: [HACKERS] Synchronized Scan update

2007-03-13 Thread Jeff Davis
On Mon, 2007-03-12 at 13:21 +, Simon Riggs wrote: > So based on those thoughts, sync_scan_offset should be fixed at 16, > rather than being variable. In addition, ss_report_loc() should only > report its position every 16 blocks, rather than do this every time, > which will reduce overhead of t

Re: [HACKERS] Synchronized Scan update

2007-03-12 Thread Simon Riggs
On Mon, 2007-03-12 at 08:42 -0700, Luke Lonergan wrote: > On 3/12/07 6:21 AM, "Simon Riggs" <[EMAIL PROTECTED]> wrote: > > > So based on those thoughts, sync_scan_offset should be fixed at 16, > > rather than being variable. In addition, ss_report_loc() should only > > report its position every 1

Re: [HACKERS] Synchronized Scan update

2007-03-12 Thread Luke Lonergan
Simon, On 3/12/07 6:21 AM, "Simon Riggs" <[EMAIL PROTECTED]> wrote: > So based on those thoughts, sync_scan_offset should be fixed at 16, > rather than being variable. In addition, ss_report_loc() should only > report its position every 16 blocks, rather than do this every time, > which will redu

Re: [HACKERS] Synchronized Scan update

2007-03-12 Thread Simon Riggs
On Fri, 2007-03-02 at 15:03 -0800, Jeff Davis wrote: > Is there any consensus about whether to include these two parameters as > GUCs or constants if my patch is to be accepted? > > (1) sync_scan_threshold: Use synchronized scanning for tables greater > than this many pages; smaller tables will no

Re: [HACKERS] Synchronized Scan update

2007-03-07 Thread Jim Nasby
On Mar 6, 2007, at 9:43 AM, Josh Berkus wrote: Don't get me wrong, I want things to be easily understandable as well but the reason you site above pretty much makes us need to remove most of the postgresql.conf, including all bgwriter, vacuum cost delay, and autovac settings. Not to mention commi

Re: [HACKERS] Synchronized Scan update

2007-03-06 Thread Josh Berkus
JD, > Don't get me wrong, I want things to be easily understandable as well > but the reason you site above pretty much > makes us need to remove most of the postgresql.conf, including all > bgwriter, vacuum cost delay, and autovac settings. > Not to mention commit delay and others ;). Wouldn't t

Re: [HACKERS] Synchronized Scan update

2007-03-05 Thread Jeff Davis
On Sun, 2007-03-04 at 11:54 +, Simon Riggs wrote: > > (2) sync_scan_offset: Start a new scan this many pages before a > > currently running scan to take advantage of the pages > > that are likely already in cache. > > I'm somewhat dubious about this parameter, I have to say, even though I > a

Re: [HACKERS] Synchronized Scan update

2007-03-04 Thread Joshua D. Drake
(2) sync_scan_offset: Start a new scan this many pages before a currently running scan to take advantage of the pages that are likely already in cache. I'm somewhat dubious about this parameter, I have to say, even though I am eager for this feature. It seems like a "magic" parameter tha

Re: [HACKERS] Synchronized Scan update

2007-03-04 Thread Simon Riggs
On Fri, 2007-03-02 at 15:03 -0800, Jeff Davis wrote: > Is there any consensus about whether to include these two parameters as > GUCs or constants if my patch is to be accepted? > > (1) sync_scan_threshold: Use synchronized scanning for tables greater > than this many pages; smaller tables will no

Re: [HACKERS] Synchronized Scan update

2007-03-02 Thread Josh Berkus
Jeff, > PS: Did you happen to get my patch for testing (sent off-list)? If > testing will take a while, that's OK, I'd just like to know whether to > expect the results before feature freeze. I'm not sure. We have a bunch to patches in our queue to test, and the benchmark guys don't really expe

Re: [HACKERS] Synchronized Scan update

2007-03-02 Thread Jeff Davis
On Fri, 2007-03-02 at 15:49 -0800, Josh Berkus wrote: > Jeff, > > > Right now they are just constants defined in a header, but a GUC might > > make sense. I'd like to know which version is more acceptable when I > > submit my final patch. > > As much as I hate the thought of more GUCs, until we h

Re: [HACKERS] Synchronized Scan update

2007-03-02 Thread Josh Berkus
Jeff, > Right now they are just constants defined in a header, but a GUC might > make sense. I'd like to know which version is more acceptable when I > submit my final patch. As much as I hate the thought of more GUCs, until we have a solid performance profile for synch scan we probably need the

Re: [HACKERS] Synchronized Scan update

2007-03-02 Thread Jeff Davis
Is there any consensus about whether to include these two parameters as GUCs or constants if my patch is to be accepted? (1) sync_scan_threshold: Use synchronized scanning for tables greater than this many pages; smaller tables will not be affected. (2) sync_scan_offset: Start a new scan this man