Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-20 Thread Kevin Grittner
Greg Smith g...@2ndquadrant.com wrote: On 06/19/2011 06:15 PM, Kevin Grittner wrote: I think the point is that if, on a fresh system, the first access to a table is something which uses a tables scan -- like select count(*) -- that all indexed access would then tend to be suppressed for that

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-20 Thread Greg Smith
Kevin Grittner wrote: But its not hard to imagine an application mix where this feature could cause a surprising ten-fold performance drop after someone does a table scan which could persist indefinitely. I'm not risking that in production without a clear mechanism to automatically recover from

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-20 Thread Kevin Grittner
Greg Smith g...@2ndquadrant.com wrote: The idea that any of this will run automatically is a dream at this point, so saying you want to automatically recover from problems with the mechanism that doesn't even exist yet is a bit premature. Well, I certainly didn't mean it to be a reason not

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-19 Thread Greg Stark
On Tue, Jun 14, 2011 at 4:04 PM, Robert Haas robertmh...@gmail.com wrote: 1. ANALYZE happens far too infrequently to believe that any data taken at ANALYZE time will still be relevant at execution time. 2. Using data gathered by ANALYZE will make plans less stable, and our users complain not

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-19 Thread Cédric Villemain
2011/6/19 Greg Stark st...@mit.edu: On Tue, Jun 14, 2011 at 4:04 PM, Robert Haas robertmh...@gmail.com wrote: 1. ANALYZE happens far too infrequently to believe that any data taken at ANALYZE time will still be relevant at execution time. 2. Using data gathered by ANALYZE will make plans less

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-19 Thread Robert Haas
On Sun, Jun 19, 2011 at 9:38 AM, Greg Stark st...@mit.edu wrote: On Tue, Jun 14, 2011 at 4:04 PM, Robert Haas robertmh...@gmail.com wrote: 1. ANALYZE happens far too infrequently to believe that any data taken at ANALYZE time will still be relevant at execution time. 2. Using data gathered by

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-19 Thread Cédric Villemain
2011/6/19 Robert Haas robertmh...@gmail.com: On Sun, Jun 19, 2011 at 9:38 AM, Greg Stark st...@mit.edu wrote: On Tue, Jun 14, 2011 at 4:04 PM, Robert Haas robertmh...@gmail.com wrote: 1. ANALYZE happens far too infrequently to believe that any data taken at ANALYZE time will still be relevant

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-19 Thread Greg Smith
On 06/19/2011 09:38 AM, Greg Stark wrote: There's another problem which I haven't seen mentioned. Because the access method will affect the cache there's the possibility of feedback loops. e.g. A freshly loaded system prefers sequential scans for a given table because without the cache the seeks

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-19 Thread Kevin Grittner
Greg Smith wrote: I'm not too concerned about the specific case you warned about because I don't see how sequential scan vs. index costing will be any different on a fresh system than it is now. I think the point is that if, on a fresh system, the first access to a table is something which

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-19 Thread Greg Smith
On 06/19/2011 06:15 PM, Kevin Grittner wrote: I think the point is that if, on a fresh system, the first access to a table is something which uses a tables scan -- like select count(*) -- that all indexed access would then tend to be suppressed for that table. After all, for each individual

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-19 Thread Robert Haas
On Sun, Jun 19, 2011 at 3:32 PM, Cédric Villemain cedric.villemain.deb...@gmail.com wrote: 2nd, I provided the patches on the last CF, exactly to allow to go to the exciting part: the cost-estimates changes. (after all, we can work on the cost estimate, and if later we find a way to use ALTER

[HACKERS] [WIP] cache estimates, cache access cost

2011-06-14 Thread Cédric Villemain
2011/5/16 Greg Smith g...@2ndquadrant.com: Cédric Villemain wrote: http://git.postgresql.org/gitweb?p=users/c2main/postgres.git;a=shortlog;h=refs/heads/analyze_cache This rebases easily to make Cedric's changes move to the end; I just pushed a version with that change to

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-14 Thread Robert Haas
On Tue, Jun 14, 2011 at 10:29 AM, Cédric Villemain cedric.villemain.deb...@gmail.com wrote: 0001-Add-reloscache-column-to-pg_class.patch 0002-Add-a-function-to-update-the-new-pg_class-cols.patch 0003-Add-ANALYZE-OSCACHE-VERBOSE-relation.patch 0004-Add-a-Hook-to-handle-OSCache-stats.patch

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-14 Thread Cédric Villemain
2011/6/14 Robert Haas robertmh...@gmail.com: On Tue, Jun 14, 2011 at 10:29 AM, Cédric Villemain cedric.villemain.deb...@gmail.com wrote: 0001-Add-reloscache-column-to-pg_class.patch 0002-Add-a-function-to-update-the-new-pg_class-cols.patch 0003-Add-ANALYZE-OSCACHE-VERBOSE-relation.patch

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-14 Thread Greg Smith
On 06/14/2011 11:04 AM, Robert Haas wrote: Even if the data were accurate and did not cause plan stability, we have no evidence that using it will improve real-world performance. That's the dependency Cédric has provided us a way to finally make progress on. Everyone says there's no

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-14 Thread Robert Haas
On Tue, Jun 14, 2011 at 1:10 PM, Greg Smith g...@2ndquadrant.com wrote: On 06/14/2011 11:04 AM, Robert Haas wrote: Even if the data were accurate and did not cause plan stability, we have no evidence that using it will improve real-world performance. That's the dependency Cédric has provided

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-14 Thread Robert Haas
On Tue, Jun 14, 2011 at 12:06 PM, Cédric Villemain cedric.villemain.deb...@gmail.com wrote: 1. ANALYZE happens far too infrequently to believe that any data taken at ANALYZE time will still be relevant at execution time. ANALYZE happens when people execute it, else it is auto-analyze and I am

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-14 Thread Cédric Villemain
2011/6/14 Robert Haas robertmh...@gmail.com: On Tue, Jun 14, 2011 at 12:06 PM, Cédric Villemain cedric.villemain.deb...@gmail.com wrote: 1. ANALYZE happens far too infrequently to believe that any data taken at ANALYZE time will still be relevant at execution time. ANALYZE happens when

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-14 Thread Alvaro Herrera
Excerpts from Cédric Villemain's message of mar jun 14 10:29:36 -0400 2011: Attached are updated patches without the plugin itself. I've also added the cache_page_cost GUC, this one is not per tablespace, like others page_cost. There are 6 patches:

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-14 Thread Cédric Villemain
2011/6/14 Alvaro Herrera alvhe...@commandprompt.com: Excerpts from Cédric Villemain's message of mar jun 14 10:29:36 -0400 2011: Attached are updated patches without the plugin itself. I've also added the cache_page_cost GUC, this one is not per tablespace, like others page_cost. There are

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-14 Thread Alvaro Herrera
Excerpts from Cédric Villemain's message of mar jun 14 17:10:20 -0400 2011: If we can have ALTER TABLE running on heavy workload, why not. I am bit scared by the effect of such reloption, it focus on HINT oriented strategy when I would like to allow a dynamic strategy from the server. This

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-14 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Cédric Villemain's message of mar jun 14 10:29:36 -0400 2011: 0001-Add-reloscache-column-to-pg_class.patch Hmm, do you really need this to be a new column? Would it work to have it be a reloption? If it's to be updated in the

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-14 Thread Greg Smith
On 06/14/2011 01:16 PM, Robert Haas wrote: But there's no reason that code (which may or may not eventually prove useful) has to be incorporated into the main tree. We don't commit code so people can go benchmark it; we ask for the benchmarking to be done first, and then if the results are

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-14 Thread Bruce Momjian
Greg Smith wrote: On 06/14/2011 01:16 PM, Robert Haas wrote: But there's no reason that code (which may or may not eventually prove useful) has to be incorporated into the main tree. We don't commit code so people can go benchmark it; we ask for the benchmarking to be done first, and

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-14 Thread Tom Lane
Greg Smith g...@2ndquadrant.com writes: On 06/14/2011 01:16 PM, Robert Haas wrote: But there's no reason that code (which may or may not eventually prove useful) has to be incorporated into the main tree. We don't commit code so people can go benchmark it; we ask for the benchmarking to be

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-14 Thread Greg Smith
On 06/14/2011 07:08 PM, Tom Lane wrote: I concur with Robert's desire to not push experimental code into the main repository, but we need to have *some* way of working with it. Maybe a separate repo where experimental branches could hang out would be helpful? Well, this one is sitting

Re: [HACKERS] [WIP] cache estimates, cache access cost

2011-06-14 Thread Robert Haas
On Tue, Jun 14, 2011 at 6:17 PM, Greg Smith g...@2ndquadrant.com wrote: Who said anything about this being a commit candidate?  The WIP in the subject says it's not intended to be.  The community asks people to submit design ideas early so that ideas around them can be explored publicly.  One