Re: [PERFORM] How does PG know if data is in memory?

2010-10-01 Thread Fabrício dos Anjos Silva
Thank you all for the replies. If PG does not know whether needed data is in memory, how does it estimate cost? There is a huge difference between access time in memory and in secondary storage. Not taking this into account results in almost "useless" estimates. I am not saying that PG does

Re: [PERFORM] How does PG know if data is in memory?

2010-10-01 Thread Craig Ringer
On 1/10/2010 7:12 PM, Fabrício dos Anjos Silva wrote: Thank you all for the replies. If PG does not know whether needed data is in memory, how does it estimate cost? There is a huge difference between access time in memory and in secondary storage. Not taking this into account results i

Re: [PERFORM] How does PG know if data is in memory?

2010-10-01 Thread Kevin Grittner
Craig Ringer wrote: > Because the query often only wants a small subset of the data, and > whole relations are rarely fully cached, it's not enough to know > that "some of relation X is cached", it has to know if the cached > parts are the parts that'll be required, or at least an > approximatio

Re: [PERFORM] How does PG know if data is in memory?

2010-10-01 Thread Pierre C
It sounds horrendously complicated to keep track of to me, and in the end it won't make query execution any faster, it'll just potentially help the planner pick a better plan. I wonder if that'd be worth the extra CPU time spent managing the cache and cache content stats, and using those ca

Re: [PERFORM] How does PG know if data is in memory?

2010-10-01 Thread Fabrício dos Anjos Silva
Craig, I agree with you. Not completely, but I do. I'm just stuck in a situation where I can't figure out what values to use for the parameters. I can't even think of a way on how to test and discover that. I followed Josh Berkus' GUC spreadsheet and some tutorials on PG wiki, but how

Re: [PERFORM] How does PG know if data is in memory?

2010-10-01 Thread Tom Lane
"Kevin Grittner" writes: > I agree with that, but I think there's an even more insidious issue > here. Biasing plans heavily toward using what is already in cache > could have a destabilizing effect on performance. Not to mention the destabilizing effect on the plans themselves. Behavior like th

Re: [PERFORM] gist indexes for distance calculations

2010-10-01 Thread Merlin Moncure
On Thu, Sep 30, 2010 at 2:33 PM, Marcelo Zabani wrote: > Hi everyone. I have a question, and it's well beyond me to even speculate > about the inner workings of postgresql on this. > > I have a "places" table, and a "coordinates" column, of type POINT. > > If I want to find every place within, for

Re: [PERFORM] How does PG know if data is in memory?

2010-10-01 Thread Samuel Gendler
2010/10/1 Fabrício dos Anjos Silva >Craig, > >I agree with you. Not completely, but I do. > >I'm just stuck in a situation where I can't figure out what values to > use for the parameters. I can't even think of a way on how to test and > discover that. >I followed Josh Berkus' GUC

Re: [PERFORM] gist indexes for distance calculations

2010-10-01 Thread Marcelo Zabani
Thanks a lot everyone for all the info! It is all really helpful. 2010/10/1 Merlin Moncure > On Thu, Sep 30, 2010 at 2:33 PM, Marcelo Zabani > wrote: > > Hi everyone. I have a question, and it's well beyond me to even speculate > > about the inner workings of postgresql on this. > > > > I hav

Re: [PERFORM] turn off caching for performance test

2010-10-01 Thread Willy-Bas Loos
> I found one query that did a seqscan anyway(with enable_seqscan off), > because doing an index scan would be more than 1M points more > expensive (to the planner). Hmm, i guess that says it all :) -- "Patriotism is the conviction that your country is superior to all others because you were born

[PERFORM] Issue for partitioning with extra check constriants

2010-10-01 Thread Josh Berkus
Simon, Greg, etc., Just barked my nose against a major performance issue with CE & partitioning, and was wondering if anyone had poked at it. The issue is this: when a partitioned table is evaluated by the planner for constraint exclusion, it evaluates ALL check constraints on each partition, reg

Re: [PERFORM] Issue for partitioning with extra check constriants

2010-10-01 Thread Tom Lane
Josh Berkus writes: > The issue is this: when a partitioned table is evaluated by the planner > for constraint exclusion, it evaluates ALL check constraints on each > partition, regardless of whether or not they include a referenced column > in the query (and whether or not they relate to partitio

Re: [PERFORM] Issue for partitioning with extra check constriants

2010-10-01 Thread Josh Berkus
> [ shrug ... ] We do not promise that the current partitioning scheme > scales to the number of partitions where this is likely to be an > interesting concern. Actually, you can demonstrate pretty significant response time delays on only 50 partitions. > We're talking "wasted effort on a dead-

Re: [PERFORM] Issue for partitioning with extra check constriants

2010-10-01 Thread Tom Lane
Josh Berkus writes: >> [ shrug ... ] We do not promise that the current partitioning scheme >> scales to the number of partitions where this is likely to be an >> interesting concern. > Actually, you can demonstrate pretty significant response time delays on > only 50 partitions. And your point