Re: [PERFORM] Scrub one large table against another (vmstat output)

2006-10-11 Thread Markus Schaber
Hi, Brendan, Brendan Curran wrote: >> What prevents you from using an aggregate function? > > I guess I could actually obtain the results in an aggregate function and > use those to maintain a summary table. There is a web view that requires > 'as accurate as possible' numbers to be queried per g

Re: [PERFORM] Scrub one large table against another

2006-10-11 Thread Jim C. Nasby
On Wed, Oct 11, 2006 at 10:53:41AM -0600, Brendan Curran wrote: > Interestingly, and thank you to Tom and Jim, the explicit JOIN improved > performance tremendously (RESULTS BELOW). I converted the entire query > to use explicit joins instead of IN and EXISTS and discovered acceptable > performa

Collect stats during seqscan (was: [PERFORM] Simple join optimized badly?)

2006-10-11 Thread Jim C. Nasby
On Wed, Oct 11, 2006 at 10:27:26AM -0400, Bucky Jordan wrote: > Also, I'm guessing this has already come up at some point, but what > about allowing PG to do some stat collection during queries? If you're > touching a lot of data (such as an import process) wouldn't it be more > efficient (and perh

Re: [PERFORM] Scrub one large table against another (vmstat output)

2006-10-11 Thread Brendan Curran
What prevents you from using an aggregate function? I guess I could actually obtain the results in an aggregate function and use those to maintain a summary table. There is a web view that requires 'as accurate as possible' numbers to be queried per group (all 40 groups are displayed on the

Re: [PERFORM] Scrub one large table against another

2006-10-11 Thread Brendan Curran
Tom Lane wrote: Brendan Curran <[EMAIL PROTECTED]> writes: So much time is being spent in the Unique and Sort leaves... I would think that it wouldn't need to do the unique portion, since there is no DISTINCT clause... There's nothing in that query suggesting that suppress.email is unique. If

Re: [PERFORM] Simple join optimized badly?

2006-10-11 Thread Mark Lewis
Tom, I'm interested in the problem of cross-column statistics from a theoretical perspective. It would be interesting to sit down and try to reason out a useful solution, or at very least to understand the problem better so I can anticipate when it might come and eat me. >From my understanding,

Re: [PERFORM] Simple join optimized badly?

2006-10-11 Thread Bruce Momjian
Heikki Linnakangas wrote: > BTW, in DB2 you can declare a table as volatile, which means that the > cardinality of the table varies greatly. The planner favors index scans > on volatile tables. Now that seems like a valuable idea. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp:/

Re: [PERFORM] Simple join optimized badly?

2006-10-11 Thread Heikki Linnakangas
Bucky Jordan wrote: Is this along the lines of "I'm loading a big table and touching every row of data, so I may as well collect some stats along the way" and "I know my data contains these statistical properties, but the analyzer wasn't able to figure that out (or maybe can't figure it out effi

Re: [PERFORM] Simple join optimized badly?

2006-10-11 Thread Bucky Jordan
> Brian Herlihy <[EMAIL PROTECTED]> writes: > > What would it take for hints to be added to postgres? > > A *whole lot* more thought and effort than has been expended on the > subject to date. > > Personally I have no use for the idea of "force the planner to do > exactly X given a query of exact