Re: [PERFORM] Performance With Joins on Large Tables

2006-09-14 Thread Joshua Marsh
Wow, that correlation value is *way* away from order.  If they werereally in exact order by dsiacctno then I'd expect to see 1.0 inthat column.  Can you take another look at the tables and confirmthe ordering?  Does the correlation change if you do an ANALYZE on thetables?  (Some small change is t

Re: [PERFORM] Performance With Joins on Large Tables

2006-09-13 Thread Tom Lane
"Joshua Marsh" <[EMAIL PROTECTED]> writes: >>> On 9/13/06, Tom Lane <[EMAIL PROTECTED]> wrote: Are the tables perhaps nearly in order by the dsiacctno fields? >> >>> My assumption would be they are in exact order. The text file I used >>> in the COPY statement had them in order, so if COPY p

Re: [PERFORM] Performance With Joins on Large Tables

2006-09-13 Thread Joshua Marsh
On 9/13/06, Tom Lane <[EMAIL PROTECTED]> wrote: "Joshua Marsh" <[EMAIL PROTECTED]> writes: >> Are the tables perhaps nearly in order by the dsiacctno fields? > My assumption would be they are in exact order. The text file I used > in the COPY statement had them in order, so if COPY preserves th

Re: [PERFORM] Performance With Joins on Large Tables

2006-09-13 Thread Tom Lane
"Joshua Marsh" <[EMAIL PROTECTED]> writes: >> Are the tables perhaps nearly in order by the dsiacctno fields? > My assumption would be they are in exact order. The text file I used > in the COPY statement had them in order, so if COPY preserves that in > the database, then it is in order. Ah. S

Re: [PERFORM] Performance With Joins on Large Tables

2006-09-13 Thread Joshua Marsh
Are the tables perhaps nearly in order by the dsiacctno fields? If that were the case, and the planner were missing it for some reason, these results would be plausible. BTW, what are you using for work_mem, and how does that compare to your available RAM? regards, tom lan

Re: [PERFORM] Performance With Joins on Large Tables

2006-09-13 Thread Tom Lane
"Joshua Marsh" <[EMAIL PROTECTED]> writes: > I have a suspision that pgsql isn't tuned to properly deal with tables > of this size. Actually, it is. Most of the planner complaints we get are from people whose tables fit in memory and they find that the default planner behavior doesn't apply real

Re: [PERFORM] Performance With Joins on Large Tables

2006-09-13 Thread Joshua Marsh
On 9/13/06, Terje Elde <[EMAIL PROTECTED]> wrote: Jeff Davis wrote: > Is it overestimating the cost of using indexes or underestimating the > cost of a seq scan, or both? Maybe explain with the 0.1 setting will > help? > If enable_seqscan is off, and cost is still set to 1, it could be t

Re: [PERFORM] Performance With Joins on Large Tables

2006-09-13 Thread Terje Elde
Jeff Davis wrote: Is it overestimating the cost of using indexes or underestimating the cost of a seq scan, or both? Maybe explain with the 0.1 setting will help? If enable_seqscan is off, and cost is still set to 1, it could be that it's quite simply forcibly underestimating the co

Re: [PERFORM] Performance With Joins on Large Tables

2006-09-13 Thread Joshua Marsh
Hmm... that sounds bad. I'm sure your system will always choose indexes with that value. Is it overestimating the cost of using indexes or underestimating the cost of a seq scan, or both? Maybe explain with the 0.1 setting will help? Regards, Jeff Davis data=# explain SELECT v.phonedire

Re: [PERFORM] Performance With Joins on Large Tables

2006-09-13 Thread Marcin Mank
> Is there anything I'm missing that is preventing it from using the index? It > just seems weird to me that other joins like this work fine and fast > with indexes, > but this one won't. Did You consider clustering both tables on the dsiacctno index? I just checked that for a 4M rows table even

Re: [PERFORM] Performance With Joins on Large Tables

2006-09-13 Thread Jeff Davis
On Wed, 2006-09-13 at 10:19 -0600, Joshua Marsh wrote: > > Right, it's just used for planning. Avoid setting it too low, if it's > > below about 2.0 you would most likely see some very strange plans. > > Certainly it doesn't make sense at all to set it below 1.0, since that > > is saying it's cheap

Re: [PERFORM] Performance With Joins on Large Tables

2006-09-13 Thread Joshua Marsh
On 9/13/06, Jeff Davis <[EMAIL PROTECTED]> wrote: On Wed, 2006-09-13 at 08:49 -0600, Joshua Marsh wrote: > That seems to have done it. Are there any side effects to this > change? I read about random_page_cost in the documentation and it > seems like this is strictly for planning. All the tabl

Re: [PERFORM] Performance With Joins on Large Tables

2006-09-13 Thread Jeff Davis
On Wed, 2006-09-13 at 08:49 -0600, Joshua Marsh wrote: > That seems to have done it. Are there any side effects to this > change? I read about random_page_cost in the documentation and it > seems like this is strictly for planning. All the tables on this > database will be indexed and of a size

Re: [PERFORM] Performance With Joins on Large Tables

2006-09-13 Thread Joshua Marsh
On 9/13/06, Jim C. Nasby <[EMAIL PROTECTED]> wrote: On Tue, Sep 12, 2006 at 04:17:34PM -0600, Joshua Marsh wrote: > data=# EXPLAIN SELECT v.phonedirect, v.editdrop, EXTRACT (EPOCH FROM > v.custdate), EXTRACT (YEAR FROM s.custdate) || '-' || EXTRACT (MONTH > FROM s.custdate) FROM view_505 v INNER

Re: [PERFORM] Performance With Joins on Large Tables

2006-09-12 Thread Jim C. Nasby
On Tue, Sep 12, 2006 at 04:17:34PM -0600, Joshua Marsh wrote: > data=# EXPLAIN SELECT v.phonedirect, v.editdrop, EXTRACT (EPOCH FROM > v.custdate), EXTRACT (YEAR FROM s.custdate) || '-' || EXTRACT (MONTH > FROM s.custdate) FROM view_505 v INNER JOIN r3s169 s ON v.dsiacctno = > s.dsiacctno; >