Re: [GENERAL] Oddball data distribution giving me planner headaches

2011-12-03 Thread Harald Fuchs
Jeff Amiel becauseimj...@yahoo.com writes: At the moment I think the only way to work around this is to denormalize your schema a bit. And I feared as much. It's biting me in other areas as well...this unusual distribution of data...certain types of customers have completely different

[GENERAL] Oddball data distribution giving me planner headaches

2011-12-02 Thread Jeff Amiel
Oddball data distribution giving me headaches. We have a distinct 'customer' table with customer_id, type and name/demographic information. Assume some 1 million rows in the customer table. We then have a customer 'relationship' table which simply contains 2 columns…designating parent and

Re: [GENERAL] Oddball data distribution giving me planner headaches

2011-12-02 Thread David Johnston
-Original Message- From: pgsql-general-ow...@postgresql.org [mailto:pgsql-general-ow...@postgresql.org] On Behalf Of Jeff Amiel Sent: Friday, December 02, 2011 3:20 PM To: pgsql-general@postgresql.org Subject: [GENERAL] Oddball data distribution giving me planner headaches explain

Re: [GENERAL] Oddball data distribution giving me planner headaches

2011-12-02 Thread Jeff Amiel
--- On Fri, 12/2/11, David Johnston pol...@yahoo.com wrote: From: David Johnston pol...@yahoo.com What kind of plan does the following give? EXPLAIN ANALYZE SELECT * FROM customer_rel p JOIN customer c ON (p.parent_customer = c.customer_id) WHERE c.customer_type = 'DISTRIBUTOR' Nearly

Re: [GENERAL] Oddball data distribution giving me planner headaches

2011-12-02 Thread David Johnston
-Original Message- From: Jeff Amiel [mailto:becauseimj...@yahoo.com] Sent: Friday, December 02, 2011 3:52 PM To: pgsql-general@postgresql.org; David Johnston Subject: RE: [GENERAL] Oddball data distribution giving me planner headaches --- On Fri, 12/2/11, David Johnston pol...@yahoo.com

Re: [GENERAL] Oddball data distribution giving me planner headaches

2011-12-02 Thread Jeff Amiel
--- On Fri, 12/2/11, David Johnston pol...@yahoo.com wrote: What happens if you disable, say, nested loops and/or index scans? planner selects different join/indexing techniques (query is slower) but row estimates (bad) remain identical. -- Sent via pgsql-general mailing list

Re: [GENERAL] Oddball data distribution giving me planner headaches

2011-12-02 Thread David Johnston
-Original Message- From: Jeff Amiel [mailto:becauseimj...@yahoo.com] Sent: Friday, December 02, 2011 4:15 PM To: pgsql-general@postgresql.org; David Johnston Subject: RE: [GENERAL] Oddball data distribution giving me planner headaches --- On Fri, 12/2/11, David Johnston pol

Re: [GENERAL] Oddball data distribution giving me planner headaches

2011-12-02 Thread Jeff Amiel
--- On Fri, 12/2/11, David Johnston pol...@yahoo.com wrote: From: David Johnston pol...@yahoo.com - My, possibly naïve, observation: So aside from the fact the estimates seem to be off the planner has still chosen the most

Re: [GENERAL] Oddball data distribution giving me planner headaches

2011-12-02 Thread Tom Lane
Jeff Amiel becauseimj...@yahoo.com writes: Oddball data distribution giving me headaches. [ 'distributor' customers have many more child customers than average ] Does this oddball data distribution doom me to poor planning forever? The only real fix for that will require cross-column

Re: [GENERAL] Oddball data distribution giving me planner headaches

2011-12-02 Thread David Johnston
-Original Message- From: Jeff Amiel [mailto:becauseimj...@yahoo.com] Sent: Friday, December 02, 2011 5:07 PM To: pgsql-general@postgresql.org; David Johnston Subject: RE: [GENERAL] Oddball data distribution giving me planner headaches --- On Fri, 12/2/11, David Johnston pol

Re: [GENERAL] Oddball data distribution giving me planner headaches

2011-12-02 Thread Jeff Amiel
--- On Fri, 12/2/11, David Johnston pol...@yahoo.com wrote: Can you wrap the query into an SQL or PL/pgSQL function so that, at least, then planner will not be able to see the embedded plan info in the outer queries?  You use-case

Re: [GENERAL] Oddball data distribution giving me planner headaches

2011-12-02 Thread Jeff Amiel
--- On Fri, 12/2/11, Tom Lane t...@sss.pgh.pa.us wrote: The only real fix for that will require cross-column statistics, which we don't have yet --- without such, there's no way for the planner to know that distributors have an atypical number of child customers. I suspected as such.

Re: [GENERAL] Oddball data distribution giving me planner headaches

2011-12-02 Thread Jeff Amiel
--- On Fri, 12/2/11, Tom Lane t...@sss.pgh.pa.us wrote: The only real fix for that will require cross-column statistics, which we don't have yet --- without such, there's no way for the planner to know that distributors have an atypical number of child customers. The only caveat that I