Re: [HACKERS] [GENERAL] Very slow queries w/ NOT IN preparation (seems like a bug, test case)

2008-11-12 Thread Tom Lane
Sergey Konoplev [EMAIL PROTECTED] writes: You are right. I've found the odd thing (that completely drives me mad) in postgresql.conf. You are able to reproduce slow-not-in queries by switching constraint_exclusion to on in your postgresql.conf and running my test (which is attached to the

Re: [HACKERS] [GENERAL] Very slow queries w/ NOT IN preparation (seems like a bug, test case)

2008-11-12 Thread Tom Lane
Richard Huxton [EMAIL PROTECTED] writes: Tom Lane wrote: Hmph. It's trying to see if the NOT IN condition is self-contradictory, which of course it isn't, but the predicate_refuted_by machinery isn't smart enough to determine that except by running through all N^2 combinations of the

Re: [HACKERS] [GENERAL] Very slow queries w/ NOT IN preparation (seems like a bug, test case)

2008-11-12 Thread Brendan Jurd
On Thu, Nov 13, 2008 at 4:52 AM, Tom Lane [EMAIL PROTECTED] wrote: Yeah. An example of a closely related expression that it *would* be able to prove self-contradictory is WHERE x = ALL (ARRAY[1, 2, ...]) or perhaps slightly more realistically WHERE x = ANY (ARRAY[1, 2, 3]) AND

Re: [HACKERS] [GENERAL] Very slow queries w/ NOT IN preparation (seems like a bug, test case)

2008-11-12 Thread Tom Lane
Brendan Jurd [EMAIL PROTECTED] writes: I guess my question is, what's the real benefit of going to all this trouble trying to prove that clauses are false? Not having to scan gigabytes of data in an excluded partition, for instance. Now the docs do say Currently, constraint_exclusion

Re: [HACKERS] [GENERAL] Very slow queries w/ NOT IN preparation (seems like a bug, test case)

2008-11-12 Thread Heikki Linnakangas
Brendan Jurd wrote: On Thu, Nov 13, 2008 at 4:52 AM, Tom Lane [EMAIL PROTECTED] wrote: Yeah. An example of a closely related expression that it *would* be able to prove self-contradictory is WHERE x = ALL (ARRAY[1, 2, ...]) or perhaps slightly more realistically WHERE x = ANY

Re: [HACKERS] [GENERAL] Very slow queries w/ NOT IN preparation (seems like a bug, test case)

2008-11-12 Thread Brendan Jurd
On Thu, Nov 13, 2008 at 5:16 AM, Tom Lane [EMAIL PROTECTED] wrote: Brendan Jurd [EMAIL PROTECTED] writes: I guess my question is, what's the real benefit of going to all this trouble trying to prove that clauses are false? Not having to scan gigabytes of data in an excluded partition, for

Re: [HACKERS] [GENERAL] Very slow queries w/ NOT IN preparation (seems like a bug, test case)

2008-11-12 Thread Tom Lane
I wrote: We could respond to this in a number of ways: 1. Tough, don't do that. 2. Put some arbitrary limit on the number of subconditions in an AND or OR clause before we give up and don't attempt to prove anything about it. 3. Put in a narrow hack that will get us out of this specific

Re: [HACKERS] [GENERAL] Very slow queries w/ NOT IN preparation (seems like a bug, test case)

2008-11-12 Thread Tom Lane
Heikki Linnakangas [EMAIL PROTECTED] writes: Which brings to mind an interesting customer case. They are running queries like WHERE id IN (...), where ... is a *very* long list of keys, against a table that's partitioned by ranges of id. The query was running slow, because while constraint

Re: [HACKERS] [GENERAL] Very slow queries w/ NOT IN preparation (seems like a bug, test case)

2008-11-12 Thread Tom Lane
I wrote: 2. Put some arbitrary limit on the number of subconditions in an AND or OR clause before we give up and don't attempt to prove anything about it. So option #2 with a cutoff of 100 items or so is looking like the best response. I've applied a patch along this line to 8.2 and up, and