Re: [HACKERS] [GENERAL] Performance of full outer join in 8.3

2009-04-20 Thread Simon Riggs
On Sat, 2009-04-18 at 08:32 -0400, Tom Lane wrote: The issues that I think would be worth having tests for are questions like will the planner push comparisons to constants down through a full join? (which was the bug that started this thread). Yes, that sounds good. With a test

Re: [HACKERS] [GENERAL] Performance of full outer join in 8.3

2009-04-18 Thread Hannu Krosing
On Wed, 2009-04-15 at 18:04 -0400, Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: I've always been scared to ask this question, in case the answer is No, but: Do we have a set of regression tests for the optimizer anywhere? Nothing beyond what is in the standard tests. While

Re: [HACKERS] [GENERAL] Performance of full outer join in 8.3

2009-04-18 Thread Andrew Dunstan
Hannu Krosing wrote: On Wed, 2009-04-15 at 18:04 -0400, Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: I've always been scared to ask this question, in case the answer is No, but: Do we have a set of regression tests for the optimizer anywhere? Nothing beyond what

Re: [HACKERS] [GENERAL] Performance of full outer join in 8.3

2009-04-18 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: Hannu Krosing wrote: Can't we make first cut at it by just running with timings on and then compare ratios of running times - maybe with 2-3X tolerance - to catch most obvious regressions ? The current regression tests are a series of yes/no answers

Re: [HACKERS] [GENERAL] Performance of full outer join in 8.3

2009-04-18 Thread Greg Stark
On Sat, Apr 18, 2009 at 1:32 PM, Tom Lane t...@sss.pgh.pa.us wrote: I'm inclined to think that some sort of fuzzy examination of EXPLAIN output (in this example, are there constant-comparison conditions in the relation scans?) might do the job, but I'm not sure how we'd go about that. If we

Re: [HACKERS] [GENERAL] Performance of full outer join in 8.3

2009-04-18 Thread Tom Lane
Greg Stark st...@enterprisedb.com writes: On Sat, Apr 18, 2009 at 1:32 PM, Tom Lane t...@sss.pgh.pa.us wrote: I'm inclined to think that some sort of fuzzy examination of EXPLAIN output (in this example, are there constant-comparison conditions in the relation scans?) might do the job, but I'm

Re: [HACKERS] [GENERAL] Performance of full outer join in 8.3

2009-04-18 Thread Tino Wildenhain
Tom Lane wrote: Greg Stark st...@enterprisedb.com writes: ... I suppose if we had explain-to-a-table then we could run explain and then run an sql query to verify the specific properties we were looking for. A similar thing could be done with xml if we had powerful enough xml predicates but

Re: [HACKERS] [GENERAL] Performance of full outer join in 8.3

2009-04-17 Thread Grzegorz Jaskiewicz
Btw, There was a EXPLAIN XML summer of code project, wasn't there ? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [GENERAL] Performance of full outer join in 8.3

2009-04-16 Thread Robert Haas
2009/4/16 Simon Riggs si...@2ndquadrant.com: On Wed, 2009-04-15 at 20:58 -0400, Robert Haas wrote: On Wed, Apr 15, 2009 at 7:39 PM, Tom Lane t...@sss.pgh.pa.us wrote: The output of EXPLAIN is nowhere near stable enough to use within the current exact-match regression test framework.  I'm not

Re: [HACKERS] [GENERAL] Performance of full outer join in 8.3

2009-04-16 Thread David Fetter
On Thu, Apr 16, 2009 at 06:12:10AM +0100, Simon Riggs wrote: EXPLAIN (option_name, ...) query Or maybe: EXPLAIN (option_name = value, ...) query It may or may not be the case that generating a useful regression test suite for the planner is too much work for anyone to bother,

Re: [HACKERS] [GENERAL] Performance of full outer join in 8.3

2009-04-16 Thread Robert Haas
On Thu, Apr 16, 2009 at 11:21 AM, David Fetter da...@fetter.org wrote: If we're going with a serialization, which I think would be an excellent idea, how about one that's light-weight and human-readable like JSON? Wow, that's a great idea for another option to EXPLAIN. Wouldn't it be nice if

Re: [HACKERS] [GENERAL] Performance of full outer join in 8.3

2009-04-16 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: I think XML explain output is a good idea, but I don't think it's a substitute for better options to control the human-readable form. Yeah. I think a well-designed XML output format for EXPLAIN is a fine thing to work on, but I don't believe it would

Re: [HACKERS] [GENERAL] Performance of full outer join in 8.3

2009-04-16 Thread Grzegorz Jaskiewicz
On 16 Apr 2009, at 16:21, David Fetter wrote: On Thu, Apr 16, 2009 at 06:12:10AM +0100, Simon Riggs wrote: I think the way to do this is to introduce plan output in XML If we're going with a serialization, which I think would be an excellent idea, how about one that's light-weight and

Re: [HACKERS] [GENERAL] Performance of full outer join in 8.3

2009-04-16 Thread Merlin Moncure
On Thu, Apr 16, 2009 at 2:04 PM, Grzegorz Jaskiewicz g...@pointblue.com.pl wrote: On 16 Apr 2009, at 16:21, David Fetter wrote: On Thu, Apr 16, 2009 at 06:12:10AM +0100, Simon Riggs wrote: I think the way to do this is to introduce plan output in XML If we're going with a serialization,

Re: [HACKERS] [GENERAL] Performance of full outer join in 8.3

2009-04-16 Thread Grzegorz Jaskiewicz
On 16 Apr 2009, at 19:41, Merlin Moncure wrote: Is that because of how the output is formatted though, or because the concepts are difficult to express? (I agree though, json is better especially for structures that are possibly highly nested). What I mean is that what postgresql displays

Re: [HACKERS] [GENERAL] Performance of full outer join in 8.3

2009-04-15 Thread Tom Lane
I wrote: =?ISO-8859-1?Q?Christian_Schr=F6der?= c...@deriva.de writes: This is the query: select isin from ts_frontend.attachment_isins full OUTER JOIN ts_frontend.rec_isins using (attachment,isin) WHERE attachment=2698120 GROUP BY isin limit 1000; Hmm. It seems 8.3 is failing to push

Re: [HACKERS] [GENERAL] Performance of full outer join in 8.3

2009-04-15 Thread Tom Lane
I wrote: On further review, this did work in 8.3 when released. I think it got broken here: http://archives.postgresql.org/pgsql-committers/2008-06/msg00336.php because that change is preventing the mergedvar = constant clause from being seen as an equivalence, when it should be seen as one.

Re: [HACKERS] [GENERAL] Performance of full outer join in 8.3

2009-04-15 Thread Simon Riggs
On Wed, 2009-04-15 at 12:34 -0400, Tom Lane wrote: On further review, this did work in 8.3 when released. I think it got broken here: http://archives.postgresql.org/pgsql-committers/2008-06/msg00336.php because that change is preventing the mergedvar = constant clause from being seen

Re: [HACKERS] [GENERAL] Performance of full outer join in 8.3

2009-04-15 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: I've always been scared to ask this question, in case the answer is No, but: Do we have a set of regression tests for the optimizer anywhere? Nothing beyond what is in the standard tests. While that's okay at catching wrong answers --- and we have

Re: [HACKERS] [GENERAL] Performance of full outer join in 8.3

2009-04-15 Thread Robert Haas
On Wed, Apr 15, 2009 at 6:04 PM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: I've always been scared to ask this question, in case the answer is No, but: Do we have a set of regression tests for the optimizer anywhere? Nothing beyond what is in the standard

Re: [HACKERS] [GENERAL] Performance of full outer join in 8.3

2009-04-15 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: We could add some regression tests that create a sample data set, ANALYZE it, and then EXPLAIN various things. The results should be deterministic, Sorry, you're wrong. The output of EXPLAIN is nowhere near stable enough to use within the current

Re: [HACKERS] [GENERAL] Performance of full outer join in 8.3

2009-04-15 Thread Robert Haas
On Wed, Apr 15, 2009 at 7:39 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: We could add some regression tests that create a sample data set, ANALYZE it, and then EXPLAIN various things.  The results should be deterministic, Sorry, you're wrong. The output

Re: [HACKERS] [GENERAL] Performance of full outer join in 8.3

2009-04-15 Thread Simon Riggs
On Wed, 2009-04-15 at 20:58 -0400, Robert Haas wrote: On Wed, Apr 15, 2009 at 7:39 PM, Tom Lane t...@sss.pgh.pa.us wrote: The output of EXPLAIN is nowhere near stable enough to use within the current exact-match regression test framework. I'm not sure it would be stable even if we