Re: [PERFORM] Any better plan for this query?..

2009-05-19 Thread Robert Haas
On Tue, May 19, 2009 at 6:49 PM, Tom Lane wrote: > Simon Riggs writes: >> On Tue, 2009-05-19 at 08:58 -0400, Tom Lane wrote: >>> Nonsense.  The planner might think some other plan is cheaper, but >>> it definitely knows how to do this, and has since at least 8.1. > >> Please look at Dimitri's pla

Re: [PERFORM] Any better plan for this query?..

2009-05-19 Thread Tom Lane
Simon Riggs writes: > On Tue, 2009-05-19 at 08:58 -0400, Tom Lane wrote: >> Nonsense. The planner might think some other plan is cheaper, but >> it definitely knows how to do this, and has since at least 8.1. > Please look at Dimitri's plan. If it can remove the pointless sort, why > does it not

Re: [PERFORM] Any better plan for this query?..

2009-05-19 Thread Merlin Moncure
On Tue, May 19, 2009 at 3:15 PM, Dimitri wrote: > On 5/19/09, Merlin Moncure wrote: >> On Tue, May 19, 2009 at 11:53 AM, Dimitri wrote: >>> the query is *once* prepared via PQexec, >>> then it's looping with "execute" via PQexec. >>> Why PQexecPrepared will be better in my case?.. >> >> It can b

Re: [PERFORM] Any better plan for this query?..

2009-05-19 Thread Dimitri
On 5/19/09, Merlin Moncure wrote: > On Tue, May 19, 2009 at 11:53 AM, Dimitri wrote: >> On 5/19/09, Merlin Moncure wrote: >>> On Mon, May 18, 2009 at 6:32 PM, Dimitri wrote: Thanks Dave for correction, but I'm also curious where the time is wasted in this case?.. 0.84ms is d

Re: [PERFORM] Any better plan for this query?..

2009-05-19 Thread Dimitri
On 5/19/09, Scott Carey wrote: > > On 5/19/09 3:46 AM, "Dimitri" wrote: > >> On 5/19/09, Scott Carey wrote: >>> >>> On 5/18/09 3:32 PM, "Dimitri" wrote: >>> On 5/18/09, Scott Carey wrote: > Great data Dimitri!' Thank you! :-) > > I see a few key trends in the po

Re: [PERFORM] Any better plan for this query?..

2009-05-19 Thread Merlin Moncure
On Tue, May 19, 2009 at 11:53 AM, Dimitri wrote: > On 5/19/09, Merlin Moncure wrote: >> On Mon, May 18, 2009 at 6:32 PM, Dimitri wrote: >>> Thanks Dave for correction, but I'm also curious where the time is >>> wasted in this case?.. >>> >>> 0.84ms is displayed by "psql" once the result output i

Re: [PERFORM] Any better plan for this query?..

2009-05-19 Thread Scott Carey
On 5/19/09 3:46 AM, "Dimitri" wrote: > On 5/19/09, Scott Carey wrote: >> >> On 5/18/09 3:32 PM, "Dimitri" wrote: >> >>> On 5/18/09, Scott Carey wrote: Great data Dimitri!' >>> >>> Thank you! :-) >>> I see a few key trends in the poor scalability: The throughput

Re: [PERFORM] Any better plan for this query?..

2009-05-19 Thread Scott Carey
On 5/19/09 5:01 AM, "Matthew Wakeling" wrote: > On Tue, 19 May 2009, Simon Riggs wrote: >>> Speaking of avoiding large sorts, I'd like to push again for partial >>> sorts. This is the situation where an index provides data sorted by >>> column "a", and the query requests data sorted by "a, b". C

Re: [PERFORM] Any better plan for this query?..

2009-05-19 Thread Simon Riggs
On Tue, 2009-05-19 at 13:01 +0100, Matthew Wakeling wrote: > That leads me on to another topic. Consider the query: > > SELECT * FROM table ORDER BY a, b > > where the column "a" is declared UNIQUE and has an index. Does Postgres > eliminate "b" from the ORDER BY, and therefore allow fetching

Re: [PERFORM] Any better plan for this query?..

2009-05-19 Thread Simon Riggs
On Tue, 2009-05-19 at 08:58 -0400, Tom Lane wrote: > Simon Riggs writes: > > Both plans for this query show an IndexScan on a two column-index, with > > an Index Condition of equality on the leading column. The ORDER BY > > specifies a sort by the second index column, so the top-level Sort is > >

Re: [PERFORM] Any better plan for this query?..

2009-05-19 Thread Dimitri
On 5/19/09, Merlin Moncure wrote: > On Mon, May 18, 2009 at 6:32 PM, Dimitri wrote: >> Thanks Dave for correction, but I'm also curious where the time is >> wasted in this case?.. >> >> 0.84ms is displayed by "psql" once the result output is printed, and I >> got similar time within my client (us

Re: [PERFORM] Any better plan for this query?..

2009-05-19 Thread Dimitri
The response time is not progressive, it's simply jumping, it's likely since 16 sessions there is a sort of serialization happening somewhere.. As well on 16 sessions the throughput in TPS is near the same as on 8 (response time is only twice bigger for the moment), but on 32 it's dramatically drop

Re: [PERFORM] Any better plan for this query?..

2009-05-19 Thread Robert Haas
On May 19, 2009, at 7:36 AM, Simon Riggs wrote: On Tue, 2009-05-19 at 12:17 +0100, Matthew Wakeling wrote: Yes, Postgres has been missing the boat on this one for a while. +1 on requesting this feature. That's an optimizer feature. Speaking of avoiding large sorts, I'd like to push agai

Re: [PERFORM] Any better plan for this query?..

2009-05-19 Thread Merlin Moncure
On Mon, May 18, 2009 at 6:32 PM, Dimitri wrote: > Thanks Dave for correction, but I'm also curious where the time is > wasted in this case?.. > > 0.84ms is displayed by "psql" once the result output is printed, and I > got similar time within my client (using libpq) which is not printing > any out

Re: [PERFORM] Any better plan for this query?..

2009-05-19 Thread Simon Riggs
On Tue, 2009-05-19 at 14:00 +0200, Dimitri wrote: > I may confirm the issue with hash join - it's repeating both with > prepared and not prepared statements - it's curious because initially > the response time is lowering near ~1ms (the lowest seen until now) > and then once workload growing to 1

Re: [PERFORM] Any better plan for this query?..

2009-05-19 Thread Tom Lane
Simon Riggs writes: > Both plans for this query show an IndexScan on a two column-index, with > an Index Condition of equality on the leading column. The ORDER BY > specifies a sort by the second index column, so the top-level Sort is > superfluous in this case. > My understanding is that we don'

Re: [PERFORM] Any better plan for this query?..

2009-05-19 Thread Matthew Wakeling
On Tue, 19 May 2009, Simon Riggs wrote: Speaking of avoiding large sorts, I'd like to push again for partial sorts. This is the situation where an index provides data sorted by column "a", and the query requests data sorted by "a, b". Currently, Postgres sorts the entire data set, whereas it need

Re: [PERFORM] Any better plan for this query?..

2009-05-19 Thread Dimitri
I may confirm the issue with hash join - it's repeating both with prepared and not prepared statements - it's curious because initially the response time is lowering near ~1ms (the lowest seen until now) and then once workload growing to 16 sessions it's jumping to 2.5ms, then with 32 sessions it's

Re: [PERFORM] Any better plan for this query?..

2009-05-19 Thread Simon Riggs
On Tue, 2009-05-19 at 12:36 +0100, Simon Riggs wrote: > Partially sorted data takes much less effort to sort (OK, not zero, I > grant) so this seems like a high complexity, lower value feature. I > agree it should be on the TODO, just IMHO at a lower priority than some > other features. Perhaps

Re: [PERFORM] Any better plan for this query?..

2009-05-19 Thread Simon Riggs
On Tue, 2009-05-19 at 12:17 +0100, Matthew Wakeling wrote: > Yes, Postgres has been missing the boat on this one for a while. +1 on > requesting this feature. That's an optimizer feature. > Speaking of avoiding large sorts, I'd like to push again for partial > sorts. This is the situation where

Re: [PERFORM] Any better plan for this query?..

2009-05-19 Thread Matthew Wakeling
On Tue, 19 May 2009, Simon Riggs wrote: Both plans for this query show an IndexScan on a two column-index, with an Index Condition of equality on the leading column. The ORDER BY specifies a sort by the second index column, so the top-level Sort is superfluous in this case. My understanding is t

Re: [PERFORM] Any better plan for this query?..

2009-05-19 Thread Simon Riggs
On Mon, 2009-05-18 at 19:00 -0400, Tom Lane wrote: > Simon Riggs writes: > > In particular, running the tests repeatedly using > > H.REF_OBJECT = '01' > > rather than varying the value seems likely to benefit MySQL. One thing to note in terms of optimisation of this query is that we

Re: [PERFORM] Any better plan for this query?..

2009-05-19 Thread Dimitri
On 5/19/09, Simon Riggs wrote: > > On Tue, 2009-05-19 at 00:33 +0200, Dimitri wrote: >> > >> > In particular, running the tests repeatedly using >> >H.REF_OBJECT = '01' >> > rather than varying the value seems likely to benefit MySQL. The >> >> let me repeat again - the reference is *r

Re: [PERFORM] Any better plan for this query?..

2009-05-19 Thread Dimitri
On 5/19/09, Scott Carey wrote: > > On 5/18/09 3:32 PM, "Dimitri" wrote: > >> On 5/18/09, Scott Carey wrote: >>> Great data Dimitri!' >> >> Thank you! :-) >> >>> >>> I see a few key trends in the poor scalability: >>> >>> The throughput scales roughly with %CPU fairly well. But CPU used >>> does

Re: [PERFORM] Any better plan for this query?..

2009-05-19 Thread Dimitri
No, Tom, the query cache was off. I put it always explicitly off on MySQL as it has scalability issues. Rgds, -Dimitri On 5/19/09, Tom Lane wrote: > Simon Riggs writes: >> In particular, running the tests repeatedly using >> H.REF_OBJECT = '01' >> rather than varying the value see

Re: [PERFORM] Any better plan for this query?..

2009-05-19 Thread Simon Riggs
On Tue, 2009-05-19 at 00:33 +0200, Dimitri wrote: > > > > In particular, running the tests repeatedly using > > H.REF_OBJECT = '01' > > rather than varying the value seems likely to benefit MySQL. The > > let me repeat again - the reference is *random*, > the '01' value I've u