Re: [PERFORM] Slow select performance despite seemingly reasonable query plan

2009-05-09 Thread Laurent Wandrebeck
2009/5/7 David Brain : > Hi, Hi, > > Some answers in-line: > >> >> Has there been a performance *change*, or are you just concerned about a >> query which doesn't seem to use "enough" disc bandwidth? > > Performance has degraded noticeably over the past few days. > >> Certainly random access like t

Re: [PERFORM] Slow select performance despite seemingly reasonable query plan

2009-05-07 Thread Nikolas Everett
On Thu, May 7, 2009 at 11:19 AM, Matthew Wakeling wrote: > On Thu, 7 May 2009, David Brain wrote: > >> Certainly random access like this index scan can be extremely slow. 2-4 >>> MB/s >>> is quite reasonable if you're fetching one 8kB block per disc seek - no >>> more >>> than 200 per second. >>>

Re: [PERFORM] Slow select performance despite seemingly reasonable query plan

2009-05-07 Thread Matthew Wakeling
On Thu, 7 May 2009, David Brain wrote: Certainly random access like this index scan can be extremely slow. 2-4 MB/s is quite reasonable if you're fetching one 8kB block per disc seek - no more than 200 per second. We have read ahead set pretty aggressively high as the SAN seems to 'like' this,

Re: [PERFORM] Slow select performance despite seemingly reasonable query plan

2009-05-07 Thread Nikolas Everett
> > Nested Loop Left Join (cost=0.00..6462463.96 rows=1894 width=110) > -> Append (cost=0.00..6453365.66 rows=1894 width=118) > -> Seq Scan on datatable sum (cost=0.00..10.75 rows=1 width=118) > Filter: ((datapointdate >= '2009-04-01 > 00:00:00'::timestamp without time

Re: [PERFORM] Slow select performance despite seemingly reasonable query plan

2009-05-07 Thread David Brain
Hi, Some answers in-line: > > Has there been a performance *change*, or are you just concerned about a > query which doesn't seem to use "enough" disc bandwidth? Performance has degraded noticeably over the past few days. > Certainly random access like this index scan can be extremely slow. 2-4

Re: [PERFORM] Slow select performance despite seemingly reasonable query plan

2009-05-07 Thread Matthew Wakeling
On Thu, 7 May 2009, David Brain wrote: This has been working reasonably well, however in the last few days I've been seeing extremely slow performance on what are essentially fairly simple 'index hitting' selects on this data. From the host side I see that the postgres query process is mostly in

Re: [PERFORM] Slow select performance despite seemingly reasonable query plan

2009-05-07 Thread David Brain
Hi, Interesting, for one index on one partition: idx_scan: 329 idx_tup_fetch: 8905730 So maybe a reindex would help? David. On Thu, May 7, 2009 at 10:26 AM, Scott Mead wrote: > On Thu, May 7, 2009 at 10:14 AM, David Brain wrote: >> >> Hi, >> >> Some context, we have a _lot_ of data, > 1TB, m

Re: [PERFORM] Slow select performance despite seemingly reasonable query plan

2009-05-07 Thread Scott Mead
On Thu, May 7, 2009 at 10:14 AM, David Brain wrote: > Hi, > > Some context, we have a _lot_ of data, > 1TB, mostly in 1 'table' - > the 'datatable' in the example below although in order to improve > performance this table is partitioned (by date range) into a number of > partition tables. Each

[PERFORM] Slow select performance despite seemingly reasonable query plan

2009-05-07 Thread David Brain
Hi, Some context, we have a _lot_ of data, > 1TB, mostly in 1 'table' - the 'datatable' in the example below although in order to improve performance this table is partitioned (by date range) into a number of partition tables. Each partition contains up to 20GB of data (tens of millons of rows),