[PERFORM] bad plan and LIMIT

2009-05-01 Thread James Nelson
Hi, I'm hoping you guys can help with improving this query I'm having a problem with. The main problem is that the query plan changes depending on the value of the LIMIT clause, with small values using a poor plan and running very slowly. The two times are roughly 5 minutes for the bad

Re: [PERFORM] bad plan and LIMIT

2009-05-01 Thread Adam Ruth
You could try changing the IN to an EXISTS, that may alter how the optimizer weighs the limit. SELECT ID FROM ps_image WHERE EXISTS (SELECT null FROM ps_gallery_image WHERE gallery_id ='G7ejKGoWS_cY' and image_id = ps_image.id) ORDER BY LOWER(FILE_NAME) ASC On 30/04/2009, at 3:51 AM,

Re: [PERFORM] bad plan and LIMIT

2009-05-01 Thread Grzegorz Jaśkiewicz
EXISTS won't help much either, postgresql is not too fast, when it comes to that sort of approach. join is always going to be fast, it is about time you learn joins and use them ;) -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription:

Re: [PERFORM] bad plan and LIMIT

2009-05-01 Thread Grzegorz Jaśkiewicz
use join instead of where in(); -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

[PERFORM] Many left outer joins with limit performance

2009-05-01 Thread Gerhard Wiesinger
Hello, I want to use postgresql for data entries (every minute) from a central heating system where the timestamp is logged in a table log. For flexibility in the future for future values and for implementing several high level types I've modelled the values in a separate key/value table

[PERFORM] Transparent table partitioning in future version of PG?

2009-05-01 Thread henk de wit
Hi, I was looking at the support that PostgreSQL offers for table partitioning at http://www.postgresql.org/docs/8.4/static/ddl-partitioning.html. The concept looks promising, but its maybe fair to say that PG itself doesn't really supports partitioning natively, but one can simulate it using

Re: [PERFORM] bad plan and LIMIT

2009-05-01 Thread Tom Lane
James Nelson ja...@photoshelter.com writes: Hi, I'm hoping you guys can help with improving this query I'm having a problem with. The main problem is that the query plan changes depending on the value of the LIMIT clause, with small values using a poor plan and running very slowly. The

Re: [PERFORM] bad plan and LIMIT

2009-05-01 Thread James Nelson
I had tried using exists but both the forms of the query (with limit and without) performed much worse. James On May 1, 2009, at 4:22 AM, Adam Ruth wrote: You could try changing the IN to an EXISTS, that may alter how the optimizer weighs the limit. SELECT ID FROM ps_image WHERE

Re: [PERFORM] bad plan and LIMIT

2009-05-01 Thread James Nelson
The 'in' form and 'join' form produce identical plans for both limit and non-limit versions of the query, which I actually think reflects well on the query planner. I also tried a form of the query with the subselect in the from clause to try and force the order the tables were evaluated

Re: [PERFORM] bad plan and LIMIT

2009-05-01 Thread James Nelson
I looked into the distribution of the filenames, in particular I ran a query to see how for into the table the 1st filename would be found. photoshelter=# select count(*) from ps_image where lower(file_name) 'a-400-001.jpg'; count - 8915832 As you can see the first row is

Re: [PERFORM] Many left outer joins with limit performance

2009-05-01 Thread Tom Lane
Gerhard Wiesinger li...@wiesinger.com writes: FROM log l -- Order is relevant here LEFT OUTER JOIN key_description k1 ON k1.description = 'Raumsolltemperatur' LEFT OUTER JOIN log_details d1 ON l.id = d1.fk_id AND d1.fk_keyid = k1.keyid Surely this query is just plain broken?

Re: [PERFORM] Transparent table partitioning in future version of PG?

2009-05-01 Thread Scott Carey
On 5/1/09 7:32 AM, henk de wit henk53...@hotmail.com wrote: Hi, I was looking at the support that PostgreSQL offers for table partitioning at http://www.postgresql.org/docs/8.4/static/ddl-partitioning.html. The concept looks promising, but its maybe fair to say that PG itself doesn't

Re: [PERFORM] performance for high-volume log insertion

2009-05-01 Thread PFC
Blocking round trips to another process on the same server should be fairly cheap--that is, writing to a socket (or pipe, or localhost TCP connection) where the other side is listening for it; and then blocking in return for the response. The act of writing to an FD that another process is

Re: [PERFORM] performance for high-volume log insertion

2009-05-01 Thread david
On Sat, 2 May 2009, PFC wrote: Blocking round trips to another process on the same server should be fairly cheap--that is, writing to a socket (or pipe, or localhost TCP connection) where the other side is listening for it; and then blocking in return for the response. The act of writing to an

Re: [PERFORM] performance for high-volume log insertion

2009-05-01 Thread Glenn Maynard
On Fri, May 1, 2009 at 8:29 PM, PFC li...@peufeu.com wrote:        Roundtrips can be quite fast but they have a hidden problem, which is that everything gets serialized. The client and server will serialize, but what usually matters most is avoiding serializing against disk I/O--and that's why