Re: [GENERAL] reducing number of ANDs speeds up query RESOLVED

2013-01-16 Thread T. E. Lawrence
On 15.01.2013, at 16:36, Tom Lane t...@sss.pgh.pa.us wrote: T. E. Lawrence t.e.lawre...@icloud.com CAMkU=1y6UuxPYbf_ky8DVDsJi=g=uQ1t0B6kwLEtdc7NLxB_- So, apparently, we need to interrupt the heavy imports on some reasonable intervals and do manual VACUUM ANALYZE? Data import

Re: [GENERAL] reducing number of ANDs speeds up query RESOLVED

2013-01-16 Thread T. E. Lawrence
On 15.01.2013, at 17:32, Jeff Janes jeff.ja...@gmail.com wrote: T.E., Fortunately in point releases from August 2012 (9.0.9, 9.1.5, etc.), the default server log settings will log both the cancel and the command triggering the cancel. So if you are running an up to date server, you can just

Re: [GENERAL] reducing number of ANDs speeds up query RESOLVED

2013-01-15 Thread T. E. Lawrence
On 15.01.2013, at 05:45, Jeff Janes jeff.ja...@gmail.com wrote: Which makes me think that, as we grew the database more than 250 times in size over a 2-3 months period, relying on autovacuum (some tables grew from 200k to 50m records, other from 1m to 500m records), the autovacuum has

Re: [GENERAL] reducing number of ANDs speeds up query RESOLVED

2013-01-14 Thread T. E. Lawrence
RESOLVED -- Dear all, Thank you for your great help and multiple advices. I discovered the problem and I have to say that it is very stupid and strange. Here is what happened. From all advices I tried first partial index. The index was built and there was no change in the speed of the slow

Re: [GENERAL] Linux Distribution Preferences?

2013-01-14 Thread T. E. Lawrence
Hey guys, I'm not sure the last time I saw this discussion, but I was somewhat curious: what would be your ideal Linux distribution for a nice solid PostgreSQL installation? We've kinda bounced back and forth between RHEL, CentOS, and Ubuntu LTS, so I was wondering what everyone else

Re: [GENERAL] Linux Distribution Preferences?

2013-01-14 Thread T. E. Lawrence
On 15.01.2013, at 00:28, Rich Shepard rshep...@appl-ecosys.com wrote: On Tue, 15 Jan 2013, T. E. Lawrence wrote: When forced on Linux we like Debian because it is so conservative (which can sometimes drive one crazy, especially if one needs some cutting edge feature). T. Take a look

Re: [GENERAL] reducing number of ANDs speeds up query

2013-01-12 Thread T. E. Lawrence
On 12.01.2013, at 07:10, Amit kapila amit.kap...@huawei.com wrote: You can try once with below query: Select * from (SELECT a.id,b.value FROM table_a a, table_b b WHERE ... AND ... ) X where X.value=...; If this doesn't work can you send the Explain .. output for both queries(the query

Re: [GENERAL] reducing number of ANDs speeds up query

2013-01-12 Thread T. E. Lawrence
Hi and thank you for your notes! You really ought to include the output of EXPLAIN ANALYZE in cases such as these (if it doesn't already point you to the culprit). I'll do so, it takes quite long... Most likely you'll find that the last condition added a sequential scan to the query plan,

Re: [GENERAL] reducing number of ANDs speeds up query

2013-01-12 Thread T. E. Lawrence
Hi and thank you! On 12.01.2013, at 11:52, Eduardo Morras emorr...@yahoo.es wrote: With the last AND b.value=... the query is extremely slow (did not wait for it to end, but more than a minute), because the value column is not indexed (contains items longer than 8K). You can construct

[GENERAL] reducing number of ANDs speeds up query

2013-01-11 Thread T. E. Lawrence
Hello, I have a pretty standard query with two tables: SELECT table_a.id FROM table_a a, table_b b WHERE ... AND ... AND b.value=...; With the last AND b.value=... the query is extremely slow (did not wait for it to end, but more than a minute), because the value column is not indexed

[GENERAL] 9.2 streaming replication issue and solution strategy

2012-11-17 Thread T. E. Lawrence
Hello, We are running 9.2 w/ streaming replication. The slave is used for heavy tsearch based data mining. Apparently depending on the business of the master the slave queries fail with different frequency with the following message — ERROR: canceling statement due to conflict with recovery

Re: [GENERAL] 9.2 streaming replication issue and solution strategy

2012-11-17 Thread T. E. Lawrence
Have you looked at the below?: http://www.postgresql.org/docs/9.2/interactive/hot-standby.html#HOT-STANDBY-CONFLICT 25.5.2. Handling Query Conflicts Yes, thank you! I am hoping to hear more from people who have running 9.2 systems w/ between 100m and 1b records, w/ streaming replication

Re: [GENERAL] Using a GIN index on an integer array to model sets of tags

2012-11-17 Thread T. E. Lawrence
I am researching how to set up a schema for querying a set of tags associated with an object. I would be interested in hearing your conclusions. I am currently researching in a similar direction. We have streaming replication where the slaves are used for data mining, storing currently about