[PERFORM] New performance documentation released

2007-05-15 Thread Greg Smith
I've been taking notes on what people ask about on this list, mixed that up with work I've been doing lately, and wrote some documentation readers of this mailing list may find useful. There are a series of articles now at http://www.westnet.com/~gsmith/content/postgresql/ about performance te

Re: [PERFORM] [PATCHES] Automatic adjustment of bgwriter_lru_maxpages

2007-05-15 Thread Greg Smith
On Tue, 15 May 2007, Jim C. Nasby wrote: Moving to -performance. No, really, moved to performance now. On Mon, May 14, 2007 at 09:55:16PM -0700, daveg wrote: What is the current thinking on bg_writer setttings for systems such as 4 core Opteron with 16GB or 32GB of memory and heavy batch wo

Re: [PERFORM] Disk Fills Up and fsck "Compresses" it

2007-05-15 Thread Jim C. Nasby
I'm guessing you're seeing the affect of softupdates. With those enabled it can take some time before the space freed by a delete will actually show up as available. On Tue, May 15, 2007 at 01:18:42PM -0700, Y Sidhu wrote: > Anyone seen PG filling up a 66 GB partition from say 40-ish percentage to

Re: [PERFORM] pg_stats how-to?

2007-05-15 Thread Jim C. Nasby
On Mon, May 14, 2007 at 08:20:49PM -0400, Tom Lane wrote: > "Y Sidhu" <[EMAIL PROTECTED]> writes: > > it may be table fragmentation. What kind of tables? We have 2 of them which > > experience lots of adds and deletes only. No updates. So a typical day > > experiences record adds a few dozen times

Re: [PERFORM] 500 requests per second

2007-05-15 Thread Jim C. Nasby
On Tue, May 15, 2007 at 11:47:29AM +0100, Richard Huxton wrote: > Tarhon-Onu Victor wrote: > >On Mon, 14 May 2007, Richard Huxton wrote: > > > >>1. Is this one client making 500 requests, or 500 clients making one > >>request per second? > > > >Up to 250 clients will make up to 500 requests pe

Re: [PERFORM] How to Run a pg_stats Query

2007-05-15 Thread Alvaro Herrera
Y Sidhu escribió: > I turned on all the stats in the conf file (below) and restarted the server. > Question is, what's the name of the database and how do I run a simple > select query? > > stats_start_collector = true > stats_command_string = true > stats_block_level = true > stats_row_level = tr

[PERFORM] How to Run a pg_stats Query

2007-05-15 Thread Y Sidhu
I turned on all the stats in the conf file (below) and restarted the server. Question is, what's the name of the database and how do I run a simple select query? stats_start_collector = true stats_command_string = true stats_block_level = true stats_row_level = true stats_reset_on_server_start =

[PERFORM] Disk Fills Up and fsck "Compresses" it

2007-05-15 Thread Y Sidhu
Anyone seen PG filling up a 66 GB partition from say 40-ish percentage to 60-ish percentage in a manner of minutes. When I run a 'fsck' the disk usage comes down to 40-ish percentage. That's about 10+ GB's variance. This is a FreeBSD 6.2 RC2, 4GB memory, Xeon 3.2 GHz '4' of the '8' CPUs in use -

Re: [PERFORM] [doc patch] a slight VACUUM / VACUUM FULL doc improvement proposal

2007-05-15 Thread Michael Stone
On Tue, May 15, 2007 at 06:43:50PM +0200, Guillaume Cottenceau wrote: patch - basically, I think the documentation under estimates (or sometimes misses) the benefit of VACUUM FULL for scans, and the needs of VACUUM FULL if the routine VACUUM hasn't been done properly since the database was put in

Re: [PERFORM] Many to many join seems slow?

2007-05-15 Thread Drew Wilson
You're right. If I redirect output to /dev/null, the query completes in 1.4s. # \o /dev/null # SELECT s.source_id, s.value as sourceValue, t.value as translationValue... ... Time: 1409.557 ms # That'll do for now. Thanks, Drew On May 15, 2007, at 7:17 AM, Heikki Linnakangas wrote: Drew

Re: [PERFORM] Many to many join seems slow?

2007-05-15 Thread Drew Wilson
Yes, I'll be filtering by string value. However, I just wanted to see how long it takes to scan all translations in a particular language. Drew On May 15, 2007, at 9:00 AM, Daniel Cristian Cruz wrote: 2007/5/15, Drew Wilson <[EMAIL PROTECTED]>: =# explain SELECT s.source_id, s.value AS sour

[PERFORM] [doc patch] a slight VACUUM / VACUUM FULL doc improvement proposal

2007-05-15 Thread Guillaume Cottenceau
Dear all, After some time spent better understanding how the VACUUM process works, what problems we had in production and how to improve our maintenance policy[1], I've come up with a little documentation patch - basically, I think the documentation under estimates (or sometimes misses) the benefi

Re: [PERFORM] Many to many join seems slow?

2007-05-15 Thread Daniel Cristian Cruz
2007/5/15, Drew Wilson <[EMAIL PROTECTED]>: =# explain SELECT s.source_id, s.value AS sourceValue, t.value AS translationValue FROM source s, translation_pair tp, translation t, language l WHERE s.source_id = tp.source_id

Re: [PERFORM] bitmap index and IS NULL predicate

2007-05-15 Thread Alexander Staubo
On 5/15/07, Jason Pinnix <[EMAIL PROTECTED]> wrote: Does the bitmap index not store a bit vector for the NULL value (i.e. a bit vector that contains a 1 for each row with a NULL value and 0 for other rows) ? You should be able to do this with a conditional index: create index ... (col) where

[PERFORM] bitmap index and IS NULL predicate

2007-05-15 Thread Jason Pinnix
Hello, I'm running version 8.2 with the bitmap index patch posted on pgsql-hackers. While selection queries with equality predicates (col = value) are able to make use of the bitmap index, those with IS NULL predicates (col IS NULL) are not able to use the bitmap index. The online manuals seem

Re: [PERFORM] Many to many join seems slow?

2007-05-15 Thread Heikki Linnakangas
Drew Wilson wrote: Merge Join (cost=524224.49..732216.29 rows=92447 width=97) (actual time=1088.871..1351.840 rows=170759 loops=1) ... Total runtime: 1366.757 ms It looks like the query actual runs in less than 3 seconds, but it takes some time to fetch 170759 rows to the client. -- Heik

Re: [PERFORM] Many to many join seems slow?

2007-05-15 Thread Drew Wilson
Please provide an EXPLAIN ANALYZE of the query. Oops, sorry about that. =# EXPLAIN ANALYZE SELECT s.source_id, s.value as sourceValue, t.value as translationValue -# FROM -# source s, -# translation_pair tp, -# translation t, -# language l -# WHERE -# s.source

Re: [PERFORM] Many to many join seems slow?

2007-05-15 Thread Alvaro Herrera
Drew Wilson escribió: > =# explain SELECT s.source_id, s.value AS sourceValue, t.value AS > translationValue > FROM > source s, > translation_pair tp, > translation t, > language l > WHERE > s.source_id = tp.source_id > AND

[PERFORM] Many to many join seems slow?

2007-05-15 Thread Drew Wilson
I'm trying to debug a query that gets all the French translations for all US string values. Ultimately, my goal is to rank them all by edit distance, and only pick the top N. However, I cannot get the basic many-to-many join to return all the results in less than 3 seconds, which seems slow

Re: [PERFORM] 500 requests per second

2007-05-15 Thread Richard Huxton
Tarhon-Onu Victor wrote: On Mon, 14 May 2007, Richard Huxton wrote: 1. Is this one client making 500 requests, or 500 clients making one request per second? Up to 250 clients will make up to 500 requests per second. Well, PG is pretty good at handling multiple clients. But if I'm under

Re: [PERFORM] 500 requests per second

2007-05-15 Thread Tarhon-Onu Victor
On Mon, 14 May 2007, Richard Huxton wrote: 1. Is this one client making 500 requests, or 500 clients making one request per second? Up to 250 clients will make up to 500 requests per second. 2. Do you expect the indexes at least to fit in RAM? not entirely... or not all of