[PERFORM] Checkpointing question

2011-02-15 Thread Strange, John W
During heavy writes times we get the checkpoint too often error, what's the real knock down effect of checkpointing too often? The documents don't really say what is wrong with checkpointing too often, does it cause block, io contention, etc, etc? From my understanding it's just IO contention,

[PERFORM] LIMIT on partitioned-table!?

2011-02-15 Thread Kim A. Brandt
Hello list, does `postgres (PostgreSQL) 8.4.5' use the LIMIT of a query when it is run on a partitioned-table or am I doing something wrong? It looks as if postgres queries all partitions and then LIMITing the records afterwards!? This results in a long (>3 minutes) running query. What can I d

Re: [PERFORM] LIMIT on partitioned-table!?

2011-02-15 Thread Shaun Thomas
On 02/15/2011 08:23 AM, Kim A. Brandt wrote: does `postgres (PostgreSQL) 8.4.5' use the LIMIT of a query when it is run on a partitioned-table or am I doing something wrong? It looks as if postgres queries all partitions and then LIMITing the records afterwards!? This results in a long (>3 minut

Re: [PERFORM] Checkpointing question

2011-02-15 Thread Kevin Grittner
"Strange, John W" wrote: > During heavy writes times we get the checkpoint too often error, > what's the real knock down effect of checkpointing too often? The main concern is that it may cause an increase in disk writes, possibly to the point of causing blocking while waiting for the disk.

[PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-15 Thread Thomas Pöhler
Hi list, first time for me here, hope you're not dealing too severely with me regarding guidelines. Giving my best. We are running PostgreSQL 8.4.4 on x86_64-unknown-linux-gnu, compiled by GCC gcc (Debian 4.3.2-1.1) 4.3.2, 64-bit on a Supermicro SuperServer 8026B-6RF. This version is dow

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-15 Thread Scott Marlowe
On Tue, Feb 15, 2011 at 10:19 AM, Thomas Pöhler wrote: > Since a few weeks we have really strange peaks on this system. User CPU is > increasing up to 100% and we have lots of SELECTs running. Are you using pooling of some kind, or do you have LOTS of connections? > There is no iowait at this ti

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-15 Thread Kevin Grittner
Thomas Pöhler wrote: > we have lots of SELECTs running. How many? Could you show your postgresql.conf file, with all comments removed? What does vmstat 1 (or similar) show at baseline and during your problem episodes? -Kevin -- Sent via pgsql-performance mailing list (pgsql-performance@

[PERFORM] pg_dumpall affecting performance

2011-02-15 Thread Mark Mikulec
Hello, I was under the impression that pg_dumpall didn't affect database performance when dumping while the db is live. However I have evidence to the contrary now - queries that are run during the pg_dumpall time take 10 to a 100 times longer to execute than normal while pg_dumpall is running. Th

Re: [PERFORM] pg_dumpall affecting performance

2011-02-15 Thread Plugge, Joe R.
I was always under the impression that pg_dump and pg_dumpall cause all data to be read in to the buffers and then out, (of course squeezing out whatever may be active). That is the big advantage to using PITR backups and using a tar or cpio method of backing up active containers and shipping o

Re: [PERFORM] pg_dumpall affecting performance

2011-02-15 Thread Steve Crawford
On 02/15/2011 10:41 AM, Mark Mikulec wrote: Hello, I was under the impression that pg_dumpall didn't affect database performance when dumping while the db is live. However I have evidence to the contrary now - queries that are run during the pg_dumpall time take 10 to a 100 times longer to execu

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-15 Thread Strange, John W
You have also run analyze verbose, and checked to make sure you don't have a ton of bloated indexes? - check the process with strace -p PID - check the diskIO with iostat, not vmstat - run analyze verbose, and possible reindex the database, or cluster the larger tables. - dump from pg_stat_activ

Re: [PERFORM] pg_dumpall affecting performance

2011-02-15 Thread Kevin Grittner
Mark Mikulec wrote: > The strange thing is that this started after my database grew by > about 25% after a large influx of data due to user load In addition to the issues already mentioned, there is the fact that to maintain consistency an entire database must be dumped in a single database tr

Re: [PERFORM] LIMIT on partitioned-table!?

2011-02-15 Thread Kim A. Brandt
Thank you Shaun, removing the ORDER BY worked. But I am afraid to ask this. How can I order by partition? It seams that the planner has picked a random(!?) order of partition to select from. The returned records, from the selected partition, are correctly sorted bythe index though. On 2011-02

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-15 Thread marcin mank
On Tue, Feb 15, 2011 at 6:19 PM, Thomas Pöhler wrote: > Hi list, > > See ganglia: http://dl.dropbox.com/u/183323/CPUloadprobsdb1.jpg > What is the bottom graph? queries/minute? Looks like Your database is just getting hammered. Maybe there is a really badly coded page somewhere (a query for each

Re: [PERFORM] LIMIT on partitioned-table!?

2011-02-15 Thread Marti Raudsepp
On Tue, Feb 15, 2011 at 21:33, Kim A. Brandt wrote: > removing the ORDER BY worked. But I am afraid to ask this. How can I order > by partition? It seams that the planner has picked a random(!?) order of > partition to select from. The returned records, from the selected partition, > are correctly

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-15 Thread Ivan Voras
On 15/02/2011 18:19, Thomas Pöhler wrote: Hi list, first time for me here, hope you’re not dealing too severely with me regarding guidelines. Giving my best. We are running PostgreSQL 8.4.4 on x86_64-unknown-linux-gnu, compiled by GCC gcc (Debian 4.3.2-1.1) 4.3.2, 64-bit on a Supermicro SuperSe

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-15 Thread Scott Marlowe
On Tue, Feb 15, 2011 at 6:00 PM, Ivan Voras wrote: > There is an old problem (which I've encountered so I'm replying but it may > or may not be in your case) in which PostgreSQL starts behaving badly even > for SELECT queries if the number of simultaneous queries exceeds the number > of logical CP

Re: [PERFORM] LIMIT on partitioned-table!?

2011-02-15 Thread Kim A. Brandt
Thank you Marti, I will go with the ``reduced number of matched rows'' and naturally be waiting for postgres 9.1 expectantly. Kind regards, Kim On 2011-02-15 22:13, Marti Raudsepp wrote: On Tue, Feb 15, 2011 at 21:33, Kim A. Brandt wrote: removing the ORDER BY worked. But I am afraid to

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-15 Thread Greg Smith
Kevin Grittner wrote: Could you show your postgresql.conf file, with all comments removed I just added a sample query to provide the data we always want here without people having to edit their config files, by querying pg_settings for it, to http://wiki.postgresql.org/wiki/Server_Configurat