Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-17 Thread Strange, John W
Sent: 16 February 2011 15:43 To: Marti Raudsepp Cc: Thomas Pöhler; pgsql-performance@postgresql.org; Felix Feinhals; Verteiler_A-Team; Björn Metzdorf Subject: Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem On Wed, Feb 16, 2011 at 6:44 AM, Marti Raudsepp wrote: > On Tue,

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-17 Thread Justin Pitts
I think adding UNION ALL SELECT 'postgres version', version(); might be a good thing. On Wed, Feb 16, 2011 at 9:55 AM, Greg Smith wrote: > Kevin Grittner wrote: >> >> In fact, I wonder whether we shouldn't leave a couple items you've >> excluded, since they are sometimes germane to problems pos

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-17 Thread Pierre C
Thomas Pöhler wrote: I remember you said you were using nginx and php-fastcgi, how many web server boxes do you have, and what are the specs ? -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpre

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-16 Thread Greg Smith
Thomas Pöhler wrote: We are running a biweekly downtime where we do a complete reindex and vaccum full. We cannot identify certain queries causing this. If you feel that you need VACUUM FULL, either something terribly wrong has happened, or someone has gotten confused. In both cases it's un

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-16 Thread Kevin Grittner
Thomas Pöhler wrote: > we are using two instances of pgbouncer v1.4 for connection > pooling. One for prepared statements (pool_mode session) and one > without (pool_mode transaction). > max_client_conn = 1 > default_pool_size = 450 Your best defense against the "thundering herd" issues

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-16 Thread Cédric Villemain
27;s of connection via pgbouncer, then moving the bouncer in a 3rd box salve the situation. > > Appreciate your help! > Thomas > > -Ursprüngliche Nachricht- > Von: Kevin Grittner [mailto:kevin.gritt...@wicourts.gov] > Gesendet: Mittwoch, 16. Februar 2011 17:09 >

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-16 Thread Thomas Pöhler
ls; Thomas Pöhler Betreff: Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem Justin Pitts wrote: > I think adding > > UNION ALL SELECT 'postgres version', version(); > > might be a good thing. Good point. Added. > Greg Smith wrote: >> Kev

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-16 Thread Kevin Grittner
Justin Pitts wrote: > I think adding > > UNION ALL SELECT 'postgres version', version(); > > might be a good thing. Good point. Added. > Greg Smith wrote: >> Kevin Grittner wrote: >>> >>> In fact, I wonder whether we shouldn't leave a couple items >>> you've excluded, since they are somet

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-16 Thread Scott Marlowe
e@postgresql.org; Felix Feinhals; > Verteiler_A-Team; Björn Metzdorf > Subject: Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem > > On Wed, Feb 16, 2011 at 6:44 AM, Marti Raudsepp wrote: >> On Tue, Feb 15, 2011 at 20:01, Scott Marlowe wrote: >>> run

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-16 Thread Scott Marlowe
On Wed, Feb 16, 2011 at 6:44 AM, Marti Raudsepp wrote: > On Tue, Feb 15, 2011 at 20:01, Scott Marlowe wrote: >> run htop and look for red.  if youi've got lots of red bar on each CPU >> but no io wait then it's waiting for memory access. > > I don't think this is true. AFAICT the red bar refers t

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-16 Thread Greg Smith
Kevin Grittner wrote: In fact, I wonder whether we shouldn't leave a couple items you've excluded, since they are sometimes germane to problems posted, like lc_collate and TimeZone. I pulled some of them out only because they're not really postgresql.conf settings; lc_collate and lc_ctype for

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-16 Thread Kevin Grittner
Greg Smith wrote: > I just added a sample query to provide the data we always want > here without people having to edit their config files, by > querying pg_settings for it, to > http://wiki.postgresql.org/wiki/Server_Configuration Nice! Thanks! A few very nice things about this: (1) You

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-16 Thread Marti Raudsepp
On Tue, Feb 15, 2011 at 20:01, Scott Marlowe wrote: > run htop and look for red.  if youi've got lots of red bar on each CPU > but no io wait then it's waiting for memory access. I don't think this is true. AFAICT the red bar refers to "system time", time that's spent in the kernel -- either in s

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-15 Thread Greg Smith
Kevin Grittner wrote: Could you show your postgresql.conf file, with all comments removed I just added a sample query to provide the data we always want here without people having to edit their config files, by querying pg_settings for it, to http://wiki.postgresql.org/wiki/Server_Configurat

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-15 Thread Scott Marlowe
On Tue, Feb 15, 2011 at 6:00 PM, Ivan Voras wrote: > There is an old problem (which I've encountered so I'm replying but it may > or may not be in your case) in which PostgreSQL starts behaving badly even > for SELECT queries if the number of simultaneous queries exceeds the number > of logical CP

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-15 Thread Ivan Voras
On 15/02/2011 18:19, Thomas Pöhler wrote: Hi list, first time for me here, hope you’re not dealing too severely with me regarding guidelines. Giving my best. We are running PostgreSQL 8.4.4 on x86_64-unknown-linux-gnu, compiled by GCC gcc (Debian 4.3.2-1.1) 4.3.2, 64-bit on a Supermicro SuperSe

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-15 Thread marcin mank
On Tue, Feb 15, 2011 at 6:19 PM, Thomas Pöhler wrote: > Hi list, > > See ganglia: http://dl.dropbox.com/u/183323/CPUloadprobsdb1.jpg > What is the bottom graph? queries/minute? Looks like Your database is just getting hammered. Maybe there is a really badly coded page somewhere (a query for each

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-15 Thread Strange, John W
lf Of Thomas Pöhler Sent: 15 February 2011 17:19 To: pgsql-performance@postgresql.org Cc: Felix Feinhals; Verteiler_A-Team; Björn Metzdorf Subject: [PERFORM] high user cpu, massive SELECTs, no io waiting problem Hi list, first time for me here, hope you're not dealing too severely with me

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-15 Thread Kevin Grittner
Thomas Pöhler wrote: > we have lots of SELECTs running. How many? Could you show your postgresql.conf file, with all comments removed? What does vmstat 1 (or similar) show at baseline and during your problem episodes? -Kevin -- Sent via pgsql-performance mailing list (pgsql-performance@

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-15 Thread Scott Marlowe
On Tue, Feb 15, 2011 at 10:19 AM, Thomas Pöhler wrote: > Since a few weeks we have really strange peaks on this system. User CPU is > increasing up to 100% and we have lots of SELECTs running. Are you using pooling of some kind, or do you have LOTS of connections? > There is no iowait at this ti

[PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-15 Thread Thomas Pöhler
Hi list, first time for me here, hope you're not dealing too severely with me regarding guidelines. Giving my best. We are running PostgreSQL 8.4.4 on x86_64-unknown-linux-gnu, compiled by GCC gcc (Debian 4.3.2-1.1) 4.3.2, 64-bit on a Supermicro SuperServer 8026B-6RF. This version is dow