Re: [GENERAL] Puzzling full database lock

2012-02-06 Thread Merlin Moncure
On Fri, Feb 3, 2012 at 2:55 PM, Christopher Opena counterv...@gmail.com wrote: Merlin, thanks for the response. no problem. if you're open to architecture suggestions you might also want to consider going with HS/SR and getting those large olap queries off your main database. you'll have to

Re: [GENERAL] Puzzling full database lock

2012-02-03 Thread Christopher Opena
Merlin, thanks for the response. My comments below, but firstly, does anyone know if autovacuum is affected by setting a statement_timeout? There was a long thread here from 2007'ish: http://thread.gmane.org/gmane.comp.db.postgresql.devel.general/80044/focus=93847 But it's unclear to me which

Re: [GENERAL] Puzzling full database lock

2012-02-03 Thread Tom Lane
Christopher Opena counterv...@gmail.com writes: Merlin, thanks for the response. My comments below, but firstly, does anyone know if autovacuum is affected by setting a statement_timeout? It is not; in all recent PG releases, the autovacuum processes are careful to force a session setting of

Re: [GENERAL] Puzzling full database lock

2012-02-02 Thread Merlin Moncure
On Wed, Feb 1, 2012 at 6:38 PM, Christopher Opena counterv...@gmail.com wrote: Hello folks, We've been running into some very strange issues of late with our PostgreSQL database(s).  We have an issue where a couple of queries push high CPU on a few of our processors and the entire database

Re: [GENERAL] Puzzling full database lock

2012-02-02 Thread Florian Weimer
* Christopher Opena: We've been running into some very strange issues of late with our PostgreSQL database(s). We have an issue where a couple of queries push high CPU on a few of our processors and the entire database locks (reads, writes, console cannot be achieved unless the high CPU

[GENERAL] Puzzling full database lock

2012-02-01 Thread Christopher Opena
Hello folks, We've been running into some very strange issues of late with our PostgreSQL database(s). We have an issue where a couple of queries push high CPU on a few of our processors and the entire database locks (reads, writes, console cannot be achieved unless the high CPU query procs are

Re: [GENERAL] Puzzling full database lock

2012-02-01 Thread Carlos Mennens
On Wed, Feb 1, 2012 at 7:38 PM, Christopher Opena counterv...@gmail.com wrote: Hello folks, We've been running into some very strange issues of late with our PostgreSQL database(s).  We have an issue where a couple of queries push high CPU on a few of our processors and the entire database

Re: [GENERAL] Puzzling full database lock

2012-02-01 Thread Christopher Opena
It's CentOS 5.5, PostgreSQL version 9.0.4 (x86_64). On Wed, Feb 1, 2012 at 4:44 PM, Carlos Mennens carlos.menn...@gmail.comwrote: On Wed, Feb 1, 2012 at 7:38 PM, Christopher Opena counterv...@gmail.com wrote: Hello folks, We've been running into some very strange issues of late with our

Re: [GENERAL] Puzzling full database lock

2012-02-01 Thread Carlos Mennens
On Wed, Feb 1, 2012 at 7:51 PM, Christopher Opena counterv...@gmail.com wrote: It's CentOS 5.5, PostgreSQL version 9.0.4 (x86_64). That seems extremely bleeding edge for CentOS. Did you compile this package from source RPM or some 3rd party package maintainer for PostgreSQL? -- Sent via

Re: [GENERAL] Puzzling full database lock

2012-02-01 Thread Alan Hodgson
On Wed, Feb 1, 2012 at 7:38 PM, Christopher Opena counterv...@gmail.com wrote: Hello folks, We've been running into some very strange issues of late with our PostgreSQL database(s). We have an issue where a couple of queries push high CPU on a few of our processors and the entire

Re: [GENERAL] Puzzling full database lock

2012-02-01 Thread Christopher Opena
It was installed from pgrpms.org's repository. On Wed, Feb 1, 2012 at 4:55 PM, Carlos Mennens carlos.menn...@gmail.comwrote: On Wed, Feb 1, 2012 at 7:51 PM, Christopher Opena counterv...@gmail.com wrote: It's CentOS 5.5, PostgreSQL version 9.0.4 (x86_64). That seems extremely bleeding edge

Re: [GENERAL] Puzzling full database lock

2012-02-01 Thread Ondrej Ivanič
Hi, On 2 February 2012 11:38, Christopher Opena counterv...@gmail.com wrote: We've been running into some very strange issues of late with our PostgreSQL database(s).  We have an issue where a couple of queries push high CPU on a few of our processors and the entire database locks (reads,

Re: [GENERAL] Puzzling full database lock

2012-02-01 Thread Christopher Opena
Do you mean 6-12% of total iowait, or per cpu? Our average iowait in the last week is 34.31% of a total 1600% with an average idle of 1451.76%. Our iowait *does* spike occasionally (today it went up to 148.01%) but it doesn't coincide with the lock happening. At the time of the lock we were at

Re: [GENERAL] Puzzling full database lock

2012-02-01 Thread Alan Hodgson
On Wednesday, February 01, 2012 05:13:15 PM Christopher Opena wrote: Do you mean 6-12% of total iowait, or per cpu? Our average iowait in the last week is 34.31% of a total 1600% with an average idle of 1451.76%. Our iowait *does* spike occasionally (today it went up to 148.01%) but it

Re: [GENERAL] Puzzling full database lock

2012-02-01 Thread Christopher Opena
Yeah, it's strange because we definitely have periods of high iowait but this is not when the locks are happening. If I could correlate it directly to that it would be so much easier. Thanks again for the response! On Wed, Feb 1, 2012 at 8:42 PM, Alan Hodgson ahodg...@simkin.ca wrote: On