Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-10-09 Thread Merlin Moncure
On Fri, Oct 9, 2009 at 3:11 AM, Shiva Raman wrote: > Dear all >   with reference to the discussions and valuable suggestions i got from the > list, the code has been reviewed and updated with explicit commit . There is > a good improvement in  performance .I am also planning to upgrade the > datab

Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-10-09 Thread Shiva Raman
Dear all with reference to the discussions and valuable suggestions i got from the list, the code has been reviewed and updated with explicit commit . There is a good improvement in performance .I am also planning to upgrade the database from 8.1 to 8.3 /8.4 . My current OS is SLES 10 SP3 def

Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-09-25 Thread Craig James
Gerhard Wiesinger wrote: Hello Craig, Are you sure this is correct? The test program (see below) with autocommit=0 counts up when an insert is done in another session and there is no commit done. I think with each new select a new implicit transaction is done when no explicit "BEGIN" has be

Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-09-25 Thread Gerhard Wiesinger
Hello Craig, Are you sure this is correct? The test program (see below) with autocommit=0 counts up when an insert is done in another session and there is no commit done. I think with each new select a new implicit transaction is done when no explicit "BEGIN" has been established. Can one

Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-09-25 Thread Shiva Raman
As suggested, i had changed the log_statement='ddl' and now it is logging only the ddl statements . thanks for the tip. Can i delete the old log files in pg_log after backing up as zip archive ? is it neccesary to keep those log files ? Regards Shiva Raman > > 2009/9/25 Grzegorz Jaśkiewicz > >

Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-09-25 Thread Grzegorz Jaśkiewicz
2009/9/25 Shiva Raman > As suggested, i had changed the log_statement='ddl' and now it is logging > only > the ddl statements . thanks for the tip. > Can i delete the old log files in pg_log after backing up as zip archive ? > is it neccesary to keep those log files ? > they're yours, you can d

Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-09-25 Thread Grzegorz Jaśkiewicz
On Fri, Sep 25, 2009 at 9:06 AM, Shiva Raman wrote: > Hi Gerhard > I also found the pg_log has 73 G of data . > > clusternode2:/var/lib/pgsql/data # du -sh pg_log/ > 73G pg_log/ > > Is it necessary to keep this Log files? Can i backup the logs and delete it > from the original directory ? Is

Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-09-25 Thread Shiva Raman
Hi Gerhard I also found the pg_log has 73 G of data . clusternode2:/var/lib/pgsql/data # du -sh pg_log/ 73G pg_log/ Is it necessary to keep this Log files? Can i backup the logs and delete it from the original directory ? Is this logs files necessary in case any data recovery to be done ? I

Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-09-25 Thread Shiva Raman
Hi Gerhard Thanks for the mail On Thu, Sep 24, 2009 at 7:19 PM, Gerhard Wiesinger wrote: > Hello Shiva, > > What I see from top (0.0%wa) you don't have any I/O problem but a major CPU > problem. But this is contrast to iostat where up to 50% of iowait is there > (sometimes). > > I think you ha

Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-09-24 Thread Craig James
Dave Dutcher wrote: You need a COMMIT for every BEGIN. If you just run a SELECT statement without first beginning a transaction, then you should not end up with a connection that is Idle in Transaction. If you are beginning a transaction, doing a select, and then not committing, then yes that i

Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-09-24 Thread Karl Denninger
Dave Dutcher wrote: >> From: Shiva Raman >> Subject: Re: [PERFORM] High CPU load on Postgres Server during Peak >> > not explicitly committed. > >> We have started updating the code on this. >> > > You need a COMMIT for every BEGIN. If you just run a SELECT statement > without first

Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-09-24 Thread Dave Dutcher
>From: Shiva Raman >Subject: Re: [PERFORM] High CPU load on Postgres Server during Peak times > >Andy Colson Wrote : , >>Eww. I think that's bad. A connection that has a transaction open will cause lots of row versions, >>which use up ram, and make it slo

Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-09-24 Thread Karl Denninger
Andy Colson wrote: > Shiva Raman wrote: >> Hi >> >> Today the load observed very high load . I am pasting the top. >> >> _*TOP *_ >> top - 12:45:23 up 79 days, 14:42, 1 user, load average: 45.84, >> 33.13, 25.84 >> Tasks: 394 total, 48 running, 346 sleeping, 0 stopped, 0 zombie >> Cpu(s): 49

Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-09-24 Thread Andy Colson
Shiva Raman wrote: Hi Today the load observed very high load . I am pasting the top. _*TOP *_ top - 12:45:23 up 79 days, 14:42, 1 user, load average: 45.84, 33.13, 25.84 Tasks: 394 total, 48 running, 346 sleeping, 0 stopped, 0 zombie Cpu(s): 49.2%us, 0.8%sy, 0.0%ni, 0.0%id, 0.0%wa,

Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-09-24 Thread Praveen DS
For 'idle in transaction' issues, you have to fix your code. I faced this issue couple of months back. How good is your exception handling? Are you rollingback/comitting your transactions while exceptions are thrown, during the course of db operations? Honestly I wouldn't go for these scripts w

Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-09-24 Thread Shiva Raman
Hi Today the load observed very high load . I am pasting the top. *TOP * top - 12:45:23 up 79 days, 14:42, 1 user, load average: 45.84, 33.13, 25.84 Tasks: 394 total, 48 running, 346 sleeping, 0 stopped, 0 zombie Cpu(s): 49.2%us, 0.8%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.1%si, 50.0%s

Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-09-23 Thread Scott Marlowe
On Wed, Sep 23, 2009 at 12:25 PM, Shiva Raman wrote: First let me say that upgrading to a later version is likely going to help as much as anything else you're likely to pick up from this discussion. Not that this discussion isn't worthwhile, it is. > If you run a 'ps ax|grep post' do you see a

Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-09-23 Thread Karl Denninger
Fernando Hevia wrote: User Access Total Number of Users is 500 Maximum number of Concurrent users will be 500 during peak time Off Peak time the maximum number of concurrent user will be around 150 to 200. >>> A connection pooler like pgpool or pgbouncer

Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-09-23 Thread Fernando Hevia
>>> >>> User Access >>> Total Number of Users is 500 >>> Maximum number of Concurrent users will be 500 during peak time >>> Off Peak time the maximum number of concurrent user will be >>> around 150 to 200. >>> >> >>A connection pooler like pgpool or pgbouncer would considerably reduce the >>burde

Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-09-23 Thread Andy Colson
Shiva Raman wrote: /If you run a 'ps ax|grep post' do you see anything that says 'idle in transaction'? (I hope that old of version will show it. my processes show up as postgres not postmaster)/ Lots of requests shows as 'idle in transaction'. Eww. I think that's bad. A connection that

Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-09-23 Thread Shiva Raman
Hi Thanks for your mail. *Some quick advice:* * * *>* *> clusternode2:~ # rpm -qa | grep postgres* *> postgresql-devel-8.1.9-1.2* *> postgresql-8.1.9-1.2* *> postgresql-docs-8.1.9-1.2* *> postgresql-server-8.1.9-1.2* *> postgresql-libs-64bit-8.1.9-1.2* *> postgresql-libs-8.1.9-1.2* *> p

Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-09-23 Thread Shiva Raman
Hi Thanks a lot for the reply. *I see you are on a pretty old version of pg. Are you vacuuming regularly?* Yes, Vaccuuming is done every day morning at 06 am It is running perfectly fine. * * *If you run a 'ps ax|grep post' do you see anything that says 'idle in transaction'? (I hope t

Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-09-22 Thread Fernando Hevia
> -Mensaje original- > De: Shiva Raman > Enviado el: Martes, 22 de Septiembre de 2009 10:55 > Para: pgsql-performance@postgresql.org > Asunto: [PERFORM] High CPU load on Postgres Server during > Peak times > > Dear all > > I am having a problem of

Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-09-22 Thread Andy Colson
Andy Colson wrote: Shiva Raman wrote: Dear all I am having a problem of high cpu loads in my postgres server during peak time. Following are the details of my setup (details as per the postgres wiki) . *Following is the output of TOP command during offpeak time.* top - 18:36:56 up 77 da

Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-09-22 Thread Andy Colson
Shiva Raman wrote: Dear all I am having a problem of high cpu loads in my postgres server during peak time. Following are the details of my setup (details as per the postgres wiki) . *Following is the output of TOP command during offpeak time.* top - 18:36:56 up 77 days, 20:33, 1 user,

Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-09-22 Thread Merlin Moncure
On Tue, Sep 22, 2009 at 9:54 AM, Shiva Raman wrote: > Dear all > >   I am having a problem of high cpu loads in my postgres server during peak > time. Following are the > details of my setup (details as per the postgres wiki) . > > * PostgreSQL version >          o Run "select pg_version();" in ps

[PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-09-22 Thread Shiva Raman
Dear all I am having a problem of high cpu loads in my postgres server during peak time. Following are the details of my setup (details as per the postgres wiki) . ** PostgreSQL version o Run "select pg_version();" in psql or PgAdmin III and provide the full, exact output.* clusterno