Re: [PERFORM] query against pg_locks leads to large memory alloc

2014-08-19 Thread Kevin Grittner
Dave Owens wrote: > On Tue, Aug 19, 2014 at 11:01 AM, Kevin Grittner wrote: >> CREATE TABLE activity_snap_1 AS SELECT * FROM pg_stat_activity; > Would the you or the list be interested in snapshots of pg_locks as well? Most definitely!  I'm sorry that copied/pasted the pg_stat_activity example

Re: [PERFORM] query against pg_locks leads to large memory alloc

2014-08-19 Thread Dave Owens
On Tue, Aug 19, 2014 at 11:01 AM, Kevin Grittner wrote: > If restart is an option, that sounds like a great idea. If you > could capture the data into tables where we can summarize to > analyze it in a meaningful way, that would be ideal. Something > like: > > CREATE TABLE activity_snap_1 AS SEL

Re: [PERFORM] query against pg_locks leads to large memory alloc

2014-08-19 Thread Kevin Grittner
Dave Owens wrote: > I wonder if it would be helpful to restart the database, then begin > gathering information pg_locks while it can still respond to queries. > I speculate that this is possible because the amount of memory needed > to query pg_locks continues to grow (around 1900MB now). If re

Re: [PERFORM] query against pg_locks leads to large memory alloc

2014-08-19 Thread Kevin Grittner
Dave Owens wrote: > I do see a handful of backends that like to stay IDLE in > transaction for minutes at a time.  We are refactoring the > application responsible for these long IDLE times, which will > hopefully reduce the duration of their connections. That may help some.  Other things to con

Re: [PERFORM] query against pg_locks leads to large memory alloc

2014-08-19 Thread Dave Owens
I wonder if it would be helpful to restart the database, then begin gathering information pg_locks while it can still respond to queries. I speculate that this is possible because the amount of memory needed to query pg_locks continues to grow (around 1900MB now). Dave Owens -- Sent via pgsql-p

Re: [PERFORM] query against pg_locks leads to large memory alloc

2014-08-19 Thread Dave Owens
Hi Andres, On Tue, Aug 19, 2014 at 10:17 AM, Andres Freund wrote: >> max_connections = 450 ...we have found that we run out of shared >> memory when max_pred_locks_per_transaction is less than 30k. > > What was the precise error message when that happened? 2014-07-31 15:00:25 PDT 53dabbea.29c7ER

Re: [PERFORM] query against pg_locks leads to large memory alloc

2014-08-19 Thread Andres Freund
On 2014-08-18 14:36:52 -0700, Dave Owens wrote: > On Mon, Aug 18, 2014 at 2:21 PM, Matheus de Oliveira > wrote: > > Do you really need such large values? What is your max_connections value? > > max_connections = 450 ...we have found that we run out of shared > memory when max_pred_locks_per_trans

Re: [PERFORM] query against pg_locks leads to large memory alloc

2014-08-19 Thread Dave Owens
On Tue, Aug 19, 2014 at 9:40 AM, Kevin Grittner wrote: > Hmm, that's not outrageous. How about long-running transactions? > Please check pg_stat_activity and pg_prepared_xacts for xact_start > or prepared (respectively) values older than a few minutes. Since > predicate locks may need to be kept

Re: [PERFORM] query against pg_locks leads to large memory alloc

2014-08-19 Thread Kevin Grittner
Dave Owens wrote: > 1358 tables > 1808 indexes Hmm, that's not outrageous.  How about long-running transactions? Please check pg_stat_activity and pg_prepared_xacts for xact_start or prepared (respectively) values older than a few minutes.  Since predicate locks may need to be kept until an over

Re: [PERFORM] query against pg_locks leads to large memory alloc

2014-08-19 Thread Dave Owens
Hi Kevin, Looking at pg_stat_all_tables and pg_stat_all_indexes on our four databases we have: 1358 tables 1808 indexes The above totals do not include template1, template0, or postgres databases. We do not use partitioned tables. Only one database has a meaningful level of concurrency (New Rel

Re: [PERFORM] query against pg_locks leads to large memory alloc

2014-08-19 Thread Kevin Grittner
Tom Lane wrote: > Kevin Grittner writes: >> Dave Owens wrote: >>> max_connections = 450 ...we have found that we run out of shared >>> memory when max_pred_locks_per_transaction is less than 30k. > >> It gathers the information in memory to return for all those locks >> (I think both the normal