Re: [GENERAL] V8.4 TOAST table problem

2013-07-17 Thread David Welton
Hi, I'm talking about our own massively bloated toast table - described in an earlier post - that I think I can replicate. I didn't mean to steal your thread, but the problem seems very similar, and we're using 9.1. I don't know a lot about Postgres internals, but to me it smells like a bug of

Re: [GENERAL] V8.4 TOAST table problem

2013-07-15 Thread David Welton
Hi, I think I could write a script to do something similar to what is happening if anyone is interested. I'd want some direction as to the best way to handle this though: it'd be easier for me to script it as Rails code because that's what the app is. Perhaps from that we can get the generated

Re: [GENERAL] V8.4 TOAST table problem

2013-07-15 Thread Bradley McCune
David, I'm sorry, but I'm not sure that I follow how this is pertinent to this particular thread. Are you proposing a way to replicate the scenario we experienced of our massively bloated TOAST table? If so, I'm not entirely sure that's doable given that the source of the issue was never clear.

Re: [GENERAL] V8.4 TOAST table problem

2013-07-15 Thread Bradley McCune
Thanks, Scott. Currently, it's a bit difficult due to resources for a complete copy of the database to be useful. I won't get into the details, but it just wasn't an option at the time. With that said, I'm definitely making it a major concern of ours for such future issues, so post mortem and

Re: [GENERAL] V8.4 TOAST table problem

2013-07-12 Thread David Welton
Hi, I have a very similar problem... details below. On Wed, Jul 3, 2013 at 5:19 PM, Paul Tilles paul.til...@noaa.gov wrote: Recently, I've had a PostgreSQL 8.2.11 server upgraded to 8.4 in order to take advantage of autovacuum features. This server exists in a very closed environment

Re: [GENERAL] V8.4 TOAST table problem

2013-07-12 Thread Bradley McCune
David, (As a preface, I have already gone forward with completely rebuilding the database which seems to have finally fixed the problem. Rebuilding the table itself had no effect, and I couldn't wait much longer to move forward.) Yes, this seems similar, however, the key difference being that

Re: [GENERAL] V8.4 TOAST table problem

2013-07-12 Thread Scott Marlowe
Did you have a long running trasnaction? Especially a prepared transaction, blocking the vacuum from reclaiming the space? On Fri, Jul 12, 2013 at 8:10 AM, Bradley McCune bradley.mcc...@noaa.gov wrote: David, (As a preface, I have already gone forward with completely rebuilding the database

Re: [GENERAL] V8.4 TOAST table problem

2013-07-12 Thread Bradley McCune
The only transactions present were IDLE for current_query. I even stopped the remote services, restarted the PostgreSQL server (assumingly, there should be no transactions occurring now), and performed another VACUUM FULL followed by REINDEX due to known fullvac index bloat in pre-9 pgsql

Re: [GENERAL] V8.4 TOAST table problem

2013-07-12 Thread Scott Marlowe
Idle in Transaction? Or plain Idle? Idle in Transaction stops vacuum from reclaiming space and is indicative of a broken application. On Fri, Jul 12, 2013 at 9:39 AM, Bradley McCune bradley.mcc...@noaa.govwrote: The only transactions present were IDLE for current_query. I even stopped the

Re: [GENERAL] V8.4 TOAST table problem

2013-07-12 Thread Scott Marlowe
Prepared transactions that are sitting still do the same thing, and show no connections. On Fri, Jul 12, 2013 at 2:25 PM, Scott Marlowe scott.marl...@gmail.comwrote: Idle in Transaction? Or plain Idle? Idle in Transaction stops vacuum from reclaiming space and is indicative of a broken

Re: [GENERAL] V8.4 TOAST table problem

2013-07-12 Thread Bradley McCune
Scott, Purely idle. I compared these transactions with our other healthy databases, and they checked out. On Fri, Jul 12, 2013 at 4:25 PM, Scott Marlowe scott.marl...@gmail.comwrote: Prepared transactions that are sitting still do the same thing, and show no connections. On Fri, Jul 12,

Re: [GENERAL] V8.4 TOAST table problem

2013-07-12 Thread Scott Marlowe
So what id select * from pg_prepared_xacts ; show? On Fri, Jul 12, 2013 at 2:30 PM, Bradley McCune bradley.mcc...@noaa.govwrote: Scott, Purely idle. I compared these transactions with our other healthy databases, and they checked out. On Fri, Jul 12, 2013 at 4:25 PM, Scott Marlowe

Re: [GENERAL] V8.4 TOAST table problem

2013-07-12 Thread Bradley McCune
Well, the issue was corrected by completely rebuilding the database a few days ago (all the way to reinitializing the database directory). With that said, I did check that table at the time, and I received an empty result set from such a SELECT statement. The same goes for

Re: [GENERAL] V8.4 TOAST table problem

2013-07-12 Thread Scott Marlowe
It's always a good idea to keep a copy of the database for a post mortem if possible. If you've found a bug, it's nice to find and fix it. If you were suffering from an operational failure of some sort, then it helps to figure that out too. On Fri, Jul 12, 2013 at 2:42 PM, Bradley McCune

[GENERAL] V8.4 TOAST table problem

2013-07-03 Thread Paul Tilles
Recently, I've had a PostgreSQL 8.2.11 server upgraded to 8.4 in order to take advantage of autovacuum features. This server exists in a very closed environment (isolated network, limited root privileges; this explains the older software in use) and runs on RHEL5.5 (i686). After the upgrade,