Re: [ADMIN] Massive table bloat

2013-02-01 Thread Sergey Konoplev
Hi all, For those who are interested in pgcompactor - v1.0rc1 is out. It contains a lot of improvements and has already been tested on a plenty of databases. The list of changes is below: 2013-02-01 v1.0rc1 - Refactored information files, PgToolkit is released under the PostgreSQL License now

Re: [ADMIN] Massive table bloat

2012-12-12 Thread Rural Hunter
于 2012/12/12 13:44, Sergey Konoplev 写道: On Tue, Dec 11, 2012 at 9:39 PM, Rural Hunter ruralhun...@gmail.com wrote: Great. It works now. Thanks a lot for your instant help! You are welcome. Thanks for your feedback and sorry for this bugs. I have noted down this issue with password and planned

[ADMIN] Massive table bloat

2012-12-11 Thread Michael Sawyers
Our dba quit last week leaving me with an interesting problem. We have a table currently using 33gb worth of space for only 152mb worth of data because of bad processes or autovacuum not being aggressive enough. I was able to confirm the size difference by doing a create table as select along

Re: [ADMIN] Massive table bloat

2012-12-11 Thread Craig James
On Tue, Dec 11, 2012 at 8:11 AM, Michael Sawyers msawy...@iii.com wrote: Our dba quit last week leaving me with an interesting problem. We have a table currently using 33gb worth of space for only 152mb worth of data because of bad processes or autovacuum not being aggressive enough. I was

Re: [ADMIN] Massive table bloat

2012-12-11 Thread Steve Crawford
On 12/11/2012 08:11 AM, Michael Sawyers wrote: Our dba quit last week leaving me with an interesting problem. We have a table currently using 33gb worth of space for only 152mb worth of data because of bad processes or autovacuum not being aggressive enough. I was able to confirm the size

Re: [ADMIN] Massive table bloat

2012-12-11 Thread Michael Sawyers
Political reasons have ruled out the dump and reload options, but restoring the entire database took several hours. I'm also restricted on version because newer versions of postgres are not supported with that specific product, including maintenance updates. So I'm trying to fix things in place

Re: [ADMIN] Massive table bloat

2012-12-11 Thread Michael Sawyers
The dump takes about 30 minutes and restore on an older dev machine took several hours to complete. I did create a cluster on the restored (un-bloated) table and it finished in ~10 minutes and I should have space for the extra copies since the actual data is very small. I had looked at pg_reorg,

Re: [ADMIN] Massive table bloat

2012-12-11 Thread Steve Crawford
On 12/11/2012 11:41 AM, Michael Sawyers wrote: Political reasons have ruled out the dump and reload options, but restoring the entire database took several hours. I'm also restricted on version because newer versions of postgres are not supported with that specific product, including

Re: [ADMIN] Massive table bloat

2012-12-11 Thread Craig James
On Tue, Dec 11, 2012 at 12:48 PM, Steve Crawford scrawf...@pinpointresearch.com wrote: On 12/11/2012 11:41 AM, Michael Sawyers wrote: Political reasons have ruled out the dump and reload options, but restoring the entire database took several hours. I'm also restricted on version because

Re: [ADMIN] Massive table bloat

2012-12-11 Thread Michael Sawyers
Thanks all for the feedback, especially on the topic of version. I plan on pushing that whenever I have an opening. But for now I've been placed in fire fighter mode with this one at the top of the list (long story behind it but it involved massive customer service issue on our end) so I need

Re: [ADMIN] Massive table bloat

2012-12-11 Thread Sergey Konoplev
On Tue, Dec 11, 2012 at 8:11 AM, Michael Sawyers msawy...@iii.com wrote: We have a table currently using 33gb worth of space for only 152mb worth of data because of bad processes or autovacuum not being aggressive enough. I was able to confirm the size difference by doing a create table as

Re: [ADMIN] Massive table bloat

2012-12-11 Thread Sergey Konoplev
On Tue, Dec 11, 2012 at 1:09 PM, Sergey Konoplev gray...@gmail.com wrote: On Tue, Dec 11, 2012 at 8:11 AM, Michael Sawyers msawy...@iii.com wrote: We have a table currently using 33gb worth of space for only 152mb worth of data because of bad processes or autovacuum not being aggressive enough.

Re: [ADMIN] Massive table bloat

2012-12-11 Thread Michael Sawyers
Thanks for the tool suggestion. I already know that I will be refused permission to use it on a live db for the first run here, but I will be using this on several test machines that I am sure are bloated to prove the point and get this added into the standard toolkit here. -- View this

Re: [ADMIN] Massive table bloat

2012-12-11 Thread Sergey Konoplev
On Tue, Dec 11, 2012 at 1:14 PM, Michael Sawyers msawy...@iii.com wrote: Thanks for the tool suggestion. I already know that I will be refused permission to use it on a live db for the first run here, but I will be using this on several test machines that I am sure are bloated to prove the

Re: [ADMIN] Massive table bloat

2012-12-11 Thread Michael Sawyers
I have no doubt about the tool, I have doubt about managements willingness to let me start using it. People here are very risk adverse, and sometimes that means difficulty in getting improved technology in the door. It happens of course, it is just slower than one might hope. Once I have

Re: [ADMIN] Massive table bloat

2012-12-11 Thread Rural Hunter
Hi Sergey, I downloaded pgtoolkit-v1.0beta3-fatscripts.tar.gz and tested it. I got error when trying this: ./pgcompactor -a -u DatabaseChooserError Can not find an adapter. at /loader/0x1c26f18/PgToolkit/DatabaseChooser.pm line 63. ./pgcompactor -d testdb -u DatabaseChooserError Can not find

Re: [ADMIN] Massive table bloat

2012-12-11 Thread Sergey Konoplev
On Tue, Dec 11, 2012 at 7:40 PM, Rural Hunter ruralhun...@gmail.com wrote: I downloaded pgtoolkit-v1.0beta3-fatscripts.tar.gz and tested it. I got error when trying this: ./pgcompactor -a -u DatabaseChooserError Can not find an adapter. at /loader/0x1c26f18/PgToolkit/DatabaseChooser.pm line

Re: [ADMIN] Massive table bloat

2012-12-11 Thread Rural Hunter
I do have psql installed. I'm on the db server. $ psql psql.bin (9.1.0) Type help for help. postgres=# \q $ ./pgcompactor -a -u DatabaseChooserError Can not find an adapter. at /loader/0x2539f18/PgToolkit/DatabaseChooser.pm line 63. 于 2012/12/12 11:46, Sergey Konoplev 写道: On Tue, Dec 11,

Re: [ADMIN] Massive table bloat

2012-12-11 Thread Sergey Konoplev
On Tue, Dec 11, 2012 at 7:57 PM, Rural Hunter ruralhun...@gmail.com wrote: I do have psql installed. I'm on the db server. $ psql psql.bin (9.1.0) Type help for help. postgres=# \q $ ./pgcompactor -a -u DatabaseChooserError Can not find an adapter. at

Re: [ADMIN] Massive table bloat

2012-12-11 Thread Rural Hunter
于 2012/12/12 12:24, Sergey Konoplev 写道: On Tue, Dec 11, 2012 at 7:57 PM, Rural Hunter ruralhun...@gmail.com wrote: I do have psql installed. I'm on the db server. $ psql psql.bin (9.1.0) Type help for help. postgres=# \q $ ./pgcompactor -a -u DatabaseChooserError Can not find an adapter. at

Re: [ADMIN] Massive table bloat

2012-12-11 Thread Sergey Konoplev
On Tue, Dec 11, 2012 at 8:30 PM, Rural Hunter ruralhun...@gmail.com wrote: No. I was running it with another db super user. should it only be run by postgres? $ echo 'SELECT 1;' | psql -q -A -t -X -U postgres -P null=NULL Password for user postgres: 1 Oh, looks like I know why it happens.

Re: [ADMIN] Massive table bloat

2012-12-11 Thread Rural Hunter
于 2012/12/12 12:47, Sergey Konoplev 写道: On Tue, Dec 11, 2012 at 8:30 PM, Rural Hunter ruralhun...@gmail.com wrote: No. I was running it with another db super user. should it only be run by postgres? $ echo 'SELECT 1;' | psql -q -A -t -X -U postgres -P null=NULL Password for user postgres: 1

Re: [ADMIN] Massive table bloat

2012-12-11 Thread Sergey Konoplev
On Tue, Dec 11, 2012 at 9:21 PM, Rural Hunter ruralhun...@gmail.com wrote: Ok, thanks. I installed dbd::pg. Now I can run it with specify additional parameters(-h, -p). Seems pgcompactor doesn't read them from env variables. However, I met another error when pgcompactor processes tables. Seems

Re: [ADMIN] Massive table bloat

2012-12-11 Thread Rural Hunter
于 2012/12/12 13:31, Sergey Konoplev 写道: Yes. It is known bug and it is fixed in the future (not yet released) version. See the attachment. Great. It works now. Thanks a lot for your instant help! -- Sergey Konoplev Database and Software Architect http://www.linkedin.com/in/grayhemp Phones:

Re: [ADMIN] Massive table bloat

2012-12-11 Thread Sergey Konoplev
On Tue, Dec 11, 2012 at 9:39 PM, Rural Hunter ruralhun...@gmail.com wrote: Great. It works now. Thanks a lot for your instant help! You are welcome. Thanks for your feedback and sorry for this bugs. I have noted down this issue with password and planned to add .pg* and env support. -- Sergey