Re: [HACKERS] VACUUM FULL out of memory

2008-01-16 Thread Simon Riggs
On Wed, 2008-01-16 at 21:53 +0100, Michael Omotayo Akinde wrote: > As far as I can see, the ulimits are set up as they should; and on a > 64-bit machine with 16GB RAM, I don't see there should be a problem > with allocating 2 GB maintenance work memory. In any case, I have > serious difficulty bel

Re: [HACKERS] VACUUM FULL out of memory

2008-01-16 Thread Michael Omotayo Akinde
Just to conclude on the issue we had here. As far as I can see, the ulimits are set up as they should; and on a 64-bit machine with 16GB RAM, I don't see there should be a problem with allocating 2 GB maintenance work memory. In any case, I have serious difficulty believing that the ulimits can be

Re: [HACKERS] VACUUM FULL out of memory

2008-01-09 Thread Michael Akinde
Thanks for the explanation on the ulimits; I can see how that could turn out a problem in some cases. Following Tom's suggestion, here is the startup script I used: #!/bin/sh ulimit -a > $PGHOST/server.ulimit pg_ctl start -l $PGHOST/server.log The ulimits seem to be the same, though: $> cat ser

Re: [HACKERS] VACUUM FULL out of memory

2008-01-08 Thread Andrew Sullivan
On Tue, Jan 08, 2008 at 05:53:28PM +, Sam Mason wrote: > What about a stored procedure in a language that allows you to do > system(3) calls? PL/bash? (I think there is something like this). But surely the ulimit before start is much easier! A ---(end of broadcast)

Re: [HACKERS] VACUUM FULL out of memory

2008-01-08 Thread Tom Lane
Sam Mason <[EMAIL PROTECTED]> writes: > On Tue, Jan 08, 2008 at 12:33:34PM -0500, Tom Lane wrote: >> The only thing I find convincing is to insert "ulimit -a >someplace" >> into the script that starts the postmaster, > What about a stored procedure in a language that allows you to do > system(3) c

Re: [HACKERS] VACUUM FULL out of memory

2008-01-08 Thread Sam Mason
On Tue, Jan 08, 2008 at 12:33:34PM -0500, Tom Lane wrote: > Andrew Sullivan <[EMAIL PROTECTED]> writes: > > On one system I used many years ago, /bin/sh wasn't what I thought it was, > > and so the ulimit that I got when logged in was not what the postmaster was > > starting under. Took me many da

Re: [HACKERS] VACUUM FULL out of memory

2008-01-08 Thread Tom Lane
Andrew Sullivan <[EMAIL PROTECTED]> writes: > On Tue, Jan 08, 2008 at 05:27:16PM +0100, Michael Akinde wrote: >> Those are the ulimits of the db_admin account (i.e., the user that set >> up and runs the DB processes). Is Postgres limited by other settings? > Are you sure? > On one system I used

Re: [HACKERS] VACUUM FULL out of memory

2008-01-08 Thread Andrew Sullivan
On Tue, Jan 08, 2008 at 05:27:16PM +0100, Michael Akinde wrote: > > > Those are the ulimits of the db_admin account (i.e., the user that set > up and runs the DB processes). Is Postgres limited by other settings? Are you sure? On one system I used many years ago, /bin/sh wasn't what I thought

Re: [HACKERS] VACUUM FULL out of memory

2008-01-08 Thread Michael Akinde
Tom Lane wrote: Michael Akinde <[EMAIL PROTECTED]> writes: $> ulimit -a core file size (blocks, -c) 1 ... What you're showing us is the conditions that prevail in your interactive session. That doesn't necessarily have a lot to do with the ulimits that init-scripts ru

Re: [HACKERS] VACUUM FULL out of memory

2008-01-08 Thread Tom Lane
Michael Akinde <[EMAIL PROTECTED]> writes: > We went over this somewhat prior to Christmas. Here's how its currently > set up. > $> ulimit -a > core file size (blocks, -c) 1 > ... What you're showing us is the conditions that prevail in your interactive session. That doesn't ne

Re: [HACKERS] VACUUM FULL out of memory

2008-01-08 Thread Andrew Sullivan
On Tue, Jan 08, 2008 at 09:50:07AM +0100, Michael Akinde wrote: > stack size (kbytes, -s) 8192 Perhaps this is the issue? (I don't know.) Also, this _is_ for the postgres user, right? That's the relevant one: the one that's actually running the back end process. Also, are you su

Re: [HACKERS] VACUUM FULL out of memory

2008-01-08 Thread Michael Akinde
Tom Lane wrote: Michael Akinde <[EMAIL PROTECTED]> writes: INFO: vacuuming "pg_catalog.pg_largeobject" ERROR: out of memory DETAIL: Failed on request of size 536870912 Are you sure this is a VACUUM FULL, and not a plain VACUUM? Very sure. Ran a VACUUM FULL again yesterday (the prior query w

Re: [HACKERS] VACUUM FULL out of memory

2008-01-07 Thread Tom Lane
Michael Akinde <[EMAIL PROTECTED]> writes: > As suggested, I tested a VACUUM FULL ANALYZE with 128MB shared_buffers > and 512 MB reserved for maintenance_work_mem (on a 32 bit machine with 4 > GB RAM). That ought to leave more than enough space for other processes > in the system. Again, the sys

Re: [HACKERS] VACUUM FULL out of memory

2008-01-07 Thread Joshua D. Drake
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Mon, 07 Jan 2008 17:33:53 +0100 "Holger Hoffstaette" <[EMAIL PROTECTED]> wrote: > On Mon, 07 Jan 2008 10:57:53 -0500, Andrew Sullivan wrote: > > > Note that you should almost never use VACUUM FULL unless you've > > really messed things up. I unde

Re: [HACKERS] VACUUM FULL out of memory

2008-01-07 Thread Holger Hoffstaette
On Mon, 07 Jan 2008 10:57:53 -0500, Andrew Sullivan wrote: > Note that you should almost never use VACUUM FULL unless you've really > messed things up. I understand from the thread that you're just testing > things out right now. But VACUUM FULL is not something you should _ever_ > need in produ

Re: [HACKERS] VACUUM FULL out of memory

2008-01-07 Thread Andrew Sullivan
On Mon, Jan 07, 2008 at 10:40:23AM +0100, Michael Akinde wrote: > As suggested, I tested a VACUUM FULL ANALYZE with 128MB shared_buffers > and 512 MB reserved for maintenance_work_mem (on a 32 bit machine with 4 > GB RAM). That ought to leave more than enough space for other processes > in the s

Re: [HACKERS] VACUUM FULL out of memory

2008-01-07 Thread Michael Akinde
Hi, The system we are building is intended to be utilized in a number of different applications, so the testing we are doing is primarily directed at stressing the system by running it through its paces and uncovering any weaknesses. I prefer to find as many problems as possible now, rather t

Re: [HACKERS] VACUUM FULL out of memory

2008-01-07 Thread Usama Dar
On Jan 7, 2008 2:40 PM, Michael Akinde <[EMAIL PROTECTED]> wrote: > As suggested, I tested a VACUUM FULL ANALYZE with 128MB shared_buffers > and 512 MB reserved for maintenance_work_mem (on a 32 bit machine with 4 > GB RAM). My Apologies if my question seems redundant and something you have alr

Re: [HACKERS] VACUUM FULL out of memory

2008-01-07 Thread Michael Akinde
As suggested, I tested a VACUUM FULL ANALYZE with 128MB shared_buffers and 512 MB reserved for maintenance_work_mem (on a 32 bit machine with 4 GB RAM). That ought to leave more than enough space for other processes in the system. Again, the system fails on the VACUUM with the following error (