Re: [GENERAL] limiting query time and/or RAM

2009-09-17 Thread Alan McKay
> An EXPLAIN (EXPLAIN ANALYSE if it's not going to hurt things) of some of > your common queries would help a lot here. Yes, we are just about to start getting into that sort of thing. -- “Don't eat anything you've ever seen advertised on TV” - Michael Pollan, author of "In Defense of F

Re: [GENERAL] limiting query time and/or RAM

2009-09-17 Thread Alan McKay
On Thu, Sep 17, 2009 at 3:35 PM, Scott Marlowe wrote: > True, but with a work_mem of 2M, I can't imagine having enough sorting > going on to need 4G of ram.  (2000 sorts? That's a lot)  I'm betting > the OP was looking at top and misunderstanding what the numbers mean, > which is pretty common rea

Re: [GENERAL] limiting query time and/or RAM

2009-09-17 Thread Alan McKay
> I'm gonna make a SWAG that you've got 4 to 4.5G shared buffers, and if > you subract that from DRS you'll find it's using a few hundred to > several hundred megs.  still a lot, but not in the 4G range you're > expecting.  What does top say about this? I've just add this in my cronjob with "top -

Re: [GENERAL] limiting query time and/or RAM

2009-09-17 Thread Sam Mason
On Thu, Sep 17, 2009 at 03:56:09PM -0400, Alan McKay wrote: > Our databases are pretty big, and our queries pretty complex. How big is "big" and how complex is "complex"? An EXPLAIN (EXPLAIN ANALYSE if it's not going to hurt things) of some of your common queries would help a lot here. -- Sam

Re: [GENERAL] limiting query time and/or RAM

2009-09-17 Thread Scott Marlowe
On Thu, Sep 17, 2009 at 1:56 PM, Alan McKay wrote: > On Thu, Sep 17, 2009 at 3:35 PM, Scott Marlowe > wrote: >> True, but with a work_mem of 2M, I can't imagine having enough sorting >> going on to need 4G of ram.  (2000 sorts? That's a lot)  I'm betting >> the OP was looking at top and misunder

Re: [GENERAL] limiting query time and/or RAM

2009-09-17 Thread Scott Marlowe
On Thu, Sep 17, 2009 at 1:35 PM, Scott Marlowe wrote: > On Thu, Sep 17, 2009 at 1:31 PM, Bill Moran wrote: >> In response to Scott Marlowe : >> >>> On Thu, Sep 17, 2009 at 12:56 PM, Alan McKay wrote: >>> > Is there any way to limit a query to a certain amount of RAM and / or >>> > certain runtim

Re: [GENERAL] limiting query time and/or RAM

2009-09-17 Thread Scott Marlowe
On Thu, Sep 17, 2009 at 1:31 PM, Bill Moran wrote: > In response to Scott Marlowe : > >> On Thu, Sep 17, 2009 at 12:56 PM, Alan McKay wrote: >> > Is there any way to limit a query to a certain amount of RAM and / or >> > certain runtime? >> > >> > i.e. automatically kill it if it exceeds either b

Re: [GENERAL] limiting query time and/or RAM

2009-09-17 Thread Bill Moran
In response to Scott Marlowe : > On Thu, Sep 17, 2009 at 12:56 PM, Alan McKay wrote: > > Is there any way to limit a query to a certain amount of RAM and / or > > certain runtime? > > > > i.e. automatically kill it if it exceeds either boundary? > > > > We've finally narrowed down our system cras

Re: [GENERAL] limiting query time and/or RAM

2009-09-17 Thread Scott Marlowe
On Thu, Sep 17, 2009 at 1:19 PM, Alan McKay wrote: >> Generally speaking work_mem limits ram used.  What are your >> non-default postgresql.conf settings? > > This cannot be right because we had queries taking 4G and see our > setting is such : Are you sure they were using that much memory? If y

Re: [GENERAL] limiting query time and/or RAM

2009-09-17 Thread Alan McKay
> Generally speaking work_mem limits ram used.  What are your > non-default postgresql.conf settings? This cannot be right because we had queries taking 4G and see our setting is such : work_mem = 2MB # min 64kB I'll have to find a copy of the default file to figure out

Re: [GENERAL] limiting query time and/or RAM

2009-09-17 Thread Scott Marlowe
On Thu, Sep 17, 2009 at 12:56 PM, Alan McKay wrote: > Is there any way to limit a query to a certain amount of RAM and / or > certain runtime? > > i.e. automatically kill it if it exceeds either boundary? > > We've finally narrowed down our system crashes and have a smoking gun, > but no way to fi

[GENERAL] limiting query time and/or RAM

2009-09-17 Thread Alan McKay
Is there any way to limit a query to a certain amount of RAM and / or certain runtime? i.e. automatically kill it if it exceeds either boundary? We've finally narrowed down our system crashes and have a smoking gun, but no way to fix it in the immediate term. This sort of limit would really help