Re: [HACKERS] Freeing plan memory
"Nigel J. Andrews" <[EMAIL PROTECTED]> writes:
> On Sat, 19 Oct 2002, Tom Lane wrote:
>> I'm not sure where the leak is in your plpython example, but I'd be
>> inclined to look to plpython itself, perhaps even just the string
>> concatenation expression in
>> plan = plpy.prepare("SELECT " + repr(a))
> Well it's not that string operation.
Actually, I'll bet it's this code in PLy_spi_prepare:
plan->plan = SPI_prepare(query, plan->nargs, plan->types);
// error check
plan->plan = SPI_saveplan(plan->plan);
// error check
The copy of the plan that's returned by SPI_prepare is being blithely
lost --- and since it's in the procCxt, it won't go away until the
plpython function is exited. Need a SPI_freeplan() here, I think.
Can you check it out and send a patch?
regards, tom lane
---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?
http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] Freeing plan memory
On Sat, 19 Oct 2002, Tom Lane wrote:
> "Nigel J. Andrews" <[EMAIL PROTECTED]> writes:
> > The leak is that memory is grabbed in SPI_prepare() for a plan within
> > whatever context is current when it does the palloc(). It may be the
> > caller's or it may be the relevent SPI one. The plan is then copied
> > out of this memory [and context] into a child of the procedure's
> > context and forgotten about, or just plain forgotten.
>
> Au contraire: SPI_prepare builds the plan in its "execCxt", which is
> reset before returning (look at _SPI_begin_call and _SPI_end_call).
> So I see no leak there.
Ah, yes, I see that now.
> I'm not sure where the leak is in your plpython example, but I'd be
> inclined to look to plpython itself, perhaps even just the string
> concatenation expression in
> plan = plpy.prepare("SELECT " + repr(a))
Well it's not that string operation.
> plpgsql used to have terrible intra-function memory leaks, and only by
> dint of much hard work has it been brought to the point where you can
> expect a long loop in a plpgsql function not to chew up memory. AFAIK,
> no one has yet done similar work for the other PL languages.
Hmmm...my test case should boil down to a fairly small number of other calls in
the SPI_prepare wrapper and a quick looks doesn't show anything
interesting. Not sure I've got the time to dedicate to investigating this but
I'll look at it as and when I can.
I'm sending a patch for plpython.c to -patches which fixes a mistake I made in
the previous patch.
--
Nigel J. Andrews
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
Re: [HACKERS] Freeing plan memory
"Nigel J. Andrews" <[EMAIL PROTECTED]> writes:
> The leak is that memory is grabbed in SPI_prepare() for a plan within
> whatever context is current when it does the palloc(). It may be the
> caller's or it may be the relevent SPI one. The plan is then copied
> out of this memory [and context] into a child of the procedure's
> context and forgotten about, or just plain forgotten.
Au contraire: SPI_prepare builds the plan in its "execCxt", which is
reset before returning (look at _SPI_begin_call and _SPI_end_call).
So I see no leak there.
I'm not sure where the leak is in your plpython example, but I'd be
inclined to look to plpython itself, perhaps even just the string
concatenation expression in
plan = plpy.prepare("SELECT " + repr(a))
plpgsql used to have terrible intra-function memory leaks, and only by
dint of much hard work has it been brought to the point where you can
expect a long loop in a plpgsql function not to chew up memory. AFAIK,
no one has yet done similar work for the other PL languages.
regards, tom lane
---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
[HACKERS] Freeing plan memory
I notice there's a leak of memory in SPI_prepare().
The full fix is nontrival and I don't want to submit a half solution so I
thought I'd check whether people think it's worth worrying about.
The leak is that memory is grabbed in SPI_prepare() for a plan within whatever
context is current when it does the palloc(). It may be the caller's or it may
be the relevent SPI one. The plan is then copied out of this memory [and
context] into a child of the procedure's context and forgotten about, or just
plain forgotten. Obviously the intention is that this memory is freed when the
context is deleted and is probably not a problem unless someone does something
like:
i = 10;
while (i--)
{
plan = SPI_prepare("SELECT 1", 0, (Oid *)NULL);
SPI_freeplan(plan); /* SPI_freeplan() is not just for SPI_saveplan() */
}
Is this worth worrying about?
Any busy person can stop reading now as the above defines the problem while the
below only shows an easily reproducable example.
FWIW, I found it while testing something like, which is a little less daft
than the above example:
create function atest1 ( ) returns int as '
a = 0
while a < 1:
plan = plpy.prepare("SELECT " + repr(a))
a = a + 1
' language 'plpython';
Here the plpython code uses SPI_freeplan to release the context holding the
plan memory when each plan object returned by plpy.prepare() is garbage
collected. This seems sensibly to happen when the plan variable is
reassigned. However I was baffled why the process still had an obvious memory
leak so looked a little closer at SPI.
--
Nigel J. Andrews
---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
