I'm trying to develop a long-running monitoring script using LWP to fetch the
raw data. Unfortunately there seems to be a really bad memory leak in LWP that
is causing my script to grow to over 100Mb core in about a day.
I've profiled the code using Devel::Leak and the leak (or at least *a* leak)
is at the perl level in the LWP code or code it calls. It leaks anywhere from
1-5 Svs per request, and sometimes more.
I'm having trouble understanding the Devel::leak output but it seems to me
like LWP is just leaking random strings like "ALRM" and empty hashes. Are all
variables in LWP lexically scoped, or is there some static state that could be
growing like this?
I checked back in the logs and saw that there have been previous complaints
about memory leaks that were mostly poo-pooed with comments like "buy more
memory" or "works for me". I hope that doesn't represent the consensus
response since neither is really appropriate. The fact is that memory leaks in
core infrastructure libraries are major problems for lots of applications.
Here is the output of Devel::Leak, the only code run between the Devel::Leak
calls looked more or less like this, which should not cause _any_ sv leaks.
{
my $ua = new LWP::UserAgent;
$ua->agent("myagent");
my $req = new HTTP::Request GET => $url;
my $res = $ua->request($req);
if ($res->is_success) {
$preexistingstring .= $res->content;
}
}
new 0x83b6334 : SV = NULL(0x0)
REFCNT = 1
FLAGS = (PADBUSY,PADMY)
new 0x83b6370 : SV = PVHV(0x84a39f0)
REFCNT = 1
FLAGS = (PADBUSY,PADMY,SHAREKEYS)
IV = 0
NV = 0
ARRAY = 0x0
KEYS = 0
FILL = 0
MAX = 7
RITER = -1
EITER = 0x0
new 0x83b6454 : SV = PV(0x83eb3f0)
REFCNT = 1
FLAGS = (TEMP,POK,pPOK)
PV = 0x8452068 "(0x84a39f0)
REFCNT = 1
FLAGS = (PADBUSY,PADMY,SHAREKEYS)"
CUR = 60
LEN = 61
new 0x83b64cc : SV = NULL(0x0)
REFCNT = 1
FLAGS = (PADBUSY,PADMY)
new 0x83b65b0 : SV = PV(0x83eb414)
REFCNT = 1
FLAGS = (TEMP,POK,pPOK)
PV = 0x84841a8 "(0x0)
REFCNT = 1
FLAGS = (PADBUSY,PADMY)"
CUR = 44
LEN = 45
new 0x83eaa94 : SV = PV(0x83d0bac)
REFCNT = 1
FLAGS = (TEMP,POK,pPOK)
PV = 0x84054c0 "(0x0)
REFCNT = 1
FLAGS = (PADBUSY,PADMY)"
CUR = 44
LEN = 45
new 0x83d7ab8 : SV = PV(0x83d0bb8)
REFCNT = 1
FLAGS = (TEMP,POK,pPOK)
PV = 0x84531c0 "(0x83d0bac)
REFCNT = 1
FLAGS = (TEMP,POK,pPOK)"
CUR = 50
LEN = 51
new 0x83d7268 : SV = PV(0x839c5c0)
REFCNT = 1
FLAGS = (POK,READONLY,pPOK)
PV = 0x8451e48 "ALRM"
CUR = 4
LEN = 5
new 0x83d6050 : SV = PV(0x83eb444)
REFCNT = 1
FLAGS = (TEMP,POK,pPOK)
PV = 0x84532d8 "(0x83eb3f0)
REFCNT = 1
FLAGS = (TEMP,POK,pPOK)"
CUR = 50
LEN = 51
old (1):
0 SV = UNKNOWN(0x83d72ec)
REFCNT = 0
FLAGS = ()
old (1):
0 SV = UNKNOWN(0x83b6328)
REFCNT = 0
FLAGS = ()
old (1):
0 SV = UNKNOWN(0x83b63c4)
REFCNT = 0
FLAGS = ()
old (1):
0 SV = UNKNOWN(0x83eab24)
REFCNT = 0
FLAGS = ()
--
greg
