From: Tom Metro
Date: Wed, 09 Mar 2011 02:45:19 -0500
Bob Rogers wrote:
>0. I assume you are already using ulimit to trigger the error
> sooner, rather than later . . .
I haven't gotten to the point of trying to intentionally reproduce the
issue, but ulimit would indeed
Ben Tilly wrote:
> Tom Metro wrote:
>> Any recommendation for tools to do this?
>
> Devel::Leak and Devel::LeakTrace are my best suggestion.
Devel::LeakTrace hasn't had a release since 2003. I see a
Devel::LeakTrace::Fast released in 2007, which says it is a rewrite of
the former.
http://search.
On Tue, Mar 8, 2011 at 11:47 PM, Tom Metro wrote:
> While the main file being processed is never entirely memory resident,
> some data from each record is aggregated. That and the lookup tables are
> expected to remain resident for the life of the process.
>
> So the OOM could be a result of an i
> "TM" == Tom Metro writes:
TM> Conor Walsh wrote:
>> You can trap OOM if your Perl is compiled to use $^M...
TM> http://perldoc.perl.org/perlvar.html
TM> $^M
TM> By default, running out of memory is an untrappable, fatal error.
TM> However, if suitably built, Perl can us
Uri Guttman wrote:
> if you run out of vram and you didn't expect it, wouldn't that signify a
> possible leak?
No, not necessarily. This is code that processes large flat files,
supplementing them with information pulled from a few large lookup
tables which are loaded into memory, as well as info
Federico Lucifredi wrote:
> I haven't tried this, but I have been mulling it for some time: using
> Dtrace to debug Perl.
>
> ...using runtime probes seems very promising for the memory
> leak/out of memory problem area. I saw a good tutorial at OSCON on
> the subject, and it had a Python section,
Bob Rogers wrote:
>0. I assume you are already using ulimit to trigger the error
> sooner, rather than later . . .
I haven't gotten to the point of trying to intentionally reproduce the
issue, but ulimit would indeed do the trick. The problem was just
recently discovered when running the code
Conor Walsh wrote:
> You can trap OOM if your Perl is compiled to use $^M...
http://perldoc.perl.org/perlvar.html
$^M
By default, running out of memory is an untrappable, fatal error.
However, if suitably built, Perl can use the contents of $^M as an
emergency memory pool after die()ing.
On Tue, Mar 8, 2011 at 7:52 PM, Tom Metro wrote:
> Some code I'm working on is triggering an out of memory error, and I'd
> like to figure out what specifically is responsible. (It's a complex
> system with dozens of libraries and it runs in parallel across a cluster
> of machines. Running the cod
> "TM" == Tom Metro writes:
TM> Some code I'm working on is triggering an out of memory error, and I'd
TM> like to figure out what specifically is responsible. (It's a complex
TM> system with dozens of libraries and it runs in parallel across a cluster
TM> of machines. Running the cod
I haven't tried this, but I have been mulling it for some time: using Dtrace to
debug Perl.
Of course, you'd have to be on *BSD or OS-X (or Soracle), But if that's Ok,
using runtime probes seems very promising for the memory leak/out of memory
problem area. I saw a good tutorial at OSCON on the
From: Tom Metro
Date: Tue, 08 Mar 2011 22:52:21 -0500
Some code I'm working on is triggering an out of memory error, and I'd
like to figure out what specifically is responsible. (It's a complex
system with dozens of libraries and it runs in parallel across a cluster
of machines.
You can trap OOM if your Perl is compiled to use $^M, which your Perl
probably isn't. Otherwise... if I were you I'd look into a combination of
old-fashioned "sprinkle print statements everywhere" and maybe some deep
magic to attach them to the start of every sub.
That said, even if you do trap O
Some code I'm working on is triggering an out of memory error, and I'd
like to figure out what specifically is responsible. (It's a complex
system with dozens of libraries and it runs in parallel across a cluster
of machines. Running the code in a debugger isn't a practical option.)
Any recommenda
14 matches
Mail list logo
