Re: Tracking down memory hogs

[Jay Buffington]

On Thu, 2006-06-22 at 11:20 -0700, Jay Buffington wrote:

I'm considering writing a PerlLogHandler that will print out the
memory usage (using GTop) before and after each request so I can find
the offending code path.

I wrote this script and it turned out to work very well.  It's small,
so I included it below inline.  It outputs lines that look like this:

[Fri Jun 23 15:29:00 2006] [error] [24962] url: /hogmem before:
154.85M; after: 490.82M; delta: 335.96M

After the weekend I'll look at packaging this up and putting it in
CPAN.  The GTop and Number::Format CPAN modules are preqs.

--------------------------------------------

# The Apache::LogMemoryUsage module is free software; you can
# redistribute it and/or modify it under the same terms as Perl itself.

package Apache::LogMemoryUsage;

use strict;
use warnings;

use GTop;
use Apache::Constants qw(OK);
use Number::Format qw(format_bytes);

# deltas lower than this threshold will not be logged
# this is in bytes
my $THRESHOLD = (1024 * 1024) * 4; # 4mb

my $gtop = GTop->new();

# should be called like this in your httpd.conf:
# PerlModule Apache::LogMemoryUsage
# PerlInitHandler Apache::LogMemoryUsage
sub handler {
       my $r = shift;

       $r->pnotes("mem_usage_before_request", $gtop->proc_mem($$)->size());

       # after the request is complete, we'll call the log_handler
       $r->push_handlers( PerlLogHandler => \&log_handler );

       return OK;

}

sub log_handler {
       my $r = shift;

       my $after_bytes = $gtop->proc_mem($$)->size();
       my $before_bytes = $r->pnotes("mem_usage_before_request");
       my $delta_bytes =  $after_bytes - $before_bytes;

       if ($delta_bytes > $THRESHOLD) {
           $r->log_error("[$$] url: " . $r->uri() .
                         "; before: " . format_bytes( $before_bytes ) .
                         "; after: " . format_bytes( $after_bytes ) .
                         "; delta: " . format_bytes( $delta_bytes ));
       }

       return OK;
}

1;