parsing large amounts of text

[Andrew Gaffney]

I'm working on a custom Perl script to parse my Apache logs and report custom 
information. When I run the following program, it ends up eating all available 
RAM (the system has 1GB) and dying. My access_log is ~410MB. Am I doing 
something wrong?

#!/usr/bin/perl
use strict;
use warnings;
use CGI();
my $months = { Jan => 1, Feb => 2, Mar => 3, Apr => 4, May => 5, Jun => 6, Jul 
=> 7, Aug => 8, Sep => 9, Oct => 10, Nov => 11, Dec => 12 };
my @requests;
my $start = time;
open LOG, "< /var/log/apache/access_log";

while(<LOG>) {
  my $line = $_;
  $line =~ /^(\d+\.\d+\.\d+\.\d+) (.+?) (.+?) \[(.+?)\] \"(?:(.+?) )?(.+)(?: 
(.+?))?\" (\d+) (.+?) \"(.+?)\" \"(.+?)\"$/;
  my ($ip, $date, $request, $requestcode, $bytesreturned, $browser) = ($1, $4, 
$6, $8, $9, $11);

  $request = CGI::unescape($request);
  push @requests, [$ip, $date, $request, $requestcode, $bytesreturned, $browser];
}
my $end = time;
my $elapsed = $end - $start;
close LOG;
print "$#requests total records. $elapsed seconds elapsed\n";
--
Andrew Gaffney
Network Administrator
Skyline Aeronautics, LLC.
636-357-1548
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
<http://learn.perl.org/> <http://learn.perl.org/first-response>