Folks,
I have been converting several shell scripts to perl, one of my recent tasks
has
been to convert several egreps into an efficient perl subroutine.
each egrep (5 total) is kicked off to parse a single log dir of ~350 log
files
that grow in size as a process runs. The total size of a log file at the
completion
of the process is ~2MB.
What are my options to create something similar to egrep within perl that
would not
cause a great deal of system load, would be rather efficient and not require
a great
deal of time to run (under 2 mins total parsing time)?
Regards,
-Ron
Here is what I am doing now
open(COM, "parse.pl |") || die("fork failed: \l$!\n");
while (<COM>) {
# increment the number of occurrences we see
# for a certain error.
# this will become our error count for
# statistical and reporting output
foreach my $test (keys %lookFor) {
if ( /Communication/ ) {
$results{$test}++ if ( $_ =~ /$test/
&& $_ =~ /failure/);
next;
}
if ( /Bind/ ) {
$results{$test}++ if ($_ =~ /$test/
&& $_ =~ /Factory/ && $_ =~
/failed/);
next;
}
if ( /SystemException/ ) {
$results{$test}++ if($_ =~ /$test/
&& $_ !~ /UnsavedChangesErro
r/);
next;
}
if ( /SystemError/ ) {
$results{$test}++ if($_ =~ /$test/
&& $_ !~ /foo/);
next;
}
if ( /ORB/ ) {
$results{$test}++ if($_ =~ /$test/
&& $_ !~ /UnsavedChangesErro
r/);
next;
}
$results{$test}++ if /$test/;
next;
}
}
parse.pl:
$lookFor="Test App Finished|Fault
2-0|ORA-|Bind|SystemError|SystemException|Communication|ORBA|Get Q Er
ror|Password succesfully changed|LogonError";
for my $key (0 .. 4) {
opendir DIR, "../logs/set$key/" or die "Can't open ../logs/set$key:
$!";
@allFiles = readdir DIR;
closedir DIR;
foreach (@allFiles) {
if ( $key == 0 )
{
push(@logFiles0,"../logs/set$key/" . $_);
}
if ( $key == 1 )
{
push(@logFiles1,"../logs/set$key/" . $_);
}
if ( $key == 2 )
{
push(@logFiles2,"../logs/set$key/" . $_);
}
if ( $key == 3 )
{
push(@logFiles3,"../logs/set$key/" . $_);
}
if ( $key == 4 )
{
push(@logFiles4,"../logs/set$key/" . $_);
}
}
}
@CMD = ( "egrep", "egrep", "egrep", "egrep", "egrep" );
@LOGS = ( \@logFiles0,\@logFiles1, \@logFiles2, \@logFiles3, \@logFiles4 );
sub spawn ($) {
return 0 unless @CMD;
unless ($cmd = shift @CMD) { # read next command from cmdfile
warn "\nDone with command $cmd\n";
return undef @CMD;
}
chomp $cmd;
exec $cmd, $lookFor, @{$LOGS[$_]} unless $pid=fork; # fork new process
for cmd
if (defined $pid) {
warn "\n$cmd\n\tforked as PID $pid ",
scalar localtime(time),"\n";
} else {
die "\nFailed fork for $cmd\n\t-- ending.\n";
}
$cmd{$pid} = $cmd;
return $pid;
}
....
....
....
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
