[linux-programming] perl killed ketika buka file besar

dv0r4k Fri, 18 Apr 2008 02:10:27 -0700

Halo semua,

Saya membuat perl script untuk ngebaca log apache terkompresi dgn gzip.Jika skrip ini saya jalankan ke log ukuran kecil script tsb berjalan dgnbaik. Tapi begitu saya jalanin di ke file besar (100MB compressed, 2GBuncompressed), script tsb akan terbunuh (sepertinya oleh OS (ulimit?)).


Kenapa yak ?? Mungkin cara open() saya salah ?? help!

EF


.:cnt_sssn.pl:.
======================================================
#!/usr/bin/perl -w

use strict;
use Time::Local;

$|=1;

my $SESSION=24; # to define 1 session,in minutes.

my $LOGF=$ARGV[0];

my @uips;
my $uiplmt=500;                         # limit before it flush to file
my $uipflcntr = 0;
my $sesslim = $SESSION * 60;
my $totalsession;

open(LOG,"gunzip -c $LOGF | ") || die "Can't open file $LOGF: $!\n";
foreach (<LOG>) {
       chomp;
       my @rawln = split(/\s{1,}/,$_,6);
       next if (not defined($rawln[5]));
       my ($ips,$timing,$url) = ($rawln[5] =~ m#^(.*) \[(.*)\] (".*)#);
       push(@uips,$ips) if ( ! grep(/$ips/,@uips) );
       if ($#uips > $uiplmt) {

print "UIP reached $uiplmt, flush to file.uip.$uipflcntr.\n";

               $,="\n";
               open(UIPFL,">> .uip.$uipflcntr");
               print UIPFL @uips;
               close(UIPFL);
               undef @uips;
               $uipflcntr++;
       }
}
close(LOG);

print "finish\n";
======================================================


Ketika di running:::
======================================================
[EMAIL PROTECTED] ~]$ ./cnt_sssn.pl /var/www/html/reports/logs/20071012.log.gz
finish
[EMAIL PROTECTED] ~]$ ./cnt_sssn.pl /var/www/html/reports/logs/20080226.log.gz
Killed

[EMAIL PROTECTED] ~]$ ll -h/var/www/html/reports/logs/{20071012.log.gz,20080226.log.gz}-rw-rw-r-- 1 batch devel 22K Oct 12 2007/var/www/html/reports/logs/20071012.log.gz-rw-rw-r-- 1 batch root 107M Feb 26 23:59/var/www/html/reports/logs/20080226.log.gz

[EMAIL PROTECTED] ~]$ dmesg | tail

HighMem: 60*4kB 46*8kB 10*16kB 2*32kB 0*64kB 0*128kB 0*256kB 1*512kB0*1024kB 0*2048kB 0*4096kB = 1344kB

Swap cache: add 5787810, delete 5780748, find 8492876/8794717, race 0+3
0 bounce buffer pages
Free swap:            0kB
524282 pages of RAM
294906 pages of HIGHMEM
5289 reserved pages
31741 pages shared
7062 pages swap cached
Out of Memory: Killed process 20900 (cnt_sssn.pl).
[EMAIL PROTECTED] ~]$
======================================================

contoh data (log apache):::
======================================================

Oct 12 23:51:15 10.0.0.2 ip_log: 10.0.0.5 "- -", - -[12/Oct/2007:23:50:50 +0800] "GET / HTTP/1.0" 200 8784 "-""check_http/1.104 (nagios-plugins 1.4.9)"Oct 12 23:53:37 10.0.0.3 ip_log: 10.0.0.5 "- -", - -[12/Oct/2007:23:53:24 +0800] "GET / HTTP/1.0" 200 8784 "-""check_http/1.104 (nagios-plugins 1.4.9)"Oct 12 23:54:15 10.0.0.2 ip_log: 10.0.0.5 "- -", - -[12/Oct/2007:23:53:50 +0800] "GET / HTTP/1.0" 200 8784 "-""check_http/1.104 (nagios-plugins 1.4.9)"

======================================================

--
Berhenti langganan: [EMAIL PROTECTED]
Arsip dan info: http://linux.or.id/milis

[linux-programming] perl killed ketika buka file besar

Kirim email ke