Darn, baca perl susah amat ya.
Anyhow, problemnya out of memory, Anda tentu sudah tahu.
Kemungkinan sih dari open() yang gunzip itu. Saya takutnya kalau gunzip() itu
dikirim ke pipe, dan open() di situ harus simpan semua datanya di buffer, 2GB
is a big number.
Kalau mau di coba, setelah open(gunzip), coba print() sesuatu. Coba cari tahu
apakah open() nya berhasil atau tidak.
Kalau misalkan tidak berhasil, mungkin sebaiknya file-nya di gunzip dulu, baru
di open() hasil uncompressed-nya, dengan asumsi hard-drive-nya masih punya
free-space banyak.
KOkon.
On Fri, Apr 18, 2008 at 2:10 AM, dv0r4k <[EMAIL PROTECTED]> wrote:
> Halo semua,
> Saya membuat perl script untuk ngebaca log apache terkompresi dgn gzip.
> Jika skrip ini saya jalankan ke log ukuran kecil script tsb berjalan dgn
> baik. Tapi begitu saya jalanin di ke file besar (100MB compressed, 2GB
> uncompressed), script tsb akan terbunuh (sepertinya oleh OS (ulimit?)).
>
> Kenapa yak ?? Mungkin cara open() saya salah ?? help!
>
> EF
>
>
> .:cnt_sssn.pl:.
> ======================================================
> #!/usr/bin/perl -w
>
> use strict;
> use Time::Local;
>
> $|=1;
>
> my $SESSION=24; # to define 1 session, in
> minutes.
> my $LOGF=$ARGV[0];
>
> my @uips;
> my $uiplmt=500; # limit before it flush to file
> my $uipflcntr = 0;
> my $sesslim = $SESSION * 60;
> my $totalsession;
>
> open(LOG,"gunzip -c $LOGF | ") || die "Can't open file $LOGF: $!\n";
> foreach (<LOG>) {
> chomp;
> my @rawln = split(/\s{1,}/,$_,6);
> next if (not defined($rawln[5]));
> my ($ips,$timing,$url) = ($rawln[5] =~ m#^(.*) \[(.*)\] (".*)#);
> push(@uips,$ips) if ( ! grep(/$ips/,@uips) );
> if ($#uips > $uiplmt) {
> print "UIP reached $uiplmt, flush to file
> .uip.$uipflcntr.\n";
> $,="\n";
> open(UIPFL,">> .uip.$uipflcntr");
> print UIPFL @uips;
> close(UIPFL);
> undef @uips;
> $uipflcntr++;
> }
> }
> close(LOG);
>
> print "finish\n";
> ======================================================
>
>
> Ketika di running:::
> ======================================================
> [EMAIL PROTECTED] ~]$ ./cnt_sssn.pl
> /var/www/html/reports/logs/20071012.log.gz
> finish
> [EMAIL PROTECTED] ~]$ ./cnt_sssn.pl
> /var/www/html/reports/logs/20080226.log.gz
> Killed
> [EMAIL PROTECTED] ~]$ ll -h
> /var/www/html/reports/logs/{20071012.log.gz,20080226.log.gz}
> -rw-rw-r-- 1 batch devel 22K Oct 12 2007
> /var/www/html/reports/logs/20071012.log.gz
> -rw-rw-r-- 1 batch root 107M Feb 26 23:59
> /var/www/html/reports/logs/20080226.log.gz
> [EMAIL PROTECTED] ~]$ dmesg | tail
> HighMem: 60*4kB 46*8kB 10*16kB 2*32kB 0*64kB 0*128kB 0*256kB 1*512kB
> 0*1024kB 0*2048kB 0*4096kB = 1344kB
> Swap cache: add 5787810, delete 5780748, find 8492876/8794717, race 0+3
> 0 bounce buffer pages
> Free swap: 0kB
> 524282 pages of RAM
> 294906 pages of HIGHMEM
> 5289 reserved pages
> 31741 pages shared
> 7062 pages swap cached
> Out of Memory: Killed process 20900 (cnt_sssn.pl).
> [EMAIL PROTECTED] ~]$
> ======================================================
>
> contoh data (log apache):::
> ======================================================
> Oct 12 23:51:15 10.0.0.2 ip_log: 10.0.0.5 "- -", - - [12/Oct/2007:23:50:50
> +0800] "GET / HTTP/1.0" 200 8784 "-" "check_http/1.104 (nagios-plugins
> 1.4.9)"
> Oct 12 23:53:37 10.0.0.3 ip_log: 10.0.0.5 "- -", - - [12/Oct/2007:23:53:24
> +0800] "GET / HTTP/1.0" 200 8784 "-" "check_http/1.104 (nagios-plugins
> 1.4.9)"
> Oct 12 23:54:15 10.0.0.2 ip_log: 10.0.0.5 "- -", - - [12/Oct/2007:23:53:50
> +0800] "GET / HTTP/1.0" 200 8784 "-" "check_http/1.104 (nagios-plugins
> 1.4.9)"
> ======================================================
--
Berhenti langganan: [EMAIL PROTECTED]
Arsip dan info: http://linux.or.id/milis
