Thanks for all the help! I'm almost there...
Should:
foreach $MyIpAddr (sort keys %{MyIpAddrInfo}) {
next if ( $MyIpAddrInfo->{MyIpAddr} <= $blocklimit ); # if less
than or equal get next key
print $output2 "$ip/32\n";
be:
foreach $MyIpAddr (sort keys %{$MyIpAddrInfo}) {
next if ( $MyIpAddrInfo->{$MyIpAddr} <= $blocklimit ); # if less
than or equal get next key
print $output2 "$ip/32\n";
( I added "$" in front of MyIpAddrInfo and MyIpAddrInfo)
It works this way.
Ryan Lamberton
----- Original Message -----
From: "Wagner, David --- Senior Programmer Analyst --- WGO"
<[EMAIL PROTECTED]>
To: "FamiLink Admin" <[EMAIL PROTECTED]>
Cc: <beginners@perl.org>
Sent: Thursday, September 29, 2005 4:58 PM
Subject: RE: a little help...
FamiLink Admin wrote:
That works! Now I see data!
$VAR1 = {
'70.117.26.250' => '1'
};
$VAR1 = {
'70.117.26.250' => '1',
'71.32.59.249' => '1'
};
$VAR1 = {
'70.117.26.250' => '1',
'71.32.59.249' => '2'
};
$VAR1 = {
'70.117.26.250' => '1',
'71.32.59.249' => '3'
};
The foreach loop does not seem to work and I get this error:
You are not using strict, so it is working and this is just informative.
What you are missing is that you collect all the data first which meets your
criteria. At the end of that processing, then you go through the foreach
checking. You have to be on the outside of the whlle loop:
while (my $line = <$slog>){ # assigns each line in turn to $line
#use an array slice to select the fields we want
my ($time, $ip, $url, $category) = (split " ", $line)[1,4,7,10];
my ($hr) = split /:/, $time;
if($flag eq $category and $hr eq $hour){
$MyIpAddrInfo->{$ip}++;
}
}
foreach $MyIpAddr (sort keys %{MyIpAddrInfo}) {
next if ( $MyIpAddrInfo->{MyIpAddr} <= $blocklimit ); # if less
than or equal get next key
print $output2 "$ip/32\n";
}
print $output1 Dumper(\%{$MyIpAddrInfo});
close $output;
Something along this line as a starter.
You should get use to using strict and warnings. Makes life much easier if
you get started that way.
Wags ;)
Name "main::MyIpAddrInfo" used only once: possible typo at ./test2.pl
line
48.
---------------------------------------------------------------------------------
#!/usr/bin/perl -w
require Mail::Send;
use Data::Dumper;
use constant IP_LIST1_FILE => "/etc/squid/iplist1.txt";
use constant IP_LIST_FILE => "/etc/squid/iplist.txt";
use constant SUSPEND_FILE => "/etc/squid/SuspendIpList.txt";
use constant LOG_FILE => "/opt/n2h2/logs/filter_log";
{
my $sysop = "[EMAIL PROTECTED]";
my $flag = "PO";
my $hour = (localtime)[2];
my $matches = 0;
my ($matched,$ip2) = &Scanlog($flag,$hour,$matches);
if($matched > 1){
$msg = new Mail::Send Subject=>'SuspendIpList',
To=>"$sysop"; $fh = $msg->open;
print $fh "Someone has tried to access $matched banned sites
today\n";
print $fh "Their IP address ($ip2) has been added to
/etc/squid/SuspendIpList.txt\n";
print $fh "To unblock them, remove their entry from the
file and run squid -k reconfigure\n";
print $fh "$matched, $ip2\n";
$fh->close; # complete the message and send it
$matched = 0;
}
else{
open my $output2, ">", SUSPEND_FILE or die "Can't write
@{[SUSPEND_FILE]}: $!";
print $output2 "10.0.0.252/32\n";
close $output2;
}
}
sub Scanlog {
my ($flag,$hour,$matches)[EMAIL PROTECTED];
my $blocklimit = 1;
$matched = 0;
open my $slog, "-|", "tail -n 25000 @{[LOG_FILE]}" or die
"Unable to open @{[LOG_FILE]}: $!";
open my $output, ">", IP_LIST_FILE or die "Can't write
@{[IP_LIST_FILE]}: $!";
open my $output1, ">", IP_LIST1_FILE or die "Can't write
@{[IP_LIST1_FILE]}: $!";
open my $output2, ">", SUSPEND_FILE or die "Can't write
@{[SUSPEND_FILE]}: $!";
my %MIAI = ();
my $MyIpAddrInfo = \%MIAI;
while (my $line = <$slog>){ # assigns each line in turn
to $line #use an array slice to select the fields we want
my ($time, $ip, $url, $category) = (split " ",
$line)[1,4,7,10]; my ($hr) = split /:/, $time;
if($flag eq $category and $hr eq $hour){
$MyIpAddrInfo->{$ip}++;
print $output Dumper(\%{$MyIpAddrInfo});
}
foreach $MyIpAddr (sort keys %{MyIpAddrInfo}) {
next if ( $MyIpAddrInfo->{MyIpAddr} <= $blocklimit );
# if less than or equal get next key
print $output1 Dumper(\%{$MyIpAddrInfo});
print $output2 "$ip/32\n";
$matched = $matches;
$matches = 0;
$ip2 = $ip;
}
}
close $output;
close $output2;
return $matched,$ip2;
}
----------------------------------------------------
Ryan Lamberton
----- Original Message -----
From: "Wagner, David --- Senior Programmer Analyst --- WGO"
<[EMAIL PROTECTED]>
To: "FamiLink Admin" <[EMAIL PROTECTED]>
Cc: <beginners@perl.org>
Sent: Thursday, September 29, 2005 3:15 PM
Subject: RE: a little help...
FamiLink Admin wrote:
David,
Thank you for your help! I am trying to get this to work. Can you
tell me what my $MyIpAddrInfo = \%MIAI; does? I am getting
This is making $MyIpAddInfo a refeence to %MIAI so to access the data
you use the -> as the connector vs $MIAI{}.
HASH(0x8133528)
for $MyIpAddrInfo if I print it out after the:
Since you have only given me this, it should be and I looked at whait
i sent you:
foreach $MyIpAddr (sort keys %{MyIpAddrInfo}) {
next if ( $MyIpAddrInfo->{MyIpAddr} <= $blocklimit ); # if less than
or equal get next key
the best way to see what you have is using Data::Dumper.
Add a use Data::Dumper at top of script.
I then would add at a high level:
$Data::Dumper::Sortkeys = 1; # sort the keys when printing.
Then you can do something like:
print Dumper(\%{$MyIpAddrInfo}); # in thise case will go to std out.
if a lot of data, then open a disk file and write to that.
Also you can before the print place this line:
$Data::Dumper::Varname = 'MyIpAddrInfo'; # this is name will appear
at the beginning of the output
print Dumper(\%{$MyIpAddrInfo}); # in thise case will go to std out.
# if a lot of data, then open a disk file and write to
Wags ;)
$MyIpAddrInfo->{$ip}++;
Ryan Lamberton
----- Original Message -----
From: "Wagner, David --- Senior Programmer Analyst --- WGO"
<[EMAIL PROTECTED]>
To: "FamiLink Admin" <[EMAIL PROTECTED]>
Cc: <beginners@perl.org>
Sent: Wednesday, September 28, 2005 5:53 PM
Subject: RE: a little help...
FamiLink Admin wrote:
I am only concerned about the IP. The rest is just to verify the
data for now. What code would I use to key the $IP in to hash for
counting?. Most of the IP's are not static but are from broadband
and don't change too often. An example log is:
-------------
[2005-09-28 10:05:03 -7:00] 127.0.0.1 71.32.59.249 216.163.137.3 -
http://www.playboy.com/ blocked 0 PO
-------------
the IP I want to count is 71.32.59.249 (for this log) and the
category is PO
I would do something like:
my %MIAI = ();
my $MyIpAddrInfo = \%MIAI;
Now as you go through the scan loop, you would take the if which is
doing the check on the $flag and the do something like
$MyIpAddInfo->{$ip}++; Now you complete your scan and then run
throuh your loop like:
foreach $MyIpAddr (sort keys %{MyIpAddrInfo}) {
next if ( $MyIpAddrInfo->{MyIpAddr} <= $blocklimit ); # if less than
or equal get next key # write your suspend and you could put
together your email at the same time }
A start.
Wags ;)
Ryan Lamberton
----- Original Message -----
From: "Wagner, David --- Senior Programmer Analyst --- WGO"
<[EMAIL PROTECTED]>
To: "FamiLink Admin" <[EMAIL PROTECTED]>
Cc: <beginners@perl.org>
Sent: Wednesday, September 28, 2005 5:18 PM
Subject: RE: a little help...
FamiLink Admin wrote:
Jeff ,
Thanks for all your help! This is what I have now (below and this
time the whole thing): I think I have included all that you
talked about plus others:
The sub scanlog does write the information to the files but it does
not return anything back to the main program and I also get the
error:
Use of uninitialized value in split at ./test.pl line 9.
Also, is there a better way of counting the number of times each IP
address gets blocked with category PO? Each time I get to the
blocklimit it writes to the file but I really just want the max
number of blocks over the limit. It will write the same IP each
time it gets over the blocklimit though.
If you are only concerned about $ip and if they went over that limit
and not desiring the detail of said offense, then you could use the
$ip as a key into a hash. Then you could count all the occurances.
At the conclusion of that processing then you could loop through the
hash and any count greater than your max, then you could write to
the suspend file. For email, then could again use the hash to put
together a list of $ip's that are over your limit.
I have not followed the topic, but unless you do something with the
$ip, I would assume that the log is just that a log. You would have
interspersed $ip and so I am unsure how you would be able to say $ip
is at fault. I see nothing in your code which isolates to the $ip.
Again, are these static ip addr or when someone logs out, they are
ready for use by someone else. If it is released then you have to
figure out when this occurs to get an accurate rcd. If static, then
not a problem.
Wags ;)
------------------------------------------------------------------------------
#!/usr/bin/perl -w require Mail::Send;
$|=1; # no buffering
use constant IP_LIST_FILE => "/etc/squid/iplist.txt";
use constant SUSPEND_FILE => "/etc/squid/SuspendIpList.txt";
use constant LOG_FILE => "/opt/n2h2/logs/filter_log";
my $sysop = "[EMAIL PROTECTED]";
my $flag = "PO";
my $hour = (split, localtime)[2];
my $blocklimit = 5;
my $matches = 0;
my $matched = 0;
{
($matched,$ip,$hour,$time,$category,$url) =
&Scanlog($flag,$hour,$blocklimit,$matches,);
if($matched > $blocklimit){
$msg = new Mail::Send Subject=>'SuspendIpList',
To=>"$sysop"; $fh = $msg->open;
print $fh "Someone has tried to access $matches banned
sites today\n"; print $fh "Their IP address ($ip) has
been added to /etc/squid/SuspendIpList.txt\n";
print $fh "To unblock them, remove their entry from the
file and run squid -k reconfigure\n";
print $fh "$matches, $ip, $hour, $time, $category,
$url\n"; $fh->close; # complete the message and
send it $matched = 0; } else{
open my $output2, ">", SUSPEND_FILE or die "Can't write
@{[SUSPEND_FILE]}: $!"; print $output2 "10.0.0.252/32\n";
close $output2; } }
sub Scanlog {
my ($flag,$hour,$blocklimit,$matches,)[EMAIL PROTECTED];
open my $slog, "-|", "tail -n 25000 @{[LOG_FILE]}" or die
"Unable to open $log:$!\n"; open my $output, ">",
IP_LIST_FILE or die "Can't write @{[IP_LIST_FILE]}: $!";
open my $output2, ">", SUSPEND_FILE or die "Can't write
@{[SUSPEND_FILE]}: $!"; while (my $line = <$slog>){
# assigns each line in turn to $line #use an array
slice to select the fields we want my ($time, $ip, $url,
$category) = (split " ", $line)[1,4,7,10]; my ($hr) =
split /:/, $time; if($flag eq $category and $hr eq
$hour){ $matches += 1 ; }
if($matches > $blocklimit){
print $output "$matches, $ip, $hour, $time,
$category, $url\n"; print $output2 "$ip/32\n";
$matched = $matches;
$matches = 0;
}
}
close $output;
close $output2;
return($matched,$ip,$hour,$time,$category,$url); }
------------------------------------------------------------------
Ryan Lamberton
----- Original Message -----
From: "Jeff 'japhy' Pinyan" <[EMAIL PROTECTED]>
To: "FamiLink Admin" <[EMAIL PROTECTED]>
Cc: <beginners@perl.org>
Sent: Wednesday, September 28, 2005 12:24 PM
Subject: Re: a little help...
On Sep 28, FamiLink Admin said:
I am trying to read a log file and get a list of how many times
an IP address get blocked each hour by category PO. An example
line in the log with a block is: -------------
[2005-09-28 10:05:03 -7:00] 127.0.0.1 71.32.59.249 216.163.137.3
- http://www.playboy.com/ blocked 0 PO
-------------
What I have kinda works but I am not sure if it is the best
practice. This is the first time programming in perl and this is
what I have so far:
Your indentation leaves much to be desired, so I've "fixed" it.
sub Scanlog {
local($ipb) = @_;
No reason to use 'local'; stick with 'my' here. But... what is
$ipb? You don't use it anywhere!
open my $slog, "-|", "tail -n 50000 $log" or die "Unable to
open $log:$!\n"; open (OUTPUT,">/etc/squid/iplist.txt");
open (OUTPUT2,">/etc/squid/SuspendIpList.txt");
You should also die if neither of those could be opened:
open(OUTPUT, ">...") or die "can't create
/etc/squid/iplist.txt: $!";
while (<$slog>){ # assigns each line in turn to $_
# use an array slice to select the fields we want
@data = (split ,$_)[1,4,10,5,7];
$hr = (split /:/ ,$data[0])[0];
$ip = "$data[1]";
Those three variables should all be declared with 'my'. Your line
assigning to @data has a typo that hasn't effected you, but it
might eventually.
my @data = (split)[1,4,10,5,7]; # why out of order?
my $hr = (split /:/, $data[0])[0];
my $ip = $data[1]; # no need to quote $data[1] here
if ($flag eq $data[2]) {
Where is $flag coming from?
if ($hr eq $hour) {
Where is $hour coming from?
Those two if statements can be combined into one, since you don't
do anything if they aren't both true.
if ($flag eq $data[2] and $hr eq $hour) {
foreach (/$data[2]/) {
$matches += 1 ;
}
I have a feeling this could lead to false positives. How do you
know that 'PO' (or whatever else $data[2] might hold) won't appear
in the URL, for instance? Perhaps this should just be
$matches++;
But where is $matches coming from?!
if ($matches > $blocklimit) {
Where does $blocklimit come from?!
$ip1 = "$data[1]/32";
Declare that with 'my'.
print OUTPUT "$matches,", "$hour, ","$ip1, ",
"@data","\n";
You could just write that as
print OUTPUT "$matches, $hour, $data[1]/32 @data\n";
print OUTPUT2 "$ip1\n";
$matched = $matches;
$matches = 0;
Where did $matched come from?
}
}
}
}
close (OUTPUT);
close (OUTPUT2);
}
You should not use any variables in a function that you did not
pass to it or create IN it.
--
Jeff "japhy" Pinyan % How can we ever be the sold short or
RPI Acacia Brother #734 % the cheated, we who for every
service http://www.perlmonks.org/ % have long ago been overpaid?
http://princeton.pm.org/ % -- Meister Eckhart
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
<http://learn.perl.org/> <http://learn.perl.org/first-response>
*******************************************************
This message contains information that is confidential
and proprietary to FedEx Freight or its affiliates.
It is intended only for the recipient named and for
the express purpose(s) described therein.
Any other use is prohibited.
*******************************************************
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
<http://learn.perl.org/> <http://learn.perl.org/first-response>
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
<http://learn.perl.org/> <http://learn.perl.org/first-response>
