FamiLink Admin wrote:
> That works! Now I see data!
>
> $VAR1 = {
> '70.117.26.250' => '1'
> };
> $VAR1 = {
> '70.117.26.250' => '1',
> '71.32.59.249' => '1'
> };
> $VAR1 = {
> '70.117.26.250' => '1',
> '71.32.59.249' => '2'
> };
> $VAR1 = {
> '70.117.26.250' => '1',
> '71.32.59.249' => '3'
> };
>
>
> The foreach loop does not seem to work and I get this error:
You are not using strict, so it is working and this is just informative.
What you are missing is that you collect all the data first which meets your
criteria. At the end of that processing, then you go through the foreach
checking. You have to be on the outside of the whlle loop:
while (my $line = <$slog>){ # assigns each line in turn to $line
#use an array slice to select the fields we want
my ($time, $ip, $url, $category) = (split " ", $line)[1,4,7,10];
my ($hr) = split /:/, $time;
if($flag eq $category and $hr eq $hour){
$MyIpAddrInfo->{$ip}++;
}
}
foreach $MyIpAddr (sort keys %{MyIpAddrInfo}) {
next if ( $MyIpAddrInfo->{MyIpAddr} <= $blocklimit ); # if less than
or equal get next key
print $output2 "$ip/32\n";
}
print $output1 Dumper(\%{$MyIpAddrInfo});
close $output;
Something along this line as a starter.
You should get use to using strict and warnings. Makes life much easier if you
get started that way.
Wags ;)
>
> Name "main::MyIpAddrInfo" used only once: possible typo at ./test2.pl
> line
> 48.
>
> ---------------------------------------------------------------------------------
> #!/usr/bin/perl -w
> require Mail::Send;
> use Data::Dumper;
> use constant IP_LIST1_FILE => "/etc/squid/iplist1.txt";
> use constant IP_LIST_FILE => "/etc/squid/iplist.txt";
> use constant SUSPEND_FILE => "/etc/squid/SuspendIpList.txt";
> use constant LOG_FILE => "/opt/n2h2/logs/filter_log";
> {
> my $sysop = "[EMAIL PROTECTED]";
> my $flag = "PO";
> my $hour = (localtime)[2];
> my $matches = 0;
> my ($matched,$ip2) = &Scanlog($flag,$hour,$matches);
> if($matched > 1){
> $msg = new Mail::Send Subject=>'SuspendIpList',
> To=>"$sysop"; $fh = $msg->open;
> print $fh "Someone has tried to access $matched banned sites
> today\n";
> print $fh "Their IP address ($ip2) has been added to
> /etc/squid/SuspendIpList.txt\n";
> print $fh "To unblock them, remove their entry from the
> file and run squid -k reconfigure\n";
> print $fh "$matched, $ip2\n";
> $fh->close; # complete the message and send it
> $matched = 0;
> }
> else{
> open my $output2, ">", SUSPEND_FILE or die "Can't write
> @{[SUSPEND_FILE]}: $!";
> print $output2 "10.0.0.252/32\n";
> close $output2;
> }
> }
> sub Scanlog {
> my ($flag,$hour,$matches)[EMAIL PROTECTED];
> my $blocklimit = 1;
> $matched = 0;
> open my $slog, "-|", "tail -n 25000 @{[LOG_FILE]}" or die
> "Unable to open @{[LOG_FILE]}: $!";
> open my $output, ">", IP_LIST_FILE or die "Can't write
> @{[IP_LIST_FILE]}: $!";
> open my $output1, ">", IP_LIST1_FILE or die "Can't write
> @{[IP_LIST1_FILE]}: $!";
> open my $output2, ">", SUSPEND_FILE or die "Can't write
> @{[SUSPEND_FILE]}: $!";
> my %MIAI = ();
> my $MyIpAddrInfo = \%MIAI;
> while (my $line = <$slog>){ # assigns each line in turn
> to $line #use an array slice to select the fields we want
> my ($time, $ip, $url, $category) = (split " ",
> $line)[1,4,7,10]; my ($hr) = split /:/, $time;
> if($flag eq $category and $hr eq $hour){
> $MyIpAddrInfo->{$ip}++;
> print $output Dumper(\%{$MyIpAddrInfo});
> }
> foreach $MyIpAddr (sort keys %{MyIpAddrInfo}) {
> next if ( $MyIpAddrInfo->{MyIpAddr} <= $blocklimit );
> # if less than or equal get next key
> print $output1 Dumper(\%{$MyIpAddrInfo});
> print $output2 "$ip/32\n";
> $matched = $matches;
> $matches = 0;
> $ip2 = $ip;
> }
> }
> close $output;
> close $output2;
> return $matched,$ip2;
> }
>
>
> ----------------------------------------------------
>
> Ryan Lamberton
>
>
> ----- Original Message -----
> From: "Wagner, David --- Senior Programmer Analyst --- WGO"
> <[EMAIL PROTECTED]>
> To: "FamiLink Admin" <[EMAIL PROTECTED]>
> Cc: <beginners@perl.org>
> Sent: Thursday, September 29, 2005 3:15 PM
> Subject: RE: a little help...
>
>
> FamiLink Admin wrote:
>> David,
>> Thank you for your help! I am trying to get this to work. Can you
>> tell me what my $MyIpAddrInfo = \%MIAI; does? I am getting
> This is making $MyIpAddInfo a refeence to %MIAI so to access the data
> you use the -> as the connector vs $MIAI{}.
>
>>
>> HASH(0x8133528)
>>
>> for $MyIpAddrInfo if I print it out after the:
> Since you have only given me this, it should be and I looked at whait
> i sent you:
>
> foreach $MyIpAddr (sort keys %{MyIpAddrInfo}) {
> next if ( $MyIpAddrInfo->{MyIpAddr} <= $blocklimit ); # if less than
> or equal get next key
>
> the best way to see what you have is using Data::Dumper.
> Add a use Data::Dumper at top of script.
> I then would add at a high level:
> $Data::Dumper::Sortkeys = 1; # sort the keys when printing.
>
> Then you can do something like:
>
> print Dumper(\%{$MyIpAddrInfo}); # in thise case will go to std out.
> if a lot of data, then open a disk file and write to that.
> Also you can before the print place this line:
> $Data::Dumper::Varname = 'MyIpAddrInfo'; # this is name will appear
> at the beginning of the output
> print Dumper(\%{$MyIpAddrInfo}); # in thise case will go to std out.
> # if a lot of data, then open a disk file and write to
> Wags ;)
>>
>> $MyIpAddrInfo->{$ip}++;
>>
>>
>> Ryan Lamberton
>>
>>
>> ----- Original Message -----
>> From: "Wagner, David --- Senior Programmer Analyst --- WGO"
>> <[EMAIL PROTECTED]>
>> To: "FamiLink Admin" <[EMAIL PROTECTED]>
>> Cc: <beginners@perl.org>
>> Sent: Wednesday, September 28, 2005 5:53 PM
>> Subject: RE: a little help...
>>
>>
>> FamiLink Admin wrote:
>>> I am only concerned about the IP. The rest is just to verify the
>>> data for now. What code would I use to key the $IP in to hash for
>>> counting?. Most of the IP's are not static but are from broadband
>>> and don't change too often. An example log is:
>>>
>>> -------------
>>> [2005-09-28 10:05:03 -7:00] 127.0.0.1 71.32.59.249 216.163.137.3 -
>>> http://www.playboy.com/ blocked 0 PO
>>> -------------
>>> the IP I want to count is 71.32.59.249 (for this log) and the
>>> category is PO
>>>
>> I would do something like:
>> my %MIAI = ();
>> my $MyIpAddrInfo = \%MIAI;
>> Now as you go through the scan loop, you would take the if which is
>> doing the check on the $flag and the do something like
>> $MyIpAddInfo->{$ip}++; Now you complete your scan and then run
>> throuh your loop like:
>>
>> foreach $MyIpAddr (sort keys %{MyIpAddrInfo}) {
>> next if ( $MyIpAddrInfo->{MyIpAddr} <= $blocklimit ); # if less than
>> or equal get next key # write your suspend and you could put
>> together your email at the same time }
>>
>> A start.
>>
>> Wags ;)
>>
>>> Ryan Lamberton
>>>
>>>
>>> ----- Original Message -----
>>> From: "Wagner, David --- Senior Programmer Analyst --- WGO"
>>> <[EMAIL PROTECTED]>
>>> To: "FamiLink Admin" <[EMAIL PROTECTED]>
>>> Cc: <beginners@perl.org>
>>> Sent: Wednesday, September 28, 2005 5:18 PM
>>> Subject: RE: a little help...
>>>
>>>
>>> FamiLink Admin wrote:
>>>> Jeff ,
>>>> Thanks for all your help! This is what I have now (below and this
>>>> time the whole thing): I think I have included all that you
>>>> talked about plus others:
>>>>
>>>> The sub scanlog does write the information to the files but it does
>>>> not return anything back to the main program and I also get the
>>>> error:
>>>>
>>>> Use of uninitialized value in split at ./test.pl line 9.
>>>>
>>>> Also, is there a better way of counting the number of times each IP
>>>> address gets blocked with category PO? Each time I get to the
>>>> blocklimit it writes to the file but I really just want the max
>>>> number of blocks over the limit. It will write the same IP each
>>>> time it gets over the blocklimit though.
>>>
>>>
>>> If you are only concerned about $ip and if they went over that limit
>>> and not desiring the detail of said offense, then you could use the
>>> $ip as a key into a hash. Then you could count all the occurances.
>>> At the conclusion of that processing then you could loop through the
>>> hash and any count greater than your max, then you could write to
>>> the suspend file. For email, then could again use the hash to put
>>> together a list of $ip's that are over your limit.
>>>
>>> I have not followed the topic, but unless you do something with the
>>> $ip, I would assume that the log is just that a log. You would have
>>> interspersed $ip and so I am unsure how you would be able to say $ip
>>> is at fault. I see nothing in your code which isolates to the $ip.
>>> Again, are these static ip addr or when someone logs out, they are
>>> ready for use by someone else. If it is released then you have to
>>> figure out when this occurs to get an accurate rcd. If static, then
>>> not a problem.
>>>
>>> Wags ;)
>>>
>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> #!/usr/bin/perl -w require Mail::Send;
>>>> $|=1; # no buffering
>>>> use constant IP_LIST_FILE => "/etc/squid/iplist.txt";
>>>> use constant SUSPEND_FILE => "/etc/squid/SuspendIpList.txt";
>>>> use constant LOG_FILE => "/opt/n2h2/logs/filter_log";
>>>> my $sysop = "[EMAIL PROTECTED]";
>>>> my $flag = "PO";
>>>> my $hour = (split, localtime)[2];
>>>> my $blocklimit = 5;
>>>> my $matches = 0;
>>>> my $matched = 0;
>>>> {
>>>> ($matched,$ip,$hour,$time,$category,$url) =
>>>> &Scanlog($flag,$hour,$blocklimit,$matches,);
>>>> if($matched > $blocklimit){
>>>> $msg = new Mail::Send Subject=>'SuspendIpList',
>>>> To=>"$sysop"; $fh = $msg->open;
>>>> print $fh "Someone has tried to access $matches banned
>>>> sites today\n"; print $fh "Their IP address ($ip) has
>>>> been added to /etc/squid/SuspendIpList.txt\n";
>>>> print $fh "To unblock them, remove their entry from the
>>>> file and run squid -k reconfigure\n";
>>>> print $fh "$matches, $ip, $hour, $time, $category,
>>>> $url\n"; $fh->close; # complete the message and
>>>> send it $matched = 0; } else{
>>>> open my $output2, ">", SUSPEND_FILE or die "Can't write
>>>> @{[SUSPEND_FILE]}: $!"; print $output2 "10.0.0.252/32\n";
>>>> close $output2; } }
>>>> sub Scanlog {
>>>> my ($flag,$hour,$blocklimit,$matches,)[EMAIL PROTECTED];
>>>> open my $slog, "-|", "tail -n 25000 @{[LOG_FILE]}" or die
>>>> "Unable to open $log:$!\n"; open my $output, ">",
>>>> IP_LIST_FILE or die "Can't write @{[IP_LIST_FILE]}: $!";
>>>> open my $output2, ">", SUSPEND_FILE or die "Can't write
>>>> @{[SUSPEND_FILE]}: $!"; while (my $line = <$slog>){
>>>> # assigns each line in turn to $line #use an array
>>>> slice to select the fields we want my ($time, $ip, $url,
>>>> $category) = (split " ", $line)[1,4,7,10]; my ($hr) =
>>>> split /:/, $time; if($flag eq $category and $hr eq
>>>> $hour){ $matches += 1 ; }
>>>> if($matches > $blocklimit){
>>>> print $output "$matches, $ip, $hour, $time,
>>>> $category, $url\n"; print $output2 "$ip/32\n";
>>>> $matched = $matches;
>>>> $matches = 0;
>>>> }
>>>> }
>>>> close $output;
>>>> close $output2;
>>>> return($matched,$ip,$hour,$time,$category,$url); }
>>>>
>>>>
>>>>
>>>> ------------------------------------------------------------------
>>>> Ryan Lamberton
>>>>
>>>>
>>>> ----- Original Message -----
>>>> From: "Jeff 'japhy' Pinyan" <[EMAIL PROTECTED]>
>>>> To: "FamiLink Admin" <[EMAIL PROTECTED]>
>>>> Cc: <beginners@perl.org>
>>>> Sent: Wednesday, September 28, 2005 12:24 PM
>>>> Subject: Re: a little help...
>>>>
>>>>
>>>>> On Sep 28, FamiLink Admin said:
>>>>>
>>>>>> I am trying to read a log file and get a list of how many times
>>>>>> an IP address get blocked each hour by category PO. An example
>>>>>> line in the log with a block is: -------------
>>>>>> [2005-09-28 10:05:03 -7:00] 127.0.0.1 71.32.59.249 216.163.137.3
>>>>>> - http://www.playboy.com/ blocked 0 PO
>>>>>> -------------
>>>>>> What I have kinda works but I am not sure if it is the best
>>>>>> practice. This is the first time programming in perl and this is
>>>>>> what I have so far:
>>>>>
>>>>> Your indentation leaves much to be desired, so I've "fixed" it.
>>>>>
>>>>>> sub Scanlog {
>>>>>> local($ipb) = @_;
>>>>>
>>>>> No reason to use 'local'; stick with 'my' here. But... what is
>>>>> $ipb? You don't use it anywhere!
>>>>>
>>>>>> open my $slog, "-|", "tail -n 50000 $log" or die "Unable to
>>>>>> open $log:$!\n"; open (OUTPUT,">/etc/squid/iplist.txt");
>>>>>> open (OUTPUT2,">/etc/squid/SuspendIpList.txt");
>>>>>
>>>>> You should also die if neither of those could be opened:
>>>>>
>>>>> open(OUTPUT, ">...") or die "can't create
>>>>> /etc/squid/iplist.txt: $!";
>>>>>
>>>>>> while (<$slog>){ # assigns each line in turn to $_
>>>>>> # use an array slice to select the fields we want
>>>>>> @data = (split ,$_)[1,4,10,5,7];
>>>>>> $hr = (split /:/ ,$data[0])[0];
>>>>>> $ip = "$data[1]";
>>>>>
>>>>> Those three variables should all be declared with 'my'. Your line
>>>>> assigning to @data has a typo that hasn't effected you, but it
>>>>> might eventually.
>>>>>
>>>>> my @data = (split)[1,4,10,5,7]; # why out of order?
>>>>> my $hr = (split /:/, $data[0])[0];
>>>>> my $ip = $data[1]; # no need to quote $data[1] here
>>>>>
>>>>>> if ($flag eq $data[2]) {
>>>>>
>>>>> Where is $flag coming from?
>>>>>
>>>>>> if ($hr eq $hour) {
>>>>>
>>>>> Where is $hour coming from?
>>>>>
>>>>> Those two if statements can be combined into one, since you don't
>>>>> do anything if they aren't both true.
>>>>>
>>>>> if ($flag eq $data[2] and $hr eq $hour) {
>>>>>
>>>>>> foreach (/$data[2]/) {
>>>>>> $matches += 1 ;
>>>>>> }
>>>>>
>>>>> I have a feeling this could lead to false positives. How do you
>>>>> know that 'PO' (or whatever else $data[2] might hold) won't appear
>>>>> in the URL, for instance? Perhaps this should just be
>>>>>
>>>>> $matches++;
>>>>>
>>>>> But where is $matches coming from?!
>>>>>
>>>>>> if ($matches > $blocklimit) {
>>>>>
>>>>> Where does $blocklimit come from?!
>>>>>
>>>>>> $ip1 = "$data[1]/32";
>>>>>
>>>>> Declare that with 'my'.
>>>>>
>>>>>> print OUTPUT "$matches,", "$hour, ","$ip1, ",
>>>>>> "@data","\n";
>>>>>
>>>>> You could just write that as
>>>>>
>>>>> print OUTPUT "$matches, $hour, $data[1]/32 @data\n";
>>>>>
>>>>>> print OUTPUT2 "$ip1\n";
>>>>>> $matched = $matches;
>>>>>> $matches = 0;
>>>>>
>>>>> Where did $matched come from?
>>>>>
>>>>>> }
>>>>>> }
>>>>>> }
>>>>>> }
>>>>>> close (OUTPUT);
>>>>>> close (OUTPUT2);
>>>>>> }
>>>>>
>>>>> You should not use any variables in a function that you did not
>>>>> pass to it or create IN it.
>>>>>
>>>>> --
>>>>> Jeff "japhy" Pinyan % How can we ever be the sold short or
>>>>> RPI Acacia Brother #734 % the cheated, we who for every
>>>>> service http://www.perlmonks.org/ % have long ago been overpaid?
>>>>> http://princeton.pm.org/ % -- Meister Eckhart
>>>>>
>>>>> --
>>>>> To unsubscribe, e-mail: [EMAIL PROTECTED]
>>>>> For additional commands, e-mail: [EMAIL PROTECTED]
>>>>> <http://learn.perl.org/> <http://learn.perl.org/first-response>
>>>
>>>
>>>
>>> *******************************************************
>>> This message contains information that is confidential
>>> and proprietary to FedEx Freight or its affiliates.
>>> It is intended only for the recipient named and for
>>> the express purpose(s) described therein.
>>> Any other use is prohibited.
>>> *******************************************************
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
<http://learn.perl.org/> <http://learn.perl.org/first-response>
