Re: [PLUG] Perl sorting problem...
On Wed, 2011-08-10 at 16:00 -0700, Randal L. Schwartz wrote: > > "Michael" == Michael C Robinson writes: > > >> Show me what might be in $a and $b there, and what order you want them > >> in. > > Michael> I saw $a <=> $b described as magic on the web, I know nothing > Michael> about it. > > It's two elements of the list. > > Michael> Binary format as in I went from a string to a binary number via: > > Michael> Net::IP::ip_iptobin($ip_string,4); > > Oh, those are simple. They're already just a series of bits. Sort them > string-wise: > > my @sorted = sort @original_list; > > Done. :) Yes, awesome, done. Okay, stage 1 completed. For stage 2, I need to take an existing sorted list of binary numbers, sort a new list of these numbers, and combine the two in sorted order. I think I'll use perl to maintain the IP list and build a server to serve requests for these numbers from a C program. First things first, how to sort into a preexisting sorted list new numbers. Can arrays be concatenated in perl? If so, I'm thinking concatenate the new list and the old one and do a sort on the result. ___ PLUG mailing list PLUG@lists.pdxlinux.org http://lists.pdxlinux.org/mailman/listinfo/plug
Re: [PLUG] Perl sorting problem...
> "Michael" == Michael C Robinson writes: >> Show me what might be in $a and $b there, and what order you want them >> in. Michael> I saw $a <=> $b described as magic on the web, I know nothing Michael> about it. It's two elements of the list. Michael> Binary format as in I went from a string to a binary number via: Michael> Net::IP::ip_iptobin($ip_string,4); Oh, those are simple. They're already just a series of bits. Sort them string-wise: my @sorted = sort @original_list; Done. :) -- Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095 http://www.stonehenge.com/merlyn/> Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc. See http://methodsandmessages.posterous.com/ for Smalltalk discussion ___ PLUG mailing list PLUG@lists.pdxlinux.org http://lists.pdxlinux.org/mailman/listinfo/plug
Re: [PLUG] Perl sorting problem...
On Mon, 2011-08-08 at 16:32 -0700, Randal L. Schwartz wrote:
> > "someone" == someone writes:
>
> someone> By the way, I can't get Perl's sort function to do the right thing
> someone> exactly either.
>
> someone> my @sorted_ips=sort {$a <=> $b} @ip_list;
>
> someone> This command sorts the first octet correctly, but not the second,
> someone> third, or fourth octets. The numbers are in binary format when sort
>
> someone> is run on them.
>
> What is "binary format"
>
> Show me what might be in $a and $b there, and what order you want them
> in.
I saw $a <=> $b described as magic on the web, I know nothing about it.
Binary format as in I went from a string to a binary number via:
Net::IP::ip_iptobin($ip_string,4);
___
PLUG mailing list
PLUG@lists.pdxlinux.org
http://lists.pdxlinux.org/mailman/listinfo/plug
Re: [PLUG] Perl sorting problem...
> "someone" == someone writes:
someone> By the way, I can't get Perl's sort function to do the right thing
someone> exactly either.
someone> my @sorted_ips=sort {$a <=> $b} @ip_list;
someone> This command sorts the first octet correctly, but not the second,
someone> third, or fourth octets. The numbers are in binary format when sort
someone> is run on them.
What is "binary format"
Show me what might be in $a and $b there, and what order you want them
in.
Be precise.
--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.posterous.com/ for Smalltalk discussion
___
PLUG mailing list
PLUG@lists.pdxlinux.org
http://lists.pdxlinux.org/mailman/listinfo/plug
Re: [PLUG] Perl sorting problem...
: The last_lowest is the last value written out to the file.
: The tmp value is the current value plucked from the list.
: My hope by going through all the values and picking every one that
: is larger than the last smallest one, the next one will be grabbed.
:
: By the way, I can't get Perl's sort function to do the right thing
: exactly either.
:
: my @sorted_ips=sort {$a <=> $b} @ip_list;
:
: This command sorts the first octet correctly, but not the second,
: third, or fourth octets. The numbers are in binary format when
: sort is run on them.
You have said in several posts that you want to sort them in binary
order. I'm really not sure what that means, but I understand you to
mean that you wish to use the integer representation for sorting
your prefixes.
You could (as somebody had pointed out earlier) use the Net::IP
module, which provides quite a few features. If you want only
sorted IPs, there are some venerable tools at your disposal. I
would suggest using the inet_aton call to turn the IP into a number.
#! /usr/bin/perl
#
# -- sort IPs by integer representation
use strict;
use Socket;
my @ips = readline(STDIN); # -- slurp
chomp(@ips);
@ips =
map { $_->[1] } # -- get second list elem
sort { $a->[0] <=> $b->[0] }# -- sort on first list elem
map { [ unpack( "N", inet_aton( $_ ) ), $_ ] } # -- make IP an integer
@ips;
print join("\n",@ips),"\n";
# -- end of file
Above is a so-called Schwartzian transform on a small list of IPs
(up to a few hundred thousand should be trivial. I ran the above on
a set of 15 million IPs I had sitting around which required about
9GB of RAM to run. The above snippet will not win any efficiency
prizes, but it should accomplish what you want.
: I don't undersand the special syntax on the sort command, but I
: suspect that I need something different there.
: The beauty of the sort function, if it will work, is that I can
: replace an entire function that is fairly long in comparison.
: This is why I'm using perl after all, it is supposed to be
: designed to do this sort of thing.
The usage of $a and $b in the sort function isn't intuitive--you
could say it's almost weird, but it is pretty powerful. If you
haven't already, try 'perldoc -f sort'. There are quite a few
examples.
-Martin
--
Martin A. Brown
http://linux-ip.net/
___
PLUG mailing list
PLUG@lists.pdxlinux.org
http://lists.pdxlinux.org/mailman/listinfo/plug
Re: [PLUG] Perl sorting problem...
someone wrote:
> Quoting Fred James :
>
>
>> someone wrote:
>>
>>> (omissions for brevity)
>>>
>>> if ( $tmp > $last_lowest )
>>> {
>>> $lowest=$tmp;
>>> }
>>>
>>>
>> (omissions for brevity)
>>
>> Quick question ... is this (above) what you really want to say?
>>
>
> The last_lowest is the last value written out to the file.
> The tmp value is the current value plucked from the list.
> My hope by going through all the values and picking every one that
> is larger than the last smallest one, the next one will be grabbed.
>
What I am reading (please correct me if I am wrong) ...
if the temporary_value is greater than the last_lowest_value,
then set the last_lowest_value to be equal to the temporary_value
What I am asking is, is that what you want?
Regards
Fred James
___
PLUG mailing list
PLUG@lists.pdxlinux.org
http://lists.pdxlinux.org/mailman/listinfo/plug
Re: [PLUG] Perl sorting problem...
Quoting Fred James :
> someone wrote:
>> (omissions for brevity)
>
>> if ( $tmp > $last_lowest )
>> {
>> $lowest=$tmp;
>> }
>>
> (omissions for brevity)
>
> Quick question ... is this (above) what you really want to say?
The last_lowest is the last value written out to the file.
The tmp value is the current value plucked from the list.
My hope by going through all the values and picking every one that
is larger than the last smallest one, the next one will be grabbed.
By the way, I can't get Perl's sort function to do the right thing
exactly either.
my @sorted_ips=sort {$a <=> $b} @ip_list;
This command sorts the first octet correctly, but not the second,
third, or fourth octets. The numbers are in binary format when sort
is run on them.
I don't undersand the special syntax on the sort command, but I suspect that
I need something different there.
The beauty of the sort function, if it will work, is that I can
replace an entire function that is fairly long in comparison. This is
why I'm using perl after all, it is supposed to be designed to do this
sort of thing.
My perl script is fairly short right now, but it is long enough to be
obnoxious.
I would have attached it instead, but I don't think attachments are allowed.
I want to give my solution, which this isn't yet, away. To get
tarpitting to work, everyone has to do it eventually. I'm sitting on
200-300 spams which seems to be limited only by my bandwidth at this
point. I don't like MailScanner and I've given up on SpamCannibal,
good idea badly implemented.
Thank you for taking a look.
This message was sent using IMP, the Internet Messaging Program.
___
PLUG mailing list
PLUG@lists.pdxlinux.org
http://lists.pdxlinux.org/mailman/listinfo/plug
Re: [PLUG] Perl sorting problem...
someone wrote:
> (omissions for brevity)
> if ( $tmp > $last_lowest )
> {
> $lowest=$tmp;
> }
>
(omissions for brevity)
Quick question ... is this (above) what you really want to say?
Regards
Fred James
___
PLUG mailing list
PLUG@lists.pdxlinux.org
http://lists.pdxlinux.org/mailman/listinfo/plug
[PLUG] Perl sorting problem...
#!/usr/bin/perl
#{ # Record IPs from lowest to highest to a file...
$last_lowest=$lowest;
$std_ip=Net::IP::ip_bintoip($last_lowest, 4);
print "LAST_LOWEST_IPv4:$std_ip\n";
# Find out if lower value is in list...
for ( $counter = 0 ; $counter < $ips_listed ; ++$counter )
{
$tmp=$ip_list[$counter];
if ( $tmp > $last_lowest )
{
$lowest=$tmp;
}
} # END search for lower value...
$std_ip=Net::IP::ip_bintoip($lowest, 4);
print OUT "Lowest:$lowest\t$std_ip\n";
} # End for loop...
} # END output_IP_list.
#---
# spam_processor.pl: Extract from spam email the originating ip this garbage
#came from.
use Net::IP; # Useful for converting IPv4 strings to binary numbers.
open ( SPAM, "spam") || die "Can't open spam!"; # mbox file containing
the spam.
open ( OUT, ">ip_list") || die "Can't open ip_list!"; # This will contain the
$ips_listed=0; # Total spam IP addresses listed.
$ip_list[0]=0b0; # Start off IP list with 0d0.
$ip_addr="000.000.000.000"; # A default IPv4 string.
$MIN_IP=0b0;
&main; # Program's first function called here ;-)
sub main
{ # This is the first routine that should be called.
while ($origin=)
{
chomp;
if ( $origin =~ /^X-Originating-IP\: \[.*\]$/ )
{
chomp;
# Extract ip address from $origin and chop off garbage...
$ip_addr = substr ( $origin, 19, 16 );
while ( $ip_addr =~ /.*\].*/ )
{
chop ( $ip_addr );
}
$status=&no_dup_enter_into_list;
print "Status: $status \n";
} # End of process acceptable line...
} # End of while loop...
&output_IP_list;
} # END main function.
#---
sub no_dup_enter_into_list
{ # Check for duplicates before inserting candidate IP address.
# Return a string indicating what happened.
# Convert IP string representation to binary integer representation...
$binip=Net::IP::ip_iptobin($ip_addr,4);
for ( $count=0 ; $count < $ips_listed ; ++$count )
{
if ( $binip eq $ip_list[$count] )
{
return ( "Sorry, duplicate found." );
}
}
&enter_into_list;
return ( "Inserted new address!" );
} # END no_dup_enter_into_list function.
sub enter_into_list
{ # Unique spam source found, put IP address in the list...
if ( $ips_listed == 0 )
{
$ip_list[0] = $binip;
}
else
{
$ip_list[$ips_listed] = $binip;
}
++$ips_listed;
} # END enter_into_list function.
#---
sub output_IP_list
{
$lowest=$ip_list[0]; # Hold an IP value to be printed out.
$last_lowest=$ip_list[0]; # Remeber the last IP chosen.
$tmp=0; # Temporarily hold an IP from the list.
$counter=0; # For searches...
for ( $searcher1=0 ; $searcher1 < $ips_listed ; ++$searcher1 )
{ # Force starting out with the lowest value, must find it...
$tmp=$ip_list[$searcher1];
if ( $tmp < $lowest )
{
$lowest=$tmp;
$last_lowest=$lowest;
}
}
$std_ip=Net::IP::ip_bintoip($lowest, 4);
print "LOWEST_IPv4:$std_ip\n";
for ( $counter2=0 ; $counter2 < $ips_listed ; ++$counter2 )
{ # Record IPs from lowest to highest to a file...
$last_lowest=$lowest;
$std_ip=Net::IP::ip_bintoip($last_lowest, 4);
print "LAST_LOWEST_IPv4:$std_ip\n";
# Find out if lower value is in list...
for ( $counter = 0 ; $counter < $ips_listed ; ++$counter )
{
$tmp=$ip_list[$counter];
if ( $tmp > $last_lowest )
{
$lowest=$tmp;
}
} # END search for lower value...
$std_ip=Net::IP::ip_bintoip($lowest, 4);
print OUT "Lowest:$lowest\t$std_ip\n";
} # End for loop...
} # END output_IP_list.
#---
The function output_IP_list makes 5 picks and then picks the same IP
like 30 times. This is clearly the wrong behavior. Why is this
happening and how do I
fix this? I'm trying to list the IPs in order in a file so
