Re: ipfw divert filter for IPv4 geo-blocking
> Am 29.07.2016 um 06:50 schrieb Julian Elischer :
> On 29/07/2016 5:22 PM, Julian Elischer wrote:
>> On 29/07/2016 4:53 PM, Dr. Rolf Jansen wrote:
Am 28.07.2016 um 23:48 schrieb Lee Brown :
That makes sense to me. Your /20 range encompasses 201.222.16.0 -
201.222.31.255.
If you want 201.222.20.0-201.222.31.255, you'll need 3 ranges:
201.222.20.0/22 (201.222.20.0-201.222.23.255)
201.222.24.0/22 (201.222.24.0-201.222.27.255)
201.222.28.0/22 (201.222.28.0-201.222.31.255)
>>>
>>> Ian, Julian and Lee,
>>>
>>> Thank you vary much for your responses. In order not bloat the thread, I
>>> answer only to one message.
>>>
>>> I found the problem. As a matter of fact, the respective IP ranges in the
>>> LACNIC delegation statistics file are 3 adjacent blocks with 1024
>>> addresses, i.e. those that you listed in your message above:
>>>
>>> $grep 201.222.2 /usr/local/etc/ipdb/IPRanges/lacnic.dat
>>> lacnic|BR|ipv4|201.222.20.0|1024|20140710|allocated|164725
>>> lacnic|BR|ipv4|201.222.24.0|1024|20140630|allocated|138376
>>> lacnic|BR|ipv4|201.222.28.0|1024|20140701|allocated|129095
>>>
>>> However, my database compilation combines adjacent blocks with the same
>>> country code, and the ranges above turn into one block of 3072 addresses,
>>> which obviously doesn't have a valid netmask - log(3072) = 11,5849625.
>>> ...
>>> ..., it is not sufficient to forget about optimization but I need to check
>>> also whether, the delegation files contain already some non-CIDR ranges,
>>> which need to be broken down.
>>
>> there is code to take ranges and produce cidr sets.
>>
>> We used to have exactly that code in the appletalk code before we took it
>> out. Appletalk uses ranges.
>> https://svnweb.freebsd.org/base/release/3.2.0/sys/netatalk/at_control.c?view=annotate#l703
>>
>
> though htat uassumes input in the form af an appletak sockaddr..
> there is also this python module
> https://pythonhosted.org/netaddr/tutorial_01.html#support-for-non-standard-address-ranges
>
>> maybe you can find other versions on the net.
>> however if you fully populate the table, you will get the correct result
>> because more specific entries will
>> override less specific entries. To do that you would have to have a way to
>> describe to your program what
>> value each table entry should output.
>> If you did what you do now, then you would specify the value for the
>> required countries, and give a default falue for "all others".
>> aggregation of adjacent ranges with same value would be an optimisation.
Don't worry, breaking down an arbitrary IP-range into a CIDR conforming set of
ranges, doesn't seem too difficult. I quickly hacked into the for() loop for
table generation in geoip.c a nested do while() for if necessary breaking down
the given range:
for (i = 0; i < n; i++)
{
if (!*ccList || findCC(CCTable, sortedIPSets[i][2]))
{
ipv4_lo = sortedIPSets[i][0];
do
{
m = intlb(sortedIPSets[i][1] - ipv4_lo + 1);
while (ipv4_lo % (k = (int)lround(exp2(m m--;
printRange(ipv4_lo, 32 - m, table_number, table_value);
}
while ((ipv4_lo += k) < sortedIPSets[i][1]);
}
}
This seems to work as expected, however, I need to check this more carefully
(in the course of the weekend) until pushing it to GitHub. At the first glance,
the tables become quite large. For example, the table for Brazil of 824 joined
entries is bloated to 6621 CIDR records.
Once I came to a conclusion, I will post it to this mailing list.
Best regards
Rolf
___
freebsd-ipfw@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
To unsubscribe, send any mail to "freebsd-ipfw-unsubscr...@freebsd.org"
Re: ipfw divert filter for IPv4 geo-blocking
On 29/07/2016 5:22 PM, Julian Elischer wrote: On 29/07/2016 4:53 PM, Dr. Rolf Jansen wrote: Am 28.07.2016 um 23:48 schrieb Lee Brown : That makes sense to me. Your /20 range encompasses 201.222.16.0 - 201.222.31.255. If you want 201.222.20.0-201.222.31.255, you'll need 3 ranges: 201.222.20.0/22 (201.222.20.0-201.222.23.255) 201.222.24.0/22 (201.222.24.0-201.222.27.255) 201.222.28.0/22 (201.222.28.0-201.222.31.255) Ian, Julian and Lee, Thank you vary much for your responses. In order not bloat the thread, I answer only to one message. I found the problem. As a matter of fact, the respective IP ranges in the LACNIC delegation statistics file are 3 adjacent blocks with 1024 addresses, i.e. those that you listed in your message above: $grep 201.222.2 /usr/local/etc/ipdb/IPRanges/lacnic.dat lacnic|BR|ipv4|201.222.20.0|1024|20140710|allocated|164725 lacnic|BR|ipv4|201.222.24.0|1024|20140630|allocated|138376 lacnic|BR|ipv4|201.222.28.0|1024|20140701|allocated|129095 However, my database compilation combines adjacent blocks with the same country code, and the ranges above turn into one block of 3072 addresses, which obviously doesn't have a valid netmask - log(3072) = 11,5849625. My divert filter daemon is agnostic about this, because the exact ranges are stored in the database and for the purpose of country code lookup the address lookup routine doesn't need to work with netmasks. I choose it this way, because I read some RIR documentation stating that delegated address blocks are not necessarily complete CIDR ranges. Even if the above ranges conform to CIDR, future delegations may be different, and I wanted to stay on the safe side. So far so good. Now, the actual problem with ipfw tables in the given context is that explicit IP ranges are not accepted but ranges must be given in CIDR notation, and I simply forgot about the tiny detail that CIDR is inherently granular, and that this may of course conflict with my CC/IP database optimization strategy. Without combining adjacencies, the complete database has 165815 instead of 83274 records. Perhaps, I add an option to the db tool for CIDR conformity. In this respect, it is not sufficient to forget about optimization but I need to check also whether, the delegation files contain already some non-CIDR ranges, which needs to be broken down. there is code to take ranges and produce cidr sets. We used to have exactly that code in the appletalk code before we took it out. Appletalk uses ranges. https://svnweb.freebsd.org/base/release/3.2.0/sys/netatalk/at_control.c?view=annotate#l703 though htat uassumes input in the form af an appletak sockaddr.. there is also this python module https://pythonhosted.org/netaddr/tutorial_01.html#support-for-non-standard-address-ranges maybe you can find other versions on the net. however if you fully populate the table, you will get the correct result because more specific entries will override less specific entries. To do that you would have to have a way to describe to your program what value each table entry should output. If you did what you do now, then you would specify the value for the required countries, and give a default falue for "all others". aggregation of adjacent ranges with same value would be an optimisation. Best regards Rolf ___ freebsd-ipfw@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ipfw To unsubscribe, send any mail to "freebsd-ipfw-unsubscr...@freebsd.org" ___ freebsd-ipfw@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ipfw To unsubscribe, send any mail to "freebsd-ipfw-unsubscr...@freebsd.org" ___ freebsd-ipfw@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ipfw To unsubscribe, send any mail to "freebsd-ipfw-unsubscr...@freebsd.org"
Re: ipfw divert filter for IPv4 geo-blocking
On 29/07/2016 4:53 PM, Dr. Rolf Jansen wrote: Am 28.07.2016 um 23:48 schrieb Lee Brown : That makes sense to me. Your /20 range encompasses 201.222.16.0 - 201.222.31.255. If you want 201.222.20.0-201.222.31.255, you'll need 3 ranges: 201.222.20.0/22 (201.222.20.0-201.222.23.255) 201.222.24.0/22 (201.222.24.0-201.222.27.255) 201.222.28.0/22 (201.222.28.0-201.222.31.255) Ian, Julian and Lee, Thank you vary much for your responses. In order not bloat the thread, I answer only to one message. I found the problem. As a matter of fact, the respective IP ranges in the LACNIC delegation statistics file are 3 adjacent blocks with 1024 addresses, i.e. those that you listed in your message above: $grep 201.222.2 /usr/local/etc/ipdb/IPRanges/lacnic.dat lacnic|BR|ipv4|201.222.20.0|1024|20140710|allocated|164725 lacnic|BR|ipv4|201.222.24.0|1024|20140630|allocated|138376 lacnic|BR|ipv4|201.222.28.0|1024|20140701|allocated|129095 However, my database compilation combines adjacent blocks with the same country code, and the ranges above turn into one block of 3072 addresses, which obviously doesn't have a valid netmask - log(3072) = 11,5849625. My divert filter daemon is agnostic about this, because the exact ranges are stored in the database and for the purpose of country code lookup the address lookup routine doesn't need to work with netmasks. I choose it this way, because I read some RIR documentation stating that delegated address blocks are not necessarily complete CIDR ranges. Even if the above ranges conform to CIDR, future delegations may be different, and I wanted to stay on the safe side. So far so good. Now, the actual problem with ipfw tables in the given context is that explicit IP ranges are not accepted but ranges must be given in CIDR notation, and I simply forgot about the tiny detail that CIDR is inherently granular, and that this may of course conflict with my CC/IP database optimization strategy. Without combining adjacencies, the complete database has 165815 instead of 83274 records. Perhaps, I add an option to the db tool for CIDR conformity. In this respect, it is not sufficient to forget about optimization but I need to check also whether, the delegation files contain already some non-CIDR ranges, which needs to be broken down. there is code to take ranges and produce cidr sets. We used to have exactly that code in the appletalk code before we took it out. Appletalk uses ranges. https://svnweb.freebsd.org/base/release/3.2.0/sys/netatalk/at_control.c?view=annotate#l703 maybe you can find other versions on the net. however if you fully populate the table, you will get the correct result because more specific entries will override less specific entries. To do that you would have to have a way to describe to your program what value each table entry should output. If you did what you do now, then you would specify the value for the required countries, and give a default falue for "all others". aggregation of adjacent ranges with same value would be an optimisation. Best regards Rolf ___ freebsd-ipfw@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ipfw To unsubscribe, send any mail to "freebsd-ipfw-unsubscr...@freebsd.org" ___ freebsd-ipfw@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ipfw To unsubscribe, send any mail to "freebsd-ipfw-unsubscr...@freebsd.org"
Re: ipfw divert filter for IPv4 geo-blocking
> Am 28.07.2016 um 23:48 schrieb Lee Brown : > > That makes sense to me. Your /20 range encompasses 201.222.16.0 - > 201.222.31.255. > If you want 201.222.20.0-201.222.31.255, you'll need 3 ranges: > > 201.222.20.0/22 (201.222.20.0-201.222.23.255) > 201.222.24.0/22 (201.222.24.0-201.222.27.255) > 201.222.28.0/22 (201.222.28.0-201.222.31.255) Ian, Julian and Lee, Thank you vary much for your responses. In order not bloat the thread, I answer only to one message. I found the problem. As a matter of fact, the respective IP ranges in the LACNIC delegation statistics file are 3 adjacent blocks with 1024 addresses, i.e. those that you listed in your message above: $grep 201.222.2 /usr/local/etc/ipdb/IPRanges/lacnic.dat lacnic|BR|ipv4|201.222.20.0|1024|20140710|allocated|164725 lacnic|BR|ipv4|201.222.24.0|1024|20140630|allocated|138376 lacnic|BR|ipv4|201.222.28.0|1024|20140701|allocated|129095 However, my database compilation combines adjacent blocks with the same country code, and the ranges above turn into one block of 3072 addresses, which obviously doesn't have a valid netmask - log(3072) = 11,5849625. My divert filter daemon is agnostic about this, because the exact ranges are stored in the database and for the purpose of country code lookup the address lookup routine doesn't need to work with netmasks. I choose it this way, because I read some RIR documentation stating that delegated address blocks are not necessarily complete CIDR ranges. Even if the above ranges conform to CIDR, future delegations may be different, and I wanted to stay on the safe side. So far so good. Now, the actual problem with ipfw tables in the given context is that explicit IP ranges are not accepted but ranges must be given in CIDR notation, and I simply forgot about the tiny detail that CIDR is inherently granular, and that this may of course conflict with my CC/IP database optimization strategy. Without combining adjacencies, the complete database has 165815 instead of 83274 records. Perhaps, I add an option to the db tool for CIDR conformity. In this respect, it is not sufficient to forget about optimization but I need to check also whether, the delegation files contain already some non-CIDR ranges, which needs to be broken down. Best regards Rolf ___ freebsd-ipfw@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ipfw To unsubscribe, send any mail to "freebsd-ipfw-unsubscr...@freebsd.org"
