As promised, I am providing the community with a number of the small software tools that I have written to assist me in analyzing both IP address allocations and ASN allocations globally and in each of the five regions. I retain the copyrights on these tools, but I hereby grant permission for them to be used freely in non-commercial contexts or for research purposes. Anyone wishing to use any of these tools in a commercial context should contact me before doing so in order to obtain an appropriate license.
Note that I am providing rather substantially more in the way of tools than I had promised. I am doing so in the hope that I may perhaps help people to get a at least some understanding of the full process I have been using to find bogon route objects in various IRRs. (I am *not* yet providing the additional script needed to perform that separate task however. I will do so if others are sincerly interested in performing their own hunts for bogon route objects.) I will first provide links to the software tools that I am sharing today. Following those links below I also provide some small description of each, including usage instructions. Note that none of these tools are in any sense "well documented". I *do not* have and do not provide formal man pages for any of them. As far as I am concerned these tools are all too small and trivial to require or warrant that kind of serious documentation. get-rir-stats.bash https://pastebin.com/raw/URX0KN35 isort.pl https://pastebin.com/raw/peeZgmXE stats2asns.pl https://pastebin.com/raw/SUgGNRUA stats2ipv4cidrs.pl https://pastebin.com/raw/A5d5iTrb stats2ipv6cidrs.pl https://pastebin.com/raw/VaA5Z9Uf Range2cidrs.pm https://pastebin.com/raw/bU2RtNdL Note that the final link above is to a Perl library module. You may install that anywhere you like on your system, but the path to the containing directory, whatever that is, must then be substituted into line 11 of the stats2ipv4cidrs.pl script. (In my case, I place library code on my system into the directory /usr/local/rfg/lib and I install my actual tools into the local directory /usr/local/rfg/bin which has also been added to my ${PATH}.) The get-rir-stats.bash script is one that I am using, via a daily cron job, to fetch all of the daily RIR stats files. This script also fetches the daily NRO stats file, however only the portion of that which pertains to IANA reserved IP address blocks and IANA reserved ASNs is used because experience has shown that the other material therein may not be as fresh as what is contained in the daily stats files obtained from the five RIRs themselves. In addition to fetching the daily stats files, the get-rir-stats.bash script also makes use of the various Perl scripts linked to above to perform some rather trivial textual transformations on the daily stats file. These transformations simply eliminate non-useful data and render what remains into a simpler and more easily used notation and format. Before installing, you should edit the definition of 'workdir' on line 8 of the get-rir-stats.bash script to suit you local preferences. On a relatively low-speed DSL line, the get-rir-stats.bash script takes approximately 3 minutes to run. If you are running the script from the command line and NOT as a cron job, then it is recommended that you provide the -v option when invoking get-rir-stats.bash. If you do so, then get-rir-stats.bash will dutifully pass that option down to the various Perls tools when they are invoked, and they themselves will provide more information about any unexpected anomalies in the input data (i.e. within the daily stats files). Anomalies in the daily stats file *do* happen, but these are generally due to transient strangness relating to specific IP blocks that have just been transfered between regions and that are thus still in a state of flux. (These occasional anomalies usually disappear within 24 hours.) The get-rir-stats.bash script assumes that you have the widely used curl program installed and in your ${PATH}. If you do not, then you will need to get and install that before using get-rir-stats.bash. Other external utility programs used by get-rir-stats.bash are as follows: sort -- UNIX built-in isort -- Perl script to sort lines in proper IPv4 order stats2asns -- Perl script to extract ASN ranges from stats files stats2ipv4cidrs -- Perl script to extract IPv4 CIDRs from stats files stats2ipv6cidrs -- Perl script to extract IPv6 CIDRs from stats files With the exception of the standard UNIX "sort" program, which you should already have installed and in your ${PATH}, the other supporting utilities are provided at the links given above. Note that all of these utilities must be installed with a normal sort of execute permission (generally 0755). Also the pathnames given in the very first line of each script will likely need to be edited to suit your local environment, and specifically the path to your local Perl interpreter. (This is true also in the case of the get-rir-stats.bash script. You may need to edit the first line of that so that the path given there will point to your local bash interpreter.) By default each of the stats2*.pl scripts selects out of its input file(s) only those IP blocks or ASNs that are listed in the input file(s) as being "assigned". This default mode is useful when hunting bogons, since when one is hunting bogons, one basically wants to look at a set of candidate IPs or IP block and then filter -out- all of the stuff associated with RIR "assigned" blocks. Whatever is left over is bogons. The various stats2* Perl scripts support a few common options which are worthy of note, specifically: -v -- verbosely report anomalies to stderr -a -- include in the output items marked as "allocated" -r -- include in the output items marked as "reserved" -d -- include in the output items marked as any of the following: assigned, allocated, available, reserved See the get-rir-stats.bash script for examples of how these options may be used in practice to achieve various results. After running the get-rir-stats.bash script you should be left with the following set of files in your chosen ${workdir}: all.asns.assigned all.asns.delegated all.ipv4cidrs.assigned all.ipv4cidrs.delegated all.ipv6cidrs.assigned all.ipv6cidrs.delegated rir.asns.assigned rir.ipv4cidrs.assigned rir.ipv6cidrs.assigned stats.afrinic stats.apnic stats.arin stats.iana stats.lacnic stats.nro stats.ripe The various stats.* files are the daily stats files as fetched from the revelant sources. Note that the stats.iana file is artificial however. This file is composed of just the lines from the stats.nro file that begin with "iana". (This file is thus a lot smaller than the whole stats.nro file, so it is esiler and faster to work with.) The rir.*.assigned files contain summarized lists of the number resources that have been assigned by any actual RIR to any actual resource member. In the case of IPv4 address blocks these will have been helpfully converted into traditional IPv4 CIDR notation. (The contents of these files are used in my bogon route object hunting efforts.) The all.*.delegated files contain lists of everything that has been delegated by IANA to the various RIRs *plus* any number resources, or blocks thereof, that are reserved by IANA itself. (The contents of these files are useful for directing the actions of my own "ipw" -- IP WHOIS utility -- so that it will direct each query about any arbitrary IP address straight to the specific RIR WHOIS server that either has or that should have the relevant WHOIS record, or which will return some definitive result that, in effect, says "this IP isn't assigned to any resource member at present".) The all.*.assigned files contain lists of everything that has been assigned by any RIR, plus those number resources that are reserved by IANA. (I don't actually remember what I am using these files for anymore, or if I am even using them at all.) That's it. I will try to answer questions if there are any, but please DO NOT assume that my time is free. One last thing. The original point of interest that prompted me to post these tools was that some folks wanted to be able to get a nicely IPv4 CIDR-ized list of every IPv4 block that is currently under the administration of AFRINIC, specifically, i.e. every IPv4 block that has been delegated by IANA to AFRINIC at the present time. Assuming that one has already run the get-rir-stats.bash script (and thus has already fetched the various daily stats files) then to get such a list, you should be able to just do the following: cd ${workdir} stats2ipv4cidrs -d stats.afrinic Output will be written to stdout. Enjoy! Regards, rfg P.S. Note that if you add up all of the stuff in the all.ipv4cidrs.delegated file, you get the entire IPv4 address range from 0.0.0.0 to 255.255.255.255, with no gaps and no overlaps. I know. I checked. (I do not know yet if the same can also be said in the case of the all.ipv6cidrs.delegated file or the all.asns.delegated file. I haven't checked those yet.) P.P.S. Writing & running these tools has revealed a few glitches in things globally. The scripts try to correct for these errors. For example, both ARIN and IANA try to lay claim to AS112. My belief is that only IANA's claim is a valid one. In a similar vein, both ARIN and IANA claim 192.175.48.0/24, but in my opinion it rightly only belongs to ARIN. (This block is *not* reserved by any RFC as far as I know, and thus IANA should butt out.) Lastly, APNIC has effectively tried to claim two IPv4 blocks reserved by RFC 5737, namely 198.51.100.0/24 and 203.0.113.0/24. But so has IANA! In my opinion, the blocks have global meaning as laid out in RFC 5737, and thus APNIC is in the wrong here, and the blocks effectively belong only to IANA. I will be notifying APNIC about that issue shortly. _______________________________________________ DBWG mailing list [email protected] https://lists.afrinic.net/mailman/listinfo/dbwg
