Re: [gentoo-user] apache blocking access based country
On Wednesday, 9 December 2020 07:03:17 GMT Remco Rijnders wrote: > On Tue, Dec 08, 2020 at 11:18:49PM -0700, thelma wrote in > > <94662af9-b159-65ca-371d-1521ab4fa...@sys-concept.com>: > >Steve, suggested a good alternative database of IP's with two letter > >country beside them. It can be sorted by country in a spreadsheet. This > >16MB file would seem like an ideal solution; but checking is it is > >missing 212.114.17 number I've checked earlier from the other source. > >The number 212.114.16 correctly identified as "FR" (France). > > > >deny from 212.114.16.0/24 > >deny from 212.114.17.0/24 > > > >The dbip-country-lite-2020-12.csv lists: > >212.114.10.0 212.114.15.255 RU > >212.114.16.0 212.114.31.255 FR > >212.114.32.0 212.114.49.39 DE > > Just wanted to point out that "212.114.17.0/24" is included in the > "212.114.16.0 212.114.31.255 FR" entry you show, so it doesn't seem to be > missing at all. For reasons of efficiency, it is preferable to have blocks > listed that are bigger than /24's if they roll up to bigger blocks per > country. The above can be written as 212.114.16.0/20 in CIDR block notation. Unwelcome website visitors are difficult to block, without incurring some processing penalty and consequently cause undesirable latency for legitimate visitors. As has already been commented a network/firewall level solution, which drops connections before they arrive at apache is more efficient, because it avoids putting any load on the webserver application itself. Apache configuration files in /etc are loaded in RAM and as long as there is enough memory will be much faster than having to parse .htaccess files for each and every page/visitor. The geoip solution works better when only a few countries are allowed to connect and everyone else is dropped. However, botnets can use VPNs or anonymising proxy servers to bypass such restrictions. Also, IP subnets change all the time, so geoip addresses have to be refreshed regularly. Depending on the website design there are various php and other scripts which function similar to fail2ban, when a visitor tries unsuccessfully to authenticate repeatedly. The sensitivity of the trigger can be adjusted for the number of failed attempts and the time the failed IP address is blocked for. I guess such a script could be deployed along with some firewall geoip block to minimise the load on the webserver. signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] apache blocking access based country
On Tue, Dec 08, 2020 at 11:18:49PM -0700, thelma wrote in <94662af9-b159-65ca-371d-1521ab4fa...@sys-concept.com>: Steve, suggested a good alternative database of IP's with two letter country beside them. It can be sorted by country in a spreadsheet. This 16MB file would seem like an ideal solution; but checking is it is missing 212.114.17 number I've checked earlier from the other source. The number 212.114.16 correctly identified as "FR" (France). deny from 212.114.16.0/24 deny from 212.114.17.0/24 The dbip-country-lite-2020-12.csv lists: 212.114.10.0212.114.15.255 RU 212.114.16.0212.114.31.255 FR 212.114.32.0212.114.49.39 DE Just wanted to point out that "212.114.17.0/24" is included in the "212.114.16.0 212.114.31.255 FR" entry you show, so it doesn't seem to be missing at all. For reasons of efficiency, it is preferable to have blocks listed that are bigger than /24's if they roll up to bigger blocks per country.
Re: [gentoo-user] apache blocking access based country
On 12/08/2020 05:25 PM, Steve Wilson wrote: > > On 09/12/2020 00:01, Grant Taylor wrote: >> On 12/8/20 4:44 PM, Steve Wilson wrote: >>> I use this as the first step to limit ssh access to one of my >>> servers: `iptables -A INPUT -p tcp -m tcp --dport 22 -m geoip ! >>> --src-cc GB -m comment --comment "Drop SSH from outside GB" -j DROP` >> >> Has the geoip match extension been updated to take into account >> MaxMind discontinuing their GeoLite database and the need to support >> GeoLite2? >> > The xt_geoip_dl script grabs a csv from > https://db-ip.com/db/download/ip-to-country-lite. I imagine there's a > method for dealing with maxmind's new version and converting to csv if > they don't already provide one for the paid service. > > Steve Steve, suggested a good alternative database of IP's with two letter country beside them. It can be sorted by country in a spreadsheet. This 16MB file would seem like an ideal solution; but checking is it is missing 212.114.17 number I've checked earlier from the other source. The number 212.114.16 correctly identified as "FR" (France). deny from 212.114.16.0/24 deny from 212.114.17.0/24 The dbip-country-lite-2020-12.csv lists: 212.114.10.0212.114.15.255 RU 212.114.16.0212.114.31.255 FR 212.114.32.0212.114.49.39 DE In addition all numbers 2.61.192.0 is missing as well.
Re: [gentoo-user] apache blocking access based country
P.S. You might also be interested in some of the feeds that Team Cymru has to offer. I think they are more friendly to scripted querying. Link - IP to ASN Mapping Service - https://team-cymru.com/community-services/ip-asn-mapping/ -- Grant. . . . unix || die
Re: [gentoo-user] apache blocking access based country
On 12/8/20 9:59 PM, the...@sys-concept.com wrote: I'll write a script to check, all the IP's from at text file with "whois" and write the output out to another file, just to be sure. I don't know how long will it take, the file contains 26611-entries (IP addresses). ProTip: Don't parse the output from WhoIs directly. Instead save it to a file. Come up with some file naming scheme that encodes the IPs and date. That way you can easily reference them in the future. Or decide that what you have cached is too old and that you need to update it. I say this because a number of WhoIs servers get fairly upset if they think they are being scripted against. So ... space out the queries and save the output for future re-use. You might be correct, Grant. Putting the IP's in apache .config file could be more efficient, instead of .htaccess file. ;-) -- Grant. . . . unix || die
Re: [gentoo-user] apache blocking access based country
On 12/8/20 8:50 PM, the...@sys-concept.com wrote: Creating ACL based on those internet sources eg. https://www.countryipblocks.net/acl.php is not reliable. I pulled a list of Russian and Ukrainian IPs from the above link and checking some of them, I found these two (and possibly more) are French IPs: deny from 212.114.16.0/24 deny from 212.114.17.0/24 I can't say as I'm surprised. IMHO GeoIP feeds are, and always have been, somewhat suspect. You can get information from RIRs based on who the allocated blocks to originally (or last update by them). Or you can get information from a service that tries to be much more accurate. Or you can get information from a Default Free Zone BGP feed. Or any combination of the above. But each thing is different quality and different amounts of work. RIPE's extended delegation list shows 212.114.16.0/21 as being delegated to France. I trust the RIR feeds more. Though, they might not be updated with IPv4 trading and resale market. Personally, I'd extract prefixes of ASNs from a DFZ BGP feed and use that to filter. It will be the most up to date of what a given provider (ASN) is advertising. If "geoip" database is based on similar sources the hole project is not a reliable control method. GeoIP is ... nebulous. You need to consider if you want to proceed with imperfect (or completely wrong data). -- Grant. . . . unix || die
Re: [gentoo-user] apache blocking access based country
On 12/08/2020 08:50 PM, the...@sys-concept.com wrote: [snip] > Creating ACL based on those internet sources eg. > https://www.countryipblocks.net/acl.php > is not reliable. I pulled a list of Russian and Ukrainian IPs from the > above link and checking some of them, I found these two (and possibly > more) are French IPs: > > deny from 212.114.16.0/24 > deny from 212.114.17.0/24 > > If "geoip" database is based on similar sources the hole project is not > a reliable control method. I'll write a script to check, all the IP's from at text file with "whois" and write the output out to another file, just to be sure. I don't know how long will it take, the file contains 26611-entries (IP addresses). You might be correct, Grant. Putting the IP's in apache .config file could be more efficient, instead of .htaccess file.
Re: [gentoo-user] apache blocking access based country
On 12/8/20 6:17 PM, the...@sys-concept.com wrote: so it might be easier to for apache, am I correct? Apache vs iptables is somewhat a preference. Though with Apache, chances are good that you would need to ban in multiple locations, possibly multiple VHOSTs or server wide. (See more below.) Either way, the apache would have to access the database where all the codes are stored or .htaccess file. Or is it easier if I incorporate the IP addressed into main .config file (in apache)? I personally prefer to put things in files that are included directly from the main Apache config file in lieu of .htaccess files. This harks back to a time when checking for a .htaccess file per page request had measurable impact. It just seemed easier to put the content in the main config file and skip looking for and processing .htaccess files on each request. I don't know what would be more efficient, storing the data somewhere outside of Apache and having it check that -or- putting the data in the config / .htaccess file(s). -- Grant. . . . unix || die
Re: [gentoo-user] apache blocking access based country
On 12/08/2020 05:25 PM, Steve Wilson wrote: > > On 09/12/2020 00:01, Grant Taylor wrote: >> On 12/8/20 4:44 PM, Steve Wilson wrote: >>> I use this as the first step to limit ssh access to one of my >>> servers: `iptables -A INPUT -p tcp -m tcp --dport 22 -m geoip ! >>> --src-cc GB -m comment --comment "Drop SSH from outside GB" -j DROP` >> >> Has the geoip match extension been updated to take into account >> MaxMind discontinuing their GeoLite database and the need to support >> GeoLite2? >> > The xt_geoip_dl script grabs a csv from > https://db-ip.com/db/download/ip-to-country-lite. I imagine there's a > method for dealing with maxmind's new version and converting to csv if > they don't already provide one for the paid service. > > Steve Creating ACL based on those internet sources eg. https://www.countryipblocks.net/acl.php is not reliable. I pulled a list of Russian and Ukrainian IPs from the above link and checking some of them, I found these two (and possibly more) are French IPs: deny from 212.114.16.0/24 deny from 212.114.17.0/24 If "geoip" database is based on similar sources the hole project is not a reliable control method.
Re: [gentoo-user] apache blocking access based country
On 12/08/2020 05:25 PM, Steve Wilson wrote: > > On 09/12/2020 00:01, Grant Taylor wrote: >> On 12/8/20 4:44 PM, Steve Wilson wrote: >>> I use this as the first step to limit ssh access to one of my >>> servers: `iptables -A INPUT -p tcp -m tcp --dport 22 -m geoip ! >>> --src-cc GB -m comment --comment "Drop SSH from outside GB" -j DROP` >> >> Has the geoip match extension been updated to take into account >> MaxMind discontinuing their GeoLite database and the need to support >> GeoLite2? >> > The xt_geoip_dl script grabs a csv from > https://db-ip.com/db/download/ip-to-country-lite. I imagine there's a > method for dealing with maxmind's new version and converting to csv if > they don't already provide one for the paid service. > > Steve Thanks. The dbip-country-lite-2020-12.csv is 16MB (file) though, it includes all the countries. The country I want to ban file is only 697kB so it might be easier to for apache, am I correct? Either way, the apache would have to access the database where all the codes are stored or .htaccess file. Or is it easier if I incorporate the IP addressed into main .config file (in apache)?
Re: [gentoo-user] apache blocking access based country
On 09/12/2020 00:01, Grant Taylor wrote: On 12/8/20 4:44 PM, Steve Wilson wrote: I use this as the first step to limit ssh access to one of my servers: `iptables -A INPUT -p tcp -m tcp --dport 22 -m geoip ! --src-cc GB -m comment --comment "Drop SSH from outside GB" -j DROP` Has the geoip match extension been updated to take into account MaxMind discontinuing their GeoLite database and the need to support GeoLite2? The xt_geoip_dl script grabs a csv from https://db-ip.com/db/download/ip-to-country-lite. I imagine there's a method for dealing with maxmind's new version and converting to csv if they don't already provide one for the paid service. Steve
Re: [gentoo-user] apache blocking access based country
On 12/8/20 4:44 PM, Steve Wilson wrote: I use this as the first step to limit ssh access to one of my servers: `iptables -A INPUT -p tcp -m tcp --dport 22 -m geoip ! --src-cc GB -m comment --comment "Drop SSH from outside GB" -j DROP` Has the geoip match extension been updated to take into account MaxMind discontinuing their GeoLite database and the need to support GeoLite2? This has the advantage that apache doesn't need to process the request, but a possible downside that you won't be able to display a message if that's a requirement. You could probably DNAT / REDIRECT to an alternate port that is a different virtual host that serves up a 403 page. -- Grant. . . . unix || die
Re: [gentoo-user] apache blocking access based country
On 08/12/2020 22:55, the...@sys-concept.com wrote: What are my options apache blocking access based on country? So fare I run onto something "geoip" or ACL (long list of IP's provided by eg:) https://www.ip2location.com/free/visitor-blocker We geoip I think I will need to install some module for apache (apache 2.2). It is using geoip.dat so it must be a long list of as well. But they are not offering any free version. wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz (doesn't work) If you don't need to provide information in the browser to blocked users, you could look at net-firewall/xtables-addons with XTABLES_ADDONS="geoip". This will allow you to block access to apache at the network level. I use this as the first step to limit ssh access to one of my servers: `iptables -A INPUT -p tcp -m tcp --dport 22 -m geoip ! --src-cc GB -m comment --comment "Drop SSH from outside GB" -j DROP` This has the advantage that apache doesn't need to process the request, but a possible downside that you won't be able to display a message if that's a requirement. Steve
Re: [gentoo-user] apache blocking access based country
On 12/08/2020 03:55 PM, the...@sys-concept.com wrote: > What are my options apache blocking access based on country? > So fare I run onto something "geoip" or ACL (long list of IP's provided by > eg:) > https://www.ip2location.com/free/visitor-blocker > > We geoip I think I will need to install some module for apache (apache 2.2). > It is using geoip.dat so it must be a long list of as well. But they are not > offering any free version. > wget > http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz > (doesn't work) Just access to the web-page that is hosted by apache. Looking at the IP addresses generated by some of those links, the document is 10-pages long.
Re: [gentoo-user] apache blocking access based country
On 12/8/20 3:55 PM, the...@sys-concept.com wrote: What are my options apache blocking access based on country? Do you want to block connections to /just/ Apache and /nothing/ else on the system? Or do you want to block connections from specified sources to anything and everything on the system? -- Grant. . . . unix || die
[gentoo-user] apache blocking access based country
What are my options apache blocking access based on country? So fare I run onto something "geoip" or ACL (long list of IP's provided by eg:) https://www.ip2location.com/free/visitor-blocker We geoip I think I will need to install some module for apache (apache 2.2). It is using geoip.dat so it must be a long list of as well. But they are not offering any free version. wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz (doesn't work) -- Thelma
