On 11/20/2017 04:25 PM, Kevin A. McGrail wrote:
Dave, can you share the script you used for that? Assuming some grep
of access log and then an awk and a sort on the IP address?
I am using awstats to get some basic information from my Apache HTTPd
logs. That list was the top 10 IPs that awstats shows. Then I did a
little grepping to determine the frequency that put them in the top 10.
I think we'll need to develop a policy that we can then block people
checking more than a certain number of times. I would say since we
publish only 1x per day, that they should check no more than 2x per
day or that they can be banned.
Could we use something like mod_evasive to limit any IP connecting more
than 3 times (one batch of ruleset files) an hour? SA instances behind
NAT'd IPs could cause a legitimate reason for more than 2x hits per day.
If we can share some logs, we can use something simple like an
.htaccess that blocks the IPs.
Do you think we could block some of these completely or should we start
with a more conservative approach with mod_evasive (or something similar)?
There may be some abusers in the future that we would want to
permanently block with a centralized .htaccess file that gets
distributed with the normal rsync pulls by each mirror.
Regards,
KAM
On 11/20/2017 2:14 PM, Dave Jones wrote:
On 11/19/2017 02:33 AM, Matthias Leisi wrote:
Heads up. DNS updates for sa-update have been enabled again. The
next rules promotion will happen in about 11 hours around 2:30 AM UTC.
Traffic on my mirror picked up at around 03:00 UTC (time in chart
below is in UTC+01:00), traffic is at similar level as previously
(300-400 kbyte/s, with some peaks).
— Matthias
I had about 25 GB of traffic on each of my 2 servers behind
sa-update.ena.com -- ~50 GB total yesterday. This would match what
you are seeing at about 300 KB/sec on each mirror -- ~600 KB/sec on
both. On track for 50-55 GB again today.
Here are the top 10 IPs that seem to be running sa-update or a curl
script most frequently:
41.76.211.56 (sa-update/svn917659/3.3.2 every 5 minutes)
108.61.28.10 (sa-update/svn917659/3.3.2 every 15 minutes)
202.191.60.145 (curl/7.19.7 every minute rotating mirrors)
202.191.60.146 (curl/7.19.7 every minute rotating mirrors)
108.163.197.66 (sa-update/svn917659/3.3.2 every 5 minutes)
208.74.121.106 (NAT'd IP? curl/7.29.0 & curl/7.19.7)
91.204.24.253 (NAT'd IP? various user agents)
207.210.201.60
78.110.96.3
190.0.150.3
--
Dave
