I believe the cause of this problem was bad bots ignoring the robots.txt
excessively hitting the ruleqa.cgi which takes a second or two to run
eating up memory and CPU.
I have setup this bad bot blocker to prevent this from happening again:
https://github.com/mitchellkrogza/apache-ultimate-bad-bot-blocker
Let's keep an eye on things for a bit and see if this helps keep this
server from falling over again.
Dave
On 01/05/2018 09:17 AM, Dave Jones wrote:
On 01/04/2018 08:07 PM, Kevin A. McGrail wrote:
On 1/4/2018 9:03 PM, Dave Jones wrote:
VM good now. Chris bumped it up to 16 GB. I am not complaining but
I think that something is going wrong that will eat up all the RAM no
matter how much there is.
I am really not used to unstable Linux boxes no matter the flavor.
This is, however, the only single one I have without swap.
The OOM killer started killing processes so it was out of memory. There
were a lot of apache2 and ruleqa.cgi processes.
The Apache logs show a lot of bots so I will setup a robots.txt to keep
them from following all of the links of the page for no good reason.
The last OOM killer problem started at 1/4/18 11:41 UTC and at that time
the Apache logs show several bots hitting the server at once. Hopefully
the robots.txt will solve our problem but we may need to enable Apache
max connections to prevent too many ruleqa.cgi's from launching at the
same time. Someone could easily open a few dozen/hundred connections
and DOS the server.
Dave
