Even if it's a DDOS, there is high likelihood that there are several hosts (zombies usually) that are working 'together' to flood your server. It would be reasonably trivial to implement a dynamic iptables script to cut those off at the knees as they're happening, and notify you about the attack... (looking for code now... I know I've done this in the past. Now where did I put those bits? ).
When/ if I find my script I'll pass it on. Rubin On Thu, 2008-05-08 at 14:35 -0400, Nick Floersch wrote: > Phil, this is the Apache config stuff I was thinking of. It *might* be > helpful only in controlling what your server does with itself when lots > of connections start rolling in. If The 255 limit is eating up memory > needed by other servers or processes, you could trim back that limit to > allow other stuff to use the resources, for example. > > Here is an example from my config file: > --- snip --- > ## > ## Server-Pool Size Regulation (MPM specific) > ## > > # prefork MPM > # StartServers ......... number of server processes to start > # MinSpareServers ...... minimum number of server processes which are > kept spare > # MaxSpareServers ...... maximum number of server processes which are > kept spare > # MaxClients ........... maximum number of server processes allowed to > start > # MaxRequestsPerChild .. maximum number of requests a server process > serves > <IfModule prefork.c> > StartServers 5 > MinSpareServers 5 > MaxSpareServers 10 > MaxClients 20 > MaxRequestsPerChild 0 > </IfModule> > > # pthread MPM > # StartServers ......... initial number of server processes to start > # MaxClients ........... maximum number of server processes allowed to > start > # MinSpareThreads ...... minimum number of worker threads which are > kept spare > # MaxSpareThreads ...... maximum number of worker threads which are > kept spare > # ThreadsPerChild ...... constant number of worker threads in each > server process > # MaxRequestsPerChild .. maximum number of requests a server process > serves > <IfModule worker.c> > StartServers 2 > MaxClients 150 > MinSpareThreads 25 > MaxSpareThreads 75 > ThreadsPerChild 25 > MaxRequestsPerChild 0 > </IfModule> > > --- snip --- > So, those are examples. The docs are at this URL: > http://httpd.apache.org/docs/2.2/mod/mpm_common.html#serverlimit > > But you may also want to look for your timeout directives to decrease > how long those threads sit waiting. Also, the KeepAlive settings may be > fouling up somehow, and worth changing a bit. > --- snip --- > Timeout 300 > > KeepAlive On > > MaxKeepAliveRequests 100 > > KeepAliveTimeout 15 > --- snip --- > > The URL for that stuff is in the Core directive docs. > http://httpd.apache.org/docs/2.2/mod/core.html#timeout > > http://httpd.apache.org/docs/2.2/mod/core.html#keepalive > > http://httpd.apache.org/docs/2.2/mod/core.html#maxkeepaliverequests > > http://httpd.apache.org/docs/2.2/mod/core.html#keepalivetimeout > > > Hopefully something here could help tune Apache to handle these requests > better. I can't help much more than you know on blocking the requests, > however. I've never seen log entries like yours where there is no IP > address recorded for the connection. That seems pretty odd to me. > > -Nick > > -----Original Message----- > From: Vermont Area Group of Unix Enthusiasts [mailto:[EMAIL PROTECTED] > On Behalf Of Nick Floersch > Sent: Thursday, May 08, 2008 1:37 PM > To: [email protected] > Subject: Re: Apache Problem > > Have you ruled out a deliberate DOS attack? > > How quickly does it shoot to 255 processes? There is an option to > increase the number of child threads the master can split off... but I > suspect that increasing the limit would just provide more room for it to > fill up. There might be some way to limit the number of connections per > client. Still, the Apache worker thread controls might be helpful to > read about here. > > Are all of the connections from the same client IP? Or are you getting > actions from lots of IPs where the child threads don't quit after > finishing the transactions? > > There is a program called Apache Top, sort of like regular UNIX Top... > but maybe it would help monitor this stuff? Or is that you are using? > > Are these threads initiating processes which interact with a database at > all? Could the database connections not be closing cleanly, leaving > Apache thinking it is still reading/writing/connected? > > What server side scripting are you using? PHP, Perl, Python, Ruby? > > What do the logs say? > > -Nick > > -----Original Message----- > From: Vermont Area Group of Unix Enthusiasts [mailto:[EMAIL PROTECTED] > On Behalf Of Phil Marshall > Sent: Thursday, May 08, 2008 1:23 PM > To: [email protected] > Subject: Apache Problem > > Here is a sample from the apache status output. We are getting > flooded with these. Apache will get filled up to 255 connections with > these things. This has happened before, but usually it goes away > quickly. Today it went on for about 45 minutes. Its fine now. Does > anyone have any idea what this is or how to block it? > > 51-9 28349 0/1/1 R 0.00 28 3 0.0 0.02 > 0.02 ? ? ..reading.. > 52-9 28350 0/1/1 R 0.00 28 0 0.0 0.000 > 0.000 ? ? ..reading.. > 53-9 28351 0/2/2 R 0.00 28 0 0.0 0.000 > 0.000 ? ? ..reading.. > 54-9 28352 0/1/1 R 0.00 28 0 0.0 0.000 > 0.000 ? ? ..reading.. > 55-9 28354 0/1/1 R 0.00 28 78 0.0 0.00 > 0.00 ? ? ..reading.. > 56-9 28355 0/1/1 R 0.00 28 0 0.0 0.000 > 0.000 ? ? ..reading.. > > --------------------- > Phil Marshall > Signal Advertising > 535 Stone Cutters Way > Montpelier, VT 05602 > 802-229-4149 > [EMAIL PROTECTED] -- Rubin Bennett RB Technologies http://thatitguy.com [EMAIL PROTECTED] (802)223-4448 "They that can give up essential liberty to obtain a little temporary security deserve neither liberty nor safety" --Benjamin Franklin, Historical Review of Pennsylvania, 1759
