Bill, If you're looking to throttle access to a particular URI (or set of URIs), give mod_throttle_access a look. It is available via the Apache Module Registry and at http://www.fremen.org/apache/mod_throttle_access.html .
Regards, Christian ----------------- Christian Gilmore Technology Leader GeT WW Global Applications Development IBM Software Group -----Original Message----- From: Bill Moseley [mailto:[EMAIL PROTECTED]] Sent: Friday, April 19, 2002 12:56 AM To: [EMAIL PROTECTED] Subject: Throttling, once again Hi, Wasn't there just a thread on throttling a few weeks ago? I had a machine hit hard yesterday with a spider that ignored robots.txt. Load average was over 90 on a dual CPU Enterprise 3500 running Solaris 2.6. It's a mod_perl server, but has a few CGI scripts that it handles, and the spider was hitting one of the CGI scripts over and over. They were valid requests, but coming in faster than they were going out. Under normal usage the CGI scripts are only accessed a few times a day, so it's not much of a problem have them served by mod_perl. And under normal peak loads RAM is not a problem. The machine also has bandwidth limitation (packet shaper is used to share the bandwidth). That combined with the spider didn't help things. Luckily there's 4GB so even at a load average of 90 it wasn't really swapping much. (Well not when I caught it, anyway). This spider was using the same IP for all requests. Anyway, I remember Randal's Stonehenge::Throttle discussed not too long ago. That seems to address this kind of problem. Is there anything else to look into? Since the front-end is mod_perl, it mean I can use mod_perl throttling solution, too, which is cool. I realize there's some fundamental hardware issues to solve, but if I can just keep the spiders from flooding the machine then the machine is getting by ok. Also, does anyone have suggestions for testing once throttling is in place? I don't want to start cutting off the good customers, but I do want to get an idea how it acts under load. ab to the rescue, I suppose. Thanks much, -- Bill Moseley mailto:[EMAIL PROTECTED]
