According to Randal L. Schwartz:
> >>>>> "Leslie" == Leslie Mikesell <[EMAIL PROTECTED]> writes:
>
> Leslie> How about an option to redirect to a different machine instead? I've
> Leslie> considered digging out an old, slow 386 to handle greedy clients
> Leslie> without obviously denying service to them.
>
> Most evil spiders I've see don't really pay attention to a redirect,
> in any way other than "this page wasn't ready". In fact, that
> redirect probably would put the scope of interest outside the spiders
> suck zone.
That would serve the purpose just as well. Most of my spiders are
actually programs written to harvest our commodities trading data
and since we don't have any stated policy to prohibit that, I don't
really want to refuse the requests or fail but I do want to limit
the impact on other activity on the server. I suspect they would
adapt to follow but it wouldn't matter to me either way. So far
most of this activity has been after the exchange closings, so it
hasn't had a serious impact on our mid-day peak usage but it may
reach a point where I have to try something. Are you sure that
keeping track of the client addresses doesn't turn it into an
overall loss compared to just completing the requests?
Les Mikesell
[EMAIL PROTECTED]
