- - - - - - - - - - - - - - - - - - - - - - - - - - - - Name: Gabi Subject: How can I avoid bombarding the servers?
I hoped this wouldn't happen, but a few of the servers I've been indexing started rejecting robots. I don't want the same thing to happen to the other servers (and I don't want to make them crash or slow down for handling too many requests either). Is there any way to tell indexer to wait a certain amount of time before sending another request to the same server, and/or to rotate the order in which servers are indexed instead of making all the requests to one server before moving on to the next? - - - - - - - - - - - - - - - - - - - - - - - - - - - - Read the full topic here: http://www.dataparksearch.org/cgi-bin/simpleforum.cgi?fid=02;post=
