On Mon, Oct 18, 2004 at 09:48:35AM -0400, /0 wrote: > do a google search for "robots.txt" > > thats the way to avoid the bots.
I'm afraid it's not; there's no way to prevent a bot from ignoring 'robots.txt'. Cheers, Michel
On Mon, Oct 18, 2004 at 09:48:35AM -0400, /0 wrote: > do a google search for "robots.txt" > > thats the way to avoid the bots.
I'm afraid it's not; there's no way to prevent a bot from ignoring 'robots.txt'. Cheers, Michel