On Tue, 6 Mar 2007, Richard Crawford wrote:
Though I'm still seeing a lot of hits.
They will still come (to see the robots file and leave hopefully) but it did make a big difference on my machine. Now Im trying to tweak it so that they can see the main pages but not further.
It looks like I have to list every quirky subdir that every domain has on it with a disallow. I havent found an example for something like "allow index.html"
Gandalf Parker _______________________________________________ vox-tech mailing list vox-tech@lists.lugod.org http://lists.lugod.org/mailman/listinfo/vox-tech
