Friday, September 27, 2002, 1:34:21 AM, Graham Chiu wrote : > On Thu, 26 Sep 2002 23:21:54 +0200 > Christian Langreiter <[EMAIL PROTECTED]> wrote:
>>> Hmm. I thought that there was not supposed to be a >>>direct >>> link anywhere to that archive to prevent spiders getting >>> to it ... as email addresses are visible on those pages. >> >>Well, spiders don't get to sites if they observe >>robots.txt, which >>spam address harvesters most certainly don't. > We had a discussion before about the rebol.org archive, > and Jeff removed all links to the archive. To get the > address, you have to use the link on the RT rebsite. So > even a spam address harvesting robot should not get there. > But I guess google had it in it's database prior to that > happening. It is also possible that people added a link to this page within their own website. This way google will access it while harvesting those people's sites. It's hard to keep a useful page from being referenced from somewhere else ;) Regards, Laurent -- Laurent Giroud mailto:[EMAIL PROTECTED] -- To unsubscribe from this list, please send an email to [EMAIL PROTECTED] with "unsubscribe" in the subject, without the quotes.
