[EMAIL PROTECTED] wrote:
If you really don't want to be crawled, you can alter the robots.txt
file (http://www.robotstxt.org/) for your web server, of do a
robots="nofollow" (http://www.robotstxt.org/meta.html). You could also
use a sitemap spec (http://sitemaps.org/) to control how often you get
crawled.
-- Cole
It's a tad off topic, but for the record, some bots and spiders will
ignore robots.txt directives, with bad ones doing the deliberate
opposite of what you're asking them to do. So be sure you're not using
robots.txt as a privacy / security measure.
_______________________________________________
UPHPU mailing list
[email protected]
http://uphpu.org/mailman/listinfo/uphpu
IRC: #uphpu on irc.freenode.net
