Hello David,
From: "David Ringsmuth" <[email protected]>
Is there a recommend way to prevent ROBOTS from crawling an Active4D (v
3+) Site?
I've googled this and found that HTML may help:
<META NAME="ROBOTS" CONTENT="NOINDEX">
<META NAME="ROBOTS" CONTENT="NOFOLLOW">
There is also a robots.txt document that may help.
Should the robots.txt document be placed in the web_decoy directory?
My understanding is that robots.txt should be in the root directory of your
domain's website (i.e., the directory that www.yourdomain.com takes you to).
Also note that these measures only work for "good" 'bots that are designed
to obey them, such as Google's site indexing 'bots. Sloppily written ones
may not follow your instructions. Malicious 'bots, such as email harvesters
for spamming purposes, definitely will not be so polite.
-Steve Makohin
Oakville, Ontario, Canada
_______________________________________________
Active4D-dev mailing list
[email protected]
http://mailman.aparajitaworld.com/mailman/listinfo/active4d-dev
Archives: http://mailman.aparajitaworld.com/archive/active4d-dev/
