>>I think his point was that good bots such as google will obey his do not crawl command.
Ok, but my point is WHY give a do not crawl command to good bots such as Google? Don't you want you site to be indexed? >> He is trying to annoy the scumbags who crawl websites to steal email address so they can spam people, I already have implemented some protection about this: the address coded in the mailto: is encrypted, and an onClick function decrypts it when a human clicks on it. >> these jerks ignore the robots file and love to follow do not follow links. I also use the revisit_after meta tag, to reduce the number of times pages are visited when I know they wont be modified often ( from 1 to 60 days) What I intent to do also, especially for those which do not obey the meta tag (Have already started) is: - update a table with all different user_agents encountered, - set some "keepOut" flag by hand for the ones I do not want, - CFABORT all pages in the Application.cfm for all undesirable. -- _______________________________________ REUSE CODE! Use custom tags; See http://www.contentbox.com/claude/customtags/tagstore.cfm (Please send any spam to this address: [EMAIL PROTECTED]) Thanks. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Logware (www.logware.us): a new and convenient web-based time tracking application. Start tracking and documenting hours spent on a project or with a client with Logware today. Try it for free with a 15 day trial account. http://www.houseoffusion.com/banners/view.cfm?bannerid=67 Message: http://www.houseoffusion.com/lists.cfm/link=i:4:222719 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4 Donations & Support: http://www.houseoffusion.com/tiny.cfm/54
