It's trivial to fake this header and many bad bots (i.e. the ones that ignore robots.txt) will pretend to be IE or another browser.
Claude S has posted his solutions in the past and that should all be in the archives. mxAjax / CFAjax docs and other useful articles: http://www.bifrost.com.au/blog/ 2009/2/20 Les Mizzell <lesm...@bellsouth.net>: > > In working out my Googlebot problem, I came across an idea of using > HTTP_USER_AGENT to identify a bot, and then exclude it from an area or > whatever (redirect it to www.disney.com or something... that's a joke) > ... assuming it gets in despite the robots.txt file. > > So, this looks like a good starting point to modify from: > > http://www.bennadel.com/blog/1083-ColdFusion-Session-Management-And-Spiders-Bots.htm > > I see how to deal with bots you can identify. Can anybody think of a way > or have an example of how to figure out that it's a unidentifiable bot > (rather than a real user with a browser) and redirect/whatever? > > Just asking.... > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;207172674;29440083;f Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:319585 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
