>>is there a good bot/bad bot list? Not as I know, anyway, one cannot rely on user agents which can be faked so easily. Personally, I let just a few known bots in, based on the IP address, the only parameter that cannot be faked. For every other request, I have some tools that analyze automatically every visitor according to some criteria as: - Does it read robots.txt? - fails in some robot trap? - reads robots.txt but reads forbidden pages any way; - requests pages at too high rate. - reads javascripts but does not execute it. - does not read CSS, - clearly idenfies itself in the user agent or not. etc... .... and of course, presence of DECLARE or http in urls is the first test ;-)
>>I have an IP- and bot-identifying based system that works pretty well but I'm always up for newer and better Such a system can only identify good bots for sure, but not bad bots and fakes. And the problem is not with good bots, but with bad guys. I also have a white list and a black list, but their only purpose is to bypass the rest of the tests. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;207172674;29440083;f Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:313828 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
