We've got a web site where we try to meticulously track page views and referers (without using the web logs). What we run into is that many of the page views are being logged by search engine robots. I _think_ this can be avoided using robots.txt, but I'm not certain.
Say I've got a page on a web site (call it "A") and it has a link to the page I'm logging on web site "B", and web site B has a robot.txt file telling the bots to not index the page on B, will the search engine still follow the link and request the page on B? What I'm interested in isn't so much that the page on B not get indexed, but that the search engine bot never request the page in the first place. Any ideas how this works? Thanks, Jim ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Archives: http://www.houseoffusion.com/cf_lists/index.cfm?forumid=4 Subscription: http://www.houseoffusion.com/cf_lists/index.cfm?method=subscribe&forumid=4 FAQ: http://www.thenetprofits.co.uk/coldfusion/faq Get the mailserver that powers this list at http://www.coolfusion.com Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
