Webboard: robots.txt problem

2001-04-19 Thread Alexander Barkov
Author: Alexander Barkov Email: [EMAIL PROTECTED] Message: It seems that robots.txt still hasn't been indexed. Run indexer -amu http://servername/robots.txt then run indexer in usual manner. > I created a robots.txt file and placed it in the root of my web site. The contents >of the rob

Webboard: robots.txt problem

2001-04-19 Thread Chris Mangano
Author: Chris Mangano Email: [EMAIL PROTECTED] Message: I created a robots.txt file and placed it in the root of my web site. The contents of the robots.txt are as follows: User-agent: * Disallow: / This should keep all search engines from indexing my site. When I run mnoGoSearch, it still i