Author: vds
Email: 
Message:
In new versions indexer often handle robots.txt file in strange manner, like this:
indexer[17870]: [1] URL: http://www.someaddr.ru/robots.txt
indexer[17870]: [1] URL: http://www.someaddr.ru/
indexer[17870]: [1] URL: http://www.someaddr.ru/robots.txt
indexer[17870]: [1] URL: http://www.someaddr.ru/news/index.shtml
indexer[17870]: [1] URL: http://www.someaddr.ru/robots.txt
indexer[17870]: [1] URL: http://www.someaddr.ru/menu.html
indexer[17870]: [1] URL: http://www.someaddr.ru/robots.txt
indexer[17870]: [1] URL: http://www.someaddr.ru/top.html
indexer[17870]: [1] URL: http://www.someaddr.ru/robots.txt
indexer[17870]: [1] URL: http://www.someaddr.ru/topleft.html
indexer[17870]: [1] URL: http://www.someaddr.ru/robots.txt
indexer[17870]: [1] URL: http://www.someaddr.ru/members/index.shtml
indexer[17870]: [1] URL: http://www.someaddr.ru/robots.txt

that does it mean?


Reply: <http://www.mnogosearch.org/board/message.php?id=3860>

___________________________________________
If you want to unsubscribe send "unsubscribe general"
to [EMAIL PROTECTED]

Reply via email to