> Anyone have a take on this? An intelligent solution? Should i
> be filtering the download process to exclude the search engines?
You should use robots.txt to specify which URLs to exclude.
Dave Watts, CTO, Fig Leaf Software
http://www.figleaf.com/
phone: 202-797-5496
fax: 202-797-5444
[Todays Threads]
[This Message]
[Subscription]
[Fast Unsubscribe]
[User Settings]
[Donations and Support]