> Anyone have a take on this? An intelligent solution? Should i
> be filtering the download process to exclude the search engines?

You should use robots.txt to specify which URLs to exclude.

Dave Watts, CTO, Fig Leaf Software
http://www.figleaf.com/
phone: 202-797-5496
fax: 202-797-5444
[Todays Threads] [This Message] [Subscription] [Fast Unsubscribe] [User Settings] [Donations and Support]

Reply via email to