> -----Original Message----- > From: Jillian Carroll [mailto:[EMAIL PROTECTED]] > Sent: Wednesday, February 05, 2003 10:18 AM > To: CF-Talk > Subject: Remotely spider websites > > > >From what I have been able to find, the verity spider will > only spider > sites on a single network domain... so a remote machine must > reside on the same domain. What I'm wondering is if anybody
Yup. Verity (and MS Index Server) are what's called "Worms" - they "burrow" through the file system of the machine itself. "Spiders", on the other hand, "walk the web" and see only what's public through HTTP. There are pros and cons to both, of course, but that's the main difference. > can recommend (or has > written) a tool that will let me spider remote sites in a > CF-friendly manner? There are a ton, but I don't know how well supported they are (or if they even exist any longer). Most of the major search engines have a "home game" version of their software (either as a software package or as a service they offer): Alta Vista: http://solutions.altavista.com/ Google: http://www.google.com/services/ Lycos: http://enterprise.lycos.com/Search/SiteSearch.asp You can dig up a lot more of these types of services by visiting http://www.searchengines.com/ Unfortunately there's no specific content directed towards DIY, but they do have the largest collection of public engines around. Now the site to go for more general information is http://www.searchenginewatch.com/ This is probably the most relevant page for you: http://www.searchenginewatch.com/resources/software.html But there's a lot more material on the site aimed at webmasters. Hope this helps, Jim Davis President, Depressed Press of Boston: http://www.DepressedPress.com/ Webmaster, First Night Boston: http://www.firstnight.org/ Senior Consultant, Metlife eCommerce IT: http://www.metlife.com/ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Archives: http://www.houseoffusion.com/cf_lists/index.cfm?forumid=4 Subscription: http://www.houseoffusion.com/cf_lists/index.cfm?method=subscribe&forumid=4 FAQ: http://www.thenetprofits.co.uk/coldfusion/faq Get the mailserver that powers this list at http://www.coolfusion.com Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
