> -----Original Message-----
> From: Meli Helmut [mailto:[EMAIL PROTECTED]
> Sent: Wednesday, August 10, 2005 2:03 PM
> To: CF-Talk
> Subject: Re: Fetching a website?
> 
> I just wanted to write an application like this on my one. This will give
> me a good idea how to parse a website etc. There are some Spider written
> in PHP so I thought to write something like this in ColdFusion.

Dave is right that CF isn't even close to the best tool for the job... but
for what it's worth this ColdFusion (CF MX 7) is the best ColdFusion ever
for this job.  ;^)

I think it's a great exercise to try - you'll learn a lot and will end up
with a nice tool that could solve a lot of problems even if it's not
suitable for a full-fledged web scraper.

(It's also something that you might consider open-sourcing for the
community... hint, hint. ;^)  )

As for performance I would highly recommend checking out the Asynch gateway
- using this in CF MX 7 you can launch multiple threads at your target to
collect all those little assets (images, scripts, styles, etc) instead of
waiting for each to come in serial.

That alone would make you app several orders of magnitude faster.

Jim Davis





~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
Logware (www.logware.us): a new and convenient web-based time tracking 
application. Start tracking and documenting hours spent on a project or with a 
client with Logware today. Try it for free with a 15 day trial account.
http://www.houseoffusion.com/banners/view.cfm?bannerid=67

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214429
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
Donations & Support: http://www.houseoffusion.com/tiny.cfm/54

Reply via email to