RE: Fetching a website?

2005-08-10 Thread Damien McKenna
 I would like to know how to grab a entire website, download 
 and save it to the harddisk. Replace all absolute paths into 
 relative paths so that the site can be viewed offline.

Cfexecute
wget

-- 
Damien McKenna - Web Developer - [EMAIL PROTECTED]
The Limu Company - http://www.thelimucompany.com/ - 407-804-1014
#include stdjoke.h


~|
Find out how CFTicket can increase your company's customer support 
efficiency by 100%
http://www.houseoffusion.com/banners/view.cfm?bannerid=49

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214374
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


Re: Fetching a website?

2005-08-10 Thread Thomas Chiverton
On Wednesday 10 August 2005 16:58, housi mueller wrote:
 I would like to know how to grab a entire website, download and save it to
 the harddisk. Replace all absolute paths into relative paths so that the
 site can be viewed offline.

cfexecute wget with the '-m' mirror option... ?

-- 

Tom Chiverton 
Advanced ColdFusion Programmer

~|
Find out how CFTicket can increase your company's customer support 
efficiency by 100%
http://www.houseoffusion.com/banners/view.cfm?bannerid=49

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214376
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


RE: Fetching a website?

2005-08-10 Thread Dawson, Michael
Start reading up on CFHTTP and Regular Expressions.  CFHTTP will convert
all links to fully-qualified links (if you set the appropriate
attribute) and the regular expressions will find all links within the
document.

M!ke 

-Original Message-
From: housi mueller [mailto:[EMAIL PROTECTED] 
Sent: Wednesday, August 10, 2005 10:58 AM
To: CF-Talk
Subject: Fetching a website?

I would like to know how to grab a entire website, download and save it
to the harddisk. Replace all absolute paths into relative paths so that
the site can be viewed offline.

I would like to write something similar like the program Httrack but in
ColdFusion. So if someone knows of tutorials, scripts etc.

Any help would be appreciated 

Thank you in advance...
 
Helmut
 

~|
Find out how CFTicket can increase your company's customer support 
efficiency by 100%
http://www.houseoffusion.com/banners/view.cfm?bannerid=49

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214378
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


Re: Fetching a website?

2005-08-10 Thread Meli Helmut
 I would like to know how to grab a entire website, download 
 and save it to the harddisk. Replace all absolute paths into 
 relative paths so that the site can be viewed offline.

Cfexecute
wget

-- 
Damien McKenna - Web Developer - [EMAIL PROTECTED]
The Limu Company - http://www.thelimucompany.com/ - 407-804-1014
#include stdjoke.h

I saw wget already. I know why to invent the wheel again, if it already 
exists... but I would like to write something like this on my one...

~|
Logware (www.logware.us): a new and convenient web-based time tracking 
application. Start tracking and documenting hours spent on a project or with a 
client with Logware today. Try it for free with a 15 day trial account.
http://www.houseoffusion.com/banners/view.cfm?bannerid=67

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214386
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


RE: Fetching a website?

2005-08-10 Thread Dave Watts
 I saw wget already. I know why to invent the wheel again, if 
 it already exists... but I would like to write something like 
 this on my one...

I submit that CF is a spectacularly poor choice for doing this, though.

Dave Watts, CTO, Fig Leaf Software
http://www.figleaf.com/

Fig Leaf Software provides the highest caliber vendor-authorized 
instruction at our training centers in Washington DC, Atlanta, 
Chicago, Baltimore, Northern Virginia, or on-site at your location. 
Visit http://training.figleaf.com/ for more information!


~|
Logware (www.logware.us): a new and convenient web-based time tracking 
application. Start tracking and documenting hours spent on a project or with a 
client with Logware today. Try it for free with a 15 day trial account.
http://www.houseoffusion.com/banners/view.cfm?bannerid=67

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214389
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


Re: Fetching a website?

2005-08-10 Thread Jochem van Dieten
Meli Helmut wrote:
 
 I saw wget already. I know why to invent the wheel again, if it already 
 exists... but I would like to write something like this on my one...

Read the source and you know what it takes :) But I would 
recommend cURL over wget because the license is less restrictive.

Jochem

~|
Discover CFTicket - The leading ColdFusion Help Desk and Trouble 
Ticket application

http://www.houseoffusion.com/banners/view.cfm?bannerid=48

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214390
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


RE: Fetching a website?

2005-08-10 Thread housi mueller
 I submit that CF is a spectacularly poor choice for doing this, though.

Maybe CF is a poor choice but it should be possible...

__
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

~|
Logware (www.logware.us): a new and convenient web-based time tracking 
application. Start tracking and documenting hours spent on a project or with a 
client with Logware today. Try it for free with a 15 day trial account.
http://www.houseoffusion.com/banners/view.cfm?bannerid=67

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214392
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


Re: Fetching a website?

2005-08-10 Thread Meli Helmut
Start reading up on CFHTTP and Regular Expressions.  CFHTTP will convert
all links to fully-qualified links (if you set the appropriate
attribute) and the regular expressions will find all links within the
document.

M!ke 


I already made some tests with CFHTTP. But it needs more. Looping trough and 
parsing all the documents, download all the objects etc etc.
I thought that maybe someone wrote already something similar in ColdFusion ...

~|
Find out how CFTicket can increase your company's customer support 
efficiency by 100%
http://www.houseoffusion.com/banners/view.cfm?bannerid=49

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214398
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


RE: Fetching a website?

2005-08-10 Thread Dave Watts
 Maybe CF is a poor choice but it should be possible...

It is certainly possible. CF gives you all of the functionality you need -
you simply need to write the code to use it. However, even if you do so, it
will probably not perform suitably, since that's not the kind of thing that
CF is designed to do well.

Dave Watts, CTO, Fig Leaf Software
http://www.figleaf.com/

Fig Leaf Software provides the highest caliber vendor-authorized 
instruction at our training centers in Washington DC, Atlanta, 
Chicago, Baltimore, Northern Virginia, or on-site at your location. 
Visit http://training.figleaf.com/ for more information!


~|
Logware (www.logware.us): a new and convenient web-based time tracking 
application. Start tracking and documenting hours spent on a project or with a 
client with Logware today. Try it for free with a 15 day trial account.
http://www.houseoffusion.com/banners/view.cfm?bannerid=67

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214401
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


Re: Fetching a website?

2005-08-10 Thread Meli Helmut
 Maybe CF is a poor choice but it should be possible...

It is certainly possible. CF gives you all of the functionality you need -
you simply need to write the code to use it. However, even if you do so, it
will probably not perform suitably, since that's not the kind of thing that
CF is designed to do well.

Maybe it sounds strange but I will try to write the code in ColdFusion. I 
thought only  that someone already has done or have seen a Code like this and 
could give me some tips, recommendations.. etc.



~|
Logware (www.logware.us): a new and convenient web-based time tracking 
application. Start tracking and documenting hours spent on a project or with a 
client with Logware today. Try it for free with a 15 day trial account.
http://www.houseoffusion.com/banners/view.cfm?bannerid=67

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214414
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


RE: Fetching a website?

2005-08-10 Thread Connie DeCinko
Actually, HTTrack works very well.  I've used it a couple times.  Any reason
you don't just use it?
 

-Original Message-
From: housi mueller [mailto:[EMAIL PROTECTED] 
Sent: Wednesday, August 10, 2005 8:58 AM
To: CF-Talk
Subject: Fetching a website?

I would like to know how to grab a entire website, download and save it to
the harddisk. Replace all absolute paths into relative paths so that the
site can be viewed offline.

I would like to write something similar like the program Httrack but in
ColdFusion. So if someone knows of tutorials, scripts etc.

Any help would be appreciated 

Thank you in advance...
 
Helmut
 




~|
Discover CFTicket - The leading ColdFusion Help Desk and Trouble 
Ticket application

http://www.houseoffusion.com/banners/view.cfm?bannerid=48

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214419
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


RE: Fetching a website?

2005-08-10 Thread Connie DeCinko
Yeah, I agree.  Seems like the wrong tool for the job.  Sure you can bang in
nails with a screw driver, but should you?
 

-Original Message-
From: Dave Watts [mailto:[EMAIL PROTECTED] 
Sent: Wednesday, August 10, 2005 9:31 AM
To: CF-Talk
Subject: RE: Fetching a website?

 I saw wget already. I know why to invent the wheel again, if it 
 already exists... but I would like to write something like this on my 
 one...

I submit that CF is a spectacularly poor choice for doing this, though.

Dave Watts, CTO, Fig Leaf Software
http://www.figleaf.com/




~|
Logware (www.logware.us): a new and convenient web-based time tracking 
application. Start tracking and documenting hours spent on a project or with a 
client with Logware today. Try it for free with a 15 day trial account.
http://www.houseoffusion.com/banners/view.cfm?bannerid=67

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214421
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


Re: Fetching a website?

2005-08-10 Thread Meli Helmut
Actually, HTTrack works very well.  I've used it a couple times.  Any reason
you don't just use it?
 

I would like to know how to grab a entire website, download and save it to
the harddisk. Replace all absolute paths into relative paths so that the
site can be viewed offline.

I would like to write something similar like the program Httrack but in
ColdFusion. So if someone knows of tutorials, scripts etc.

Any help would be appreciated 

Thank you in advance...


I just wanted to write an application like this on my one. This will give me a 
good idea how to parse a website etc. There are some Spider written in PHP so I 
thought to write something like this in ColdFusion.


Helmut

~|
Logware (www.logware.us): a new and convenient web-based time tracking 
application. Start tracking and documenting hours spent on a project or with a 
client with Logware today. Try it for free with a 15 day trial account.
http://www.houseoffusion.com/banners/view.cfm?bannerid=67

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214422
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


Re: Fetching a website?

2005-08-10 Thread Jochem van Dieten
Meli Helmut wrote:
 
 I just wanted to write an application like this on my one. This will give me 
 a good idea how to parse a website etc. There are some Spider written in PHP 
 so I thought to write something like this in ColdFusion.

Then maybe it is a good idea to take a look at the source of 
those PHP spiders.

Jochem

~|
Find out how CFTicket can increase your company's customer support 
efficiency by 100%
http://www.houseoffusion.com/banners/view.cfm?bannerid=49

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214425
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


RE: Fetching a website?

2005-08-10 Thread Ian Skinner
As many have said, it can be done.  But as many have also said, it's not 
something CF is going to be good at, so I doubt you are going to find anything 
already done in it.

But if you want to do this; the major pieces you are going to need are cfhttp 
which will give you a page.  Refind() will allow you to parse the returnd page 
to look for links of all types.  I'm not sure what one does about fetching 
binary files such as images, but my first guess would be the base64 functions 
and cffile.  You'll also be wanting the ability to fire off new requests to get 
linked pages found with the refind() functions.  MX7 enterprise, if you have 
access to it, would be a big help with its asynchrous gateway functionality.

Have Fun.


--
Ian Skinner
Web Programmer
BloodSource
www.BloodSource.org
Sacramento, CA
 
C code. C code run. Run code run. Please!
- Cynthia Dunning

Confidentiality Notice:  This message including any
attachments is for the sole use of the intended
recipient(s) and may contain confidential and privileged
information. Any unauthorized review, use, disclosure or
distribution is prohibited. If you are not the
intended recipient, please contact the sender and
delete any copies of this message. 




~|
Find out how CFTicket can increase your company's customer support 
efficiency by 100%
http://www.houseoffusion.com/banners/view.cfm?bannerid=49

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214426
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


RE: Fetching a website?

2005-08-10 Thread Connie DeCinko
I see, you just love a challenge!  Hey, get it to work and submit it for a
CFDJ article.
 

-Original Message-
From: Meli Helmut [mailto:[EMAIL PROTECTED] 
Sent: Wednesday, August 10, 2005 11:03 AM
To: CF-Talk
Subject: Re: Fetching a website?

I just wanted to write an application like this on my one. This will give me
a good idea how to parse a website etc. There are some Spider written in PHP
so I thought to write something like this in ColdFusion.





~|
Logware (www.logware.us): a new and convenient web-based time tracking 
application. Start tracking and documenting hours spent on a project or with a 
client with Logware today. Try it for free with a 15 day trial account.
http://www.houseoffusion.com/banners/view.cfm?bannerid=67

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214427
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


Re: Fetching a website?

2005-08-10 Thread Meli Helmut
 I saw wget already. I know why to invent the wheel again, if 
 it already exists... but I would like to write something like 
 this on my one...

I submit that CF is a spectacularly poor choice for doing this, though.

Could you also tell me please why CF is a poor choice to do this and which 
Script Language would you recommend??

~|
Logware (www.logware.us): a new and convenient web-based time tracking 
application. Start tracking and documenting hours spent on a project or with a 
client with Logware today. Try it for free with a 15 day trial account.
http://www.houseoffusion.com/banners/view.cfm?bannerid=67

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214428
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


RE: Fetching a website?

2005-08-10 Thread Jim Davis
 -Original Message-
 From: Meli Helmut [mailto:[EMAIL PROTECTED]
 Sent: Wednesday, August 10, 2005 2:03 PM
 To: CF-Talk
 Subject: Re: Fetching a website?
 
 I just wanted to write an application like this on my one. This will give
 me a good idea how to parse a website etc. There are some Spider written
 in PHP so I thought to write something like this in ColdFusion.

Dave is right that CF isn't even close to the best tool for the job... but
for what it's worth this ColdFusion (CF MX 7) is the best ColdFusion ever
for this job.  ;^)

I think it's a great exercise to try - you'll learn a lot and will end up
with a nice tool that could solve a lot of problems even if it's not
suitable for a full-fledged web scraper.

(It's also something that you might consider open-sourcing for the
community... hint, hint. ;^)  )

As for performance I would highly recommend checking out the Asynch gateway
- using this in CF MX 7 you can launch multiple threads at your target to
collect all those little assets (images, scripts, styles, etc) instead of
waiting for each to come in serial.

That alone would make you app several orders of magnitude faster.

Jim Davis





~|
Logware (www.logware.us): a new and convenient web-based time tracking 
application. Start tracking and documenting hours spent on a project or with a 
client with Logware today. Try it for free with a 15 day trial account.
http://www.houseoffusion.com/banners/view.cfm?bannerid=67

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214429
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


Re: Fetching a website?

2005-08-10 Thread Mike Chabot
I have used WebCopier to do what you are asking for. I think it is a great 
shareware program.
 -Mike Chabot
 On 8/10/05, housi mueller [EMAIL PROTECTED] wrote: 
 
 I would like to know how to grab a entire website, download and save it to 
 the harddisk. Replace all absolute paths into relative paths so that the 
 site can be viewed offline.
 
 I would like to write something similar like the program Httrack but in 
 ColdFusion. So if someone knows of tutorials, scripts etc.
 
 Any help would be appreciated
 
 Thank you in advance...
 
 Helmut


~|
Discover CFTicket - The leading ColdFusion Help Desk and Trouble 
Ticket application

http://www.houseoffusion.com/banners/view.cfm?bannerid=48

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214432
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


Re: Fetching a website?

2005-08-10 Thread Jerry Johnson
Can't you just us the cfhttp file and path attributes to save the
binary/included files directly to disk?

On 8/10/05, Ian Skinner [EMAIL PROTECTED] wrote:
 As many have said, it can be done.  But as many have also said, it's not 
 something CF is going to be good at, so I doubt you are going to find 
 anything already done in it.
 
 But if you want to do this; the major pieces you are going to need are 
 cfhttp which will give you a page.  Refind() will allow you to parse the 
 returnd page to look for links of all types.  I'm not sure what one does 
 about fetching binary files such as images, but my first guess would be the 
 base64 functions and cffile.  You'll also be wanting the ability to fire off 
 new requests to get linked pages found with the refind() functions.  MX7 
 enterprise, if you have access to it, would be a big help with its asynchrous 
 gateway functionality.

~|
Discover CFTicket - The leading ColdFusion Help Desk and Trouble 
Ticket application

http://www.houseoffusion.com/banners/view.cfm?bannerid=48

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214433
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


RE: Fetching a website?

2005-08-10 Thread Ian Skinner
Can't you just us the cfhttp file and path attributes to save the 
binary/included files directly to disk?


I don't know.  I have never, ever tried to get a binary file from another 
system using http.  My gut says it's probably possible; my head has no idea 
how.  It would be interesting to know, if anybody has time to experiment (or 
has already done so).


--
Ian Skinner
Web Programmer
BloodSource
www.BloodSource.org
Sacramento, CA
 
C code. C code run. Run code run. Please!
- Cynthia Dunning

Confidentiality Notice:  This message including any
attachments is for the sole use of the intended
recipient(s) and may contain confidential and privileged
information. Any unauthorized review, use, disclosure or
distribution is prohibited. If you are not the
intended recipient, please contact the sender and
delete any copies of this message. 




~|
Logware (www.logware.us): a new and convenient web-based time tracking 
application. Start tracking and documenting hours spent on a project or with a 
client with Logware today. Try it for free with a 15 day trial account.
http://www.houseoffusion.com/banners/view.cfm?bannerid=67

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214439
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


Re: Fetching a website?

2005-08-10 Thread Meli Helmut
Dave is right that CF isn't even close to the best tool for the job... but
for what it's worth this ColdFusion (CF MX 7) is the best ColdFusion ever
for this job.  ;^)

I think it's a great exercise to try - you'll learn a lot and will end up
with a nice tool that could solve a lot of problems even if it's not
suitable for a full-fledged web scraper.

(It's also something that you might consider open-sourcing for the
community... hint, hint. ;^)  )

As for performance I would highly recommend checking out the Asynch gateway
- using this in CF MX 7 you can launch multiple threads at your target to
collect all those little assets (images, scripts, styles, etc) instead of
waiting for each to come in serial.

That alone would make you app several orders of magnitude faster.

Jim Davis

Thank you for support me. I will consider your tips and even to open-source the 
project.. =:0)

~|
Logware (www.logware.us): a new and convenient web-based time tracking 
application. Start tracking and documenting hours spent on a project or with a 
client with Logware today. Try it for free with a 15 day trial account.
http://www.houseoffusion.com/banners/view.cfm?bannerid=67

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214438
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


RE: Fetching a website?

2005-08-10 Thread Connie DeCinko
That's one thing the stand alone apps can do, is allow you to set how many
threads you want to use to pull down the site, rather than one at a time.
For a large site, it can mean minutes vs. hours.
 

-Original Message-
From: Jim Davis [mailto:[EMAIL PROTECTED] 
Sent: Wednesday, August 10, 2005 11:16 AM
To: CF-Talk
Subject: RE: Fetching a website?

As for performance I would highly recommend checking out the Asynch gateway
- using this in CF MX 7 you can launch multiple threads at your target to
collect all those little assets (images, scripts, styles, etc) instead of
waiting for each to come in serial.

That alone would make you app several orders of magnitude faster.

Jim Davis




~|
Logware (www.logware.us): a new and convenient web-based time tracking 
application. Start tracking and documenting hours spent on a project or with a 
client with Logware today. Try it for free with a 15 day trial account.
http://www.houseoffusion.com/banners/view.cfm?bannerid=67

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214440
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


Re: Fetching a website?

2005-08-10 Thread Meli Helmut
I see, you just love a challenge!  Hey, get it to work and submit it for a
CFDJ article.

When I get it to work I will definitely publish the project...

~|
Find out how CFTicket can increase your company's customer support 
efficiency by 100%
http://www.houseoffusion.com/banners/view.cfm?bannerid=49

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214442
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


RE: Fetching a website?

2005-08-10 Thread Connie DeCinko
That's one thing the stand alone apps can do, is allow you to set how many
threads you want to use to pull down the site, rather than one at a time.
For a large site, it can mean minutes vs. hours.
 

-Original Message-
From: Jim Davis [mailto:[EMAIL PROTECTED] 
Sent: Wednesday, August 10, 2005 11:16 AM
To: CF-Talk
Subject: RE: Fetching a website?

As for performance I would highly recommend checking out the Asynch gateway
- using this in CF MX 7 you can launch multiple threads at your target to
collect all those little assets (images, scripts, styles, etc) instead of
waiting for each to come in serial.

That alone would make you app several orders of magnitude faster.

Jim Davis




~|
Find out how CFTicket can increase your company's customer support 
efficiency by 100%
http://www.houseoffusion.com/banners/view.cfm?bannerid=49

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214443
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


Re: Fetching a website?

2005-08-10 Thread Jerry Johnson
I should have stated that, rather than asking.

You can just use the cfhttp file and path attributes to save a
binary/include file directly to disk.

cfhttp url=http://www.lawyersweeklyusa.com/images/lwlogo.gif;
path=C:\ file=image.gif/cfhttp

Works like a charm. Lovely for dling PDFs from state and federal court sites.

On 8/10/05, Ian Skinner [EMAIL PROTECTED] wrote:
 Can't you just us the cfhttp file and path attributes to save the 
 binary/included files directly to disk?
 
 
 I don't know.  I have never, ever tried to get a binary file from another 
 system using http.  My gut says it's probably possible; my head has no idea 
 how.  It would be interesting to know, if anybody has time to experiment (or 
 has already done so).

~|
Logware (www.logware.us): a new and convenient web-based time tracking 
application. Start tracking and documenting hours spent on a project or with a 
client with Logware today. Try it for free with a 15 day trial account.
http://www.houseoffusion.com/banners/view.cfm?bannerid=67

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214445
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


RE: Fetching a website?

2005-08-10 Thread Ian Skinner
cfhttp url=http://www.lawyersweeklyusa.com/images/lwlogo.gif;
path=C:\ file=image.gif/cfhttp

Works like a charm. Lovely for dling PDFs from state and federal court sites.


Cool, I will file that to the back of my head for the future when I may need to 
do this.  Just one of those things I've never needed to do in my 8 year CF 
career.  


--
Ian Skinner
Web Programmer
BloodSource
www.BloodSource.org
Sacramento, CA
 
C code. C code run. Run code run. Please!
- Cynthia Dunning

Confidentiality Notice:  This message including any
attachments is for the sole use of the intended
recipient(s) and may contain confidential and privileged
information. Any unauthorized review, use, disclosure or
distribution is prohibited. If you are not the
intended recipient, please contact the sender and
delete any copies of this message. 




~|
Logware (www.logware.us): a new and convenient web-based time tracking 
application. Start tracking and documenting hours spent on a project or with a 
client with Logware today. Try it for free with a 15 day trial account.
http://www.houseoffusion.com/banners/view.cfm?bannerid=67

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214446
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


RE: Fetching a website?

2005-08-10 Thread Jim Davis
 -Original Message-
 From: Meli Helmut [mailto:[EMAIL PROTECTED]
 Sent: Wednesday, August 10, 2005 2:16 PM
 To: CF-Talk
 Subject: Re: Fetching a website?
 
  I saw wget already. I know why to invent the wheel again, if
  it already exists... but I would like to write something like
  this on my one...
 
 I submit that CF is a spectacularly poor choice for doing this, though.
 
 Could you also tell me please why CF is a poor choice to do this and which
 Script Language would you recommend??

The problem isn't one of capability, it's one of performance.

The main problem is that plain CF is a serialized, procedural language -
you can only do one thing at a time.  An app like the one you describe, to
be fast, really needs to have good control over threading and process
management (when a single web page might require several dozen HTTP requests
doing them in serial just kills an apps performance).

Using the ASynch gateway you can launch multiple processes at the same time
helps immensely, but each of these requests is a full request in CF - all
of the overhead related to any CF request is there.

A more atomic language (like C or Java) can launch threads in a very
efficient way that allows you to get your specific job done with as little
overhead as possible.

In general you're looking at a well written CF application being orders of
magnitude slower than a well written Java or C application.  (Although, of
course, a great CF app can still beat a crappy Java or C application.)

As for scripting languages... none of them are perfectly suited this kind of
highly concurrent task.  Script languages are optimized for use, not for
performance generally.

But you COULD build it in a lot of languages.  JavaScript or VBScript
(leveraging COM objects in a Windows Scripting Host or HTA Container) could
do it.
 
PERL may be the best choice but you'd really need to add on some of the more
esoteric extensions to get the request performance up.  PERL is very good at
parsing text, but it's not the parsing so much as the HTTP traffic that
worries me.  So PERL may not be significantly faster than CF on such a job.

In the right container (something that wasn't security limited to a host
site for example) pretty much any scripting language with the ability to do
HTTP and string manipulation could do it.  Python and TCL come to mind.
Heck, you COULD do this in Lingo if you wanted.  ;^)

Jim Davis




~|
Logware (www.logware.us): a new and convenient web-based time tracking 
application. Start tracking and documenting hours spent on a project or with a 
client with Logware today. Try it for free with a 15 day trial account.
http://www.houseoffusion.com/banners/view.cfm?bannerid=67

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214448
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


RE: Fetching a website?

2005-08-10 Thread Russ
Why not just write a simple java app that could be perhaps called from cf?
I've been usually switching to java and writing some simple threaded classes
when I ran into these kinds of projects... 

-Original Message-
From: Jim Davis [mailto:[EMAIL PROTECTED] 
Sent: Wednesday, August 10, 2005 3:05 PM
To: CF-Talk
Subject: RE: Fetching a website?

 -Original Message-
 From: Meli Helmut [mailto:[EMAIL PROTECTED]
 Sent: Wednesday, August 10, 2005 2:16 PM
 To: CF-Talk
 Subject: Re: Fetching a website?
 
  I saw wget already. I know why to invent the wheel again, if
  it already exists... but I would like to write something like
  this on my one...
 
 I submit that CF is a spectacularly poor choice for doing this, though.
 
 Could you also tell me please why CF is a poor choice to do this and which
 Script Language would you recommend??

The problem isn't one of capability, it's one of performance.

The main problem is that plain CF is a serialized, procedural language -
you can only do one thing at a time.  An app like the one you describe, to
be fast, really needs to have good control over threading and process
management (when a single web page might require several dozen HTTP requests
doing them in serial just kills an apps performance).

Using the ASynch gateway you can launch multiple processes at the same time
helps immensely, but each of these requests is a full request in CF - all
of the overhead related to any CF request is there.

A more atomic language (like C or Java) can launch threads in a very
efficient way that allows you to get your specific job done with as little
overhead as possible.

In general you're looking at a well written CF application being orders of
magnitude slower than a well written Java or C application.  (Although, of
course, a great CF app can still beat a crappy Java or C application.)

As for scripting languages... none of them are perfectly suited this kind of
highly concurrent task.  Script languages are optimized for use, not for
performance generally.

But you COULD build it in a lot of languages.  JavaScript or VBScript
(leveraging COM objects in a Windows Scripting Host or HTA Container) could
do it.
 
PERL may be the best choice but you'd really need to add on some of the more
esoteric extensions to get the request performance up.  PERL is very good at
parsing text, but it's not the parsing so much as the HTTP traffic that
worries me.  So PERL may not be significantly faster than CF on such a job.

In the right container (something that wasn't security limited to a host
site for example) pretty much any scripting language with the ability to do
HTTP and string manipulation could do it.  Python and TCL come to mind.
Heck, you COULD do this in Lingo if you wanted.  ;^)

Jim Davis






~|
Logware (www.logware.us): a new and convenient web-based time tracking 
application. Start tracking and documenting hours spent on a project or with a 
client with Logware today. Try it for free with a 15 day trial account.
http://www.houseoffusion.com/banners/view.cfm?bannerid=67

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214449
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


RE: Fetching a website?

2005-08-10 Thread housi mueller
Thank you for the detailed explanation. 
Now I learned something new. Right now I am not concerned
about performence so I will go one with CF.
 
Thank you
 
Helmut

Jim Davis [EMAIL PROTECTED] wrote:

  I saw wget already. I know why to invent the wheel again, if
  it already exists... but I would like to write something like
  this on my one...
 
 I submit that CF is a spectacularly poor choice for doing this, though.
 
 Could you also tell me please why CF is a poor choice to do this and which
 Script Language would you recommend??

The problem isn't one of capability, it's one of performance.

The main problem is that plain CF is a serialized, procedural language -
you can only do one thing at a time. An app like the one you describe, to
be fast, really needs to have good control over threading and process
management (when a single web page might require several dozen HTTP requests
doing them in serial just kills an apps performance).

Using the ASynch gateway you can launch multiple processes at the same time
helps immensely, but each of these requests is a full request in CF - all
of the overhead related to any CF request is there.

A more atomic language (like C or Java) can launch threads in a very
efficient way that allows you to get your specific job done with as little
overhead as possible.

In general you're looking at a well written CF application being orders of
magnitude slower than a well written Java or C application. (Although, of
course, a great CF app can still beat a crappy Java or C application.)

As for scripting languages... none of them are perfectly suited this kind of
highly concurrent task. Script languages are optimized for use, not for
performance generally.

But you COULD build it in a lot of languages. JavaScript or VBScript
(leveraging COM objects in a Windows Scripting Host or HTA Container) could
do it.

PERL may be the best choice but you'd really need to add on some of the more
esoteric extensions to get the request performance up. PERL is very good at
parsing text, but it's not the parsing so much as the HTTP traffic that
worries me. So PERL may not be significantly faster than CF on such a job.

In the right container (something that wasn't security limited to a host
site for example) pretty much any scripting language with the ability to do
HTTP and string manipulation could do it. Python and TCL come to mind.
Heck, you COULD do this in Lingo if you wanted. ;^)

Jim Davis


__
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

~|
Discover CFTicket - The leading ColdFusion Help Desk and Trouble 
Ticket application

http://www.houseoffusion.com/banners/view.cfm?bannerid=48

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214451
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


RE: Fetching a website?

2005-08-10 Thread housi mueller
At the moment I do have only basic knowledge about java. It would be not 
sufficient to write threaded classes. But I will work on it =:0)

Russ [EMAIL PROTECTED] wrote:
Why not just write a simple java app that could be perhaps called from cf?
I've been usually switching to java and writing some simple threaded classes
when I ran into these kinds of projects... 


-
 Start your day with Yahoo! - make it your home page 

~|
Logware (www.logware.us): a new and convenient web-based time tracking 
application. Start tracking and documenting hours spent on a project or with a 
client with Logware today. Try it for free with a 15 day trial account.
http://www.houseoffusion.com/banners/view.cfm?bannerid=67

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214455
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


RE: Fetching a website?

2005-08-10 Thread Jim Davis
 -Original Message-
 From: Russ [mailto:[EMAIL PROTECTED]
 Sent: Wednesday, August 10, 2005 3:32 PM
 To: CF-Talk
 Subject: RE: Fetching a website?
 
 Why not just write a simple java app that could be perhaps called from cf?
 I've been usually switching to java and writing some simple threaded
 classes
 when I ran into these kinds of projects...

Well... I think the universal answer to that might be because I don't know
Java.  ;^)

I couldn't do it - I don't know Java (although, dammit, I've really got to
learn).

I still maintain that a well-written CF app can still beat a crappily
written pure Java app.  But if you know both already then it's no question
which should be used here.

Jim Davis





~|
Logware (www.logware.us): a new and convenient web-based time tracking 
application. Start tracking and documenting hours spent on a project or with a 
client with Logware today. Try it for free with a 15 day trial account.
http://www.houseoffusion.com/banners/view.cfm?bannerid=67

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214454
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


Re: Fetching a website?

2005-08-10 Thread Jochem van Dieten
housi mueller wrote:
 At the moment I do have only basic knowledge about java. It would be not 
 sufficient to write threaded classes. But I will work on it =:0)

An unthreaded java class will be magnitudes faster than a 
synchronous CF application because it can maintain the 
connection. CF will have to re-establish the connection on each 
request.

Jochem

~|
Logware (www.logware.us): a new and convenient web-based time tracking 
application. Start tracking and documenting hours spent on a project or with a 
client with Logware today. Try it for free with a 15 day trial account.
http://www.houseoffusion.com/banners/view.cfm?bannerid=67

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214464
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


RE: Fetching a website?

2005-08-10 Thread Dave Watts
 I still maintain that a well-written CF app can still beat a
 crappily written pure Java app.

Absolutely. However, part of writing a well-written application is choosing
the appropriate language for the task.

Dave Watts, CTO, Fig Leaf Software
http://www.figleaf.com/

Fig Leaf Software provides the highest caliber vendor-authorized 
instruction at our training centers in Washington DC, Atlanta, 
Chicago, Baltimore, Northern Virginia, or on-site at your location. 
Visit http://training.figleaf.com/ for more information!


~|
Logware (www.logware.us): a new and convenient web-based time tracking 
application. Start tracking and documenting hours spent on a project or with a 
client with Logware today. Try it for free with a 15 day trial account.
http://www.houseoffusion.com/banners/view.cfm?bannerid=67

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214467
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


RE: Fetching a website?

2005-08-10 Thread Jim Davis
 -Original Message-
 From: Dave Watts [mailto:[EMAIL PROTECTED]
 Sent: Wednesday, August 10, 2005 4:47 PM
 To: CF-Talk
 Subject: RE: Fetching a website?
 
  I still maintain that a well-written CF app can still beat a
  crappily written pure Java app.
 
 Absolutely. However, part of writing a well-written application is
 choosing
 the appropriate language for the task.

With one caveat: choose the most appropriate language that you can use well.

This may mean hiring somebody to do it or learning something new before
tackling it - but the best tool overall is not the best tool if you don't
know how to use properly.

Jim Davis





~|
Find out how CFTicket can increase your company's customer support 
efficiency by 100%
http://www.houseoffusion.com/banners/view.cfm?bannerid=49

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214469
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


RE: Fetching a website?

2005-08-10 Thread Dave Watts
   I still maintain that a well-written CF app can still beat a
   crappily written pure Java app.
  
  Absolutely. However, part of writing a well-written application is
  choosing the appropriate language for the task.
 
 With one caveat: choose the most appropriate language that 
 you can use well.
 
 This may mean hiring somebody to do it or learning something 
 new before tackling it - but the best tool overall is not the 
 best tool if you don't know how to use properly.

Well, this is getting a bit silly I think. If I know how to use a hammer,
but I don't know how to use a wrench, you probably don't want me working on
your plumbing.

Dave Watts, CTO, Fig Leaf Software
http://www.figleaf.com/

Fig Leaf Software provides the highest caliber vendor-authorized 
instruction at our training centers in Washington DC, Atlanta, 
Chicago, Baltimore, Northern Virginia, or on-site at your location. 
Visit http://training.figleaf.com/ for more information!


~|
Logware (www.logware.us): a new and convenient web-based time tracking 
application. Start tracking and documenting hours spent on a project or with a 
client with Logware today. Try it for free with a 15 day trial account.
http://www.houseoffusion.com/banners/view.cfm?bannerid=67

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214491
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


RE: Fetching a website?

2005-08-10 Thread Jim Davis
 -Original Message-
 From: Dave Watts [mailto:[EMAIL PROTECTED]
 Sent: Wednesday, August 10, 2005 6:24 PM
 To: CF-Talk
 Subject: RE: Fetching a website?
 
I still maintain that a well-written CF app can still beat a
crappily written pure Java app.
  
   Absolutely. However, part of writing a well-written application is
   choosing the appropriate language for the task.
 
  With one caveat: choose the most appropriate language that
  you can use well.
 
  This may mean hiring somebody to do it or learning something
  new before tackling it - but the best tool overall is not the
  best tool if you don't know how to use properly.
 
 Well, this is getting a bit silly I think. If I know how to use a hammer,
 but I don't know how to use a wrench, you probably don't want me working
 on
 your plumbing.

Exactly - but here's my point:

We have these conversations a lot.  Somebody asks how to do something and
somebody (sometimes myself) gets all high-and-mighty that they should be
using a better, more appropriate tool.

But the world doesn't work that way.  We have limitations.

Sometimes we have to do the best we can with what we have and what we know
in the time we have.

I agree it's getting silly.  But so is chastising people who want to
complete tasks as best they can with what they got.

This isn't a hammer and a wrench situation.  This is a shovel and a
backhoe situation: both can get the job done but one is vastly superior, but
requires much more training and resources.

Which is exactly why the best tool for the job may not be the right tool for
the job.

Jim Davis




~|
Find out how CFTicket can increase your company's customer support 
efficiency by 100%
http://www.houseoffusion.com/banners/view.cfm?bannerid=49

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214498
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


RE: Fetching a website?

2005-08-10 Thread Dave Watts
 We have these conversations a lot. Somebody asks how to do 
 something and somebody (sometimes myself) gets all high-and-
 mighty that they should be using a better, more appropriate tool.
 
 But the world doesn't work that way.  We have limitations.
 
 Sometimes we have to do the best we can with what we have and 
 what we know in the time we have.
 
 I agree it's getting silly. But so is chastising people who want 
 to complete tasks as best they can with what they got.

Your representation is inaccurate. The original conversation went something
like this:

Q: I want to write a CF application which fetches a site, like HTTrack does.
A: Use wget with CFEXECUTE.
Q: But I want to do it in CF, just because I can.
A: Well ok then, but it's going to suck compared to, well, almost anything
else, because CF isn't good at that.

Now, I don't have a problem with that - there's nothing wrong with doing
something for fun, because you can. But it's certainly not the best approach
to solving the problem. There's nothing high or mighty about that. So, I
stand by my proposition that the original poster would probably be better
off spending his time learning another language to do this than to do it
with CF.

Dave Watts, CTO, Fig Leaf Software
http://www.figleaf.com/

Fig Leaf Software provides the highest caliber vendor-authorized 
instruction at our training centers in Washington DC, Atlanta, 
Chicago, Baltimore, Northern Virginia, or on-site at your location. 
Visit http://training.figleaf.com/ for more information!


~|
Logware (www.logware.us): a new and convenient web-based time tracking 
application. Start tracking and documenting hours spent on a project or with a 
client with Logware today. Try it for free with a 15 day trial account.
http://www.houseoffusion.com/banners/view.cfm?bannerid=67

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214501
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54


RE: Fetching a website?

2005-08-10 Thread Jim Davis
 -Original Message-
 From: Dave Watts [mailto:[EMAIL PROTECTED]
 Sent: Wednesday, August 10, 2005 7:21 PM
 To: CF-Talk
 Subject: RE: Fetching a website?
 
  We have these conversations a lot. Somebody asks how to do
  something and somebody (sometimes myself) gets all high-and-
  mighty that they should be using a better, more appropriate tool.

 Q: I want to write a CF application which fetches a site, like HTTrack
 does.
 A: Use wget with CFEXECUTE.
 Q: But I want to do it in CF, just because I can.
 A: Well ok then, but it's going to suck compared to, well, almost anything
 else, because CF isn't good at that.

I think the point was specifically that he wanted to do it in CF because he
_counldn't_.  ;^)

He was purposefully setting himself a challenge to his skills - which I'm
all for.  He'll learn a lot.

But in the end I agree with you much more than I disagree - it may be better
for him to learn something else to do it - if it made sense (and in this
case it doesn't seem to me).

Jim Davis





~|
Find out how CFTicket can increase your company's customer support 
efficiency by 100%
http://www.houseoffusion.com/banners/view.cfm?bannerid=49

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:214505
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Donations  Support: http://www.houseoffusion.com/tiny.cfm/54