RE: Summary: A CF limitation in building a spider?
> Does anyone know if a template will timeout if called from > the command line (eg as a sceduled task). I'm thinking that > it will only timeout when called from the browser, and then > only because the browser stops waiting. Can someone confirm > or deny this? Templates will timeout if they take longer than the specified maximum timeout value, as long as there is a specified maximum timeout in the CF Administrator, whether called from the browser or from the command line. You can override this timeout using the RequestTimeout URL parameter: SET QUERY_STRING=RequestTimeout=100 SET TEMPLATE_PATH=C:\Inetpub\wwwroot\myfile.cfm C:\CFUSION\BIN\CFML.EXE Dave Watts, CTO, Fig Leaf Software http://www.figleaf.com/ voice: (202) 797-5496 fax: (202) 797-5444 ~~ Structure your ColdFusion code with Fusebox. Get the official book at http://www.fusionauthority.com/bkinfo.cfm Archives: http://www.mail-archive.com/cf-talk@houseoffusion.com/ Unsubscribe: http://www.houseoffusion.com/index.cfm?sidebar=lists
RE: Summary: A CF limitation in building a spider?
I still come back to my point (see the summary). CF is not the correct tool to build a spider. It's oriented for web applications and effectively parsing and inserting HTML/WAP/A.N.other text (although CF5 will probably change all that). For spiders, using it for anything other than a simple spider is not sensible at all. Look out the Java spiders (or better still write one!). It's really not that difficult to see how you can extend CF using Java to do this. Of course you can use cfschedule to stop crashing and infinite loops. Still, when you build an http spider, it should be very quick indeed. The process should be: 1: send out an http request for the url 2: receive information (ie error message or text) back from the server 3: parse the text to get all info needed and put into an array 4: start the process again with the next url Doing this in CF takes up time and CF's resources that it doesn't need to take up. Leave it to some other tool to handle outside the "web serving" environment. Bear in mind that CF's Regular Expressions are not the best, and as far as parsing text goes, there are much better languages to parse text in. CF should not handle too much text parsing. Paul > -Original Message- > From: Daniel Lancelot [mailto:[EMAIL PROTECTED]] > Sent: 06 December 2000 11:07 > To: CF-Talk > Subject: RE: Summary: A CF limitation in building a spider? > > > depends on the settings in cfadmin - you can set pages to time > out after so > many min/sec to prevent the server from ccrashing on infinite loops. (bad > programming) > > :> -Original Message- > :> From: Bruce Heerssen [mailto:[EMAIL PROTECTED]] > :> Sent: 05 December 2000 23:43 > :> To: CF-Talk > :> Subject: RE: Summary: A CF limitation in building a spider? > :> > :> > :> Does anyone know if a template will timeout if called from > :> the command line (eg > :> as a sceduled task). I'm thinking that it will only timeout > :> when called from the > :> browser, and then only because the browser stops waiting. > :> Can someone confirm or > :> deny this? > :> > :> Thanks > :> > :> -- Bruce > :> > :> > -Original Message- > :> > From: Phill Gibson [mailto:[EMAIL PROTECTED]] > :> > > :> > > :> > I will probably also eventually put the CFHTTP one to > :> > work with . > :> > > :> > > :> > Phill Gibson > :> > Velawebs Web Designs > :> > www.Velawebs.com > :> > [EMAIL PROTECTED] > :> > > :> > :> > :> > ~~ Structure your ColdFusion code with Fusebox. Get the official book at http://www.fusionauthority.com/bkinfo.cfm Archives: http://www.mail-archive.com/cf-talk@houseoffusion.com/ Unsubscribe: http://www.houseoffusion.com/index.cfm?sidebar=lists
RE: Summary: A CF limitation in building a spider?
Look at the CoolFusion.com cfx tag, which calls the CFserver directly without going thru' the webserver.. it basically works the same way that a webserver does when passing a request on to a application server. (I persume!). Justin >-Original Message- >From: Bruce Heerssen [mailto:[EMAIL PROTECTED]] >Sent: Tuesday, December 05, 2000 11:43 PM >To: CF-Talk >Subject: RE: Summary: A CF limitation in building a spider? > > >Does anyone know if a template will timeout if called from the >command line (eg >as a sceduled task). I'm thinking that it will only timeout when >called from the >browser, and then only because the browser stops waiting. Can >someone confirm or >deny this? > >Thanks > >-- Bruce > >> -Original Message- >> From: Phill Gibson [mailto:[EMAIL PROTECTED]] >> >> >> I will probably also eventually put the CFHTTP one to >> work with . >> >> >> Phill Gibson >> Velawebs Web Designs >> www.Velawebs.com >> [EMAIL PROTECTED] >> > > > ~~ Structure your ColdFusion code with Fusebox. Get the official book at http://www.fusionauthority.com/bkinfo.cfm Archives: http://www.mail-archive.com/cf-talk@houseoffusion.com/ Unsubscribe: http://www.houseoffusion.com/index.cfm?sidebar=lists
RE: Summary: A CF limitation in building a spider?
depends on the settings in cfadmin - you can set pages to time out after so many min/sec to prevent the server from ccrashing on infinite loops. (bad programming) :> -Original Message- :> From: Bruce Heerssen [mailto:[EMAIL PROTECTED]] :> Sent: 05 December 2000 23:43 :> To: CF-Talk :> Subject: RE: Summary: A CF limitation in building a spider? :> :> :> Does anyone know if a template will timeout if called from :> the command line (eg :> as a sceduled task). I'm thinking that it will only timeout :> when called from the :> browser, and then only because the browser stops waiting. :> Can someone confirm or :> deny this? :> :> Thanks :> :> -- Bruce :> :> > -Original Message- :> > From: Phill Gibson [mailto:[EMAIL PROTECTED]] :> > :> > :> > I will probably also eventually put the CFHTTP one to :> > work with . :> > :> > :> > Phill Gibson :> > Velawebs Web Designs :> > www.Velawebs.com :> > [EMAIL PROTECTED] :> > :> :> :> ~~ Structure your ColdFusion code with Fusebox. Get the official book at http://www.fusionauthority.com/bkinfo.cfm Archives: http://www.mail-archive.com/cf-talk@houseoffusion.com/ Unsubscribe: http://www.houseoffusion.com/index.cfm?sidebar=lists
RE: Summary: A CF limitation in building a spider?
Scheduled tasks in particular accept a timeout attribute; you can set them individually. -Original Message- From: Bruce Heerssen [mailto:[EMAIL PROTECTED]] Sent: Tuesday, December 05, 2000 3:43 PM To: CF-Talk Subject: RE: Summary: A CF limitation in building a spider? Does anyone know if a template will timeout if called from the command line (eg as a sceduled task). I'm thinking that it will only timeout when called from the browser, and then only because the browser stops waiting. Can someone confirm or deny this? Thanks -- Bruce > -Original Message- > From: Phill Gibson [mailto:[EMAIL PROTECTED]] > > > I will probably also eventually put the CFHTTP one to > work with . > > > Phill Gibson > Velawebs Web Designs > www.Velawebs.com > [EMAIL PROTECTED] > ~~ Structure your ColdFusion code with Fusebox. Get the official book at http://www.fusionauthority.com/bkinfo.cfm Archives: http://www.mail-archive.com/cf-talk@houseoffusion.com/ Unsubscribe: http://www.houseoffusion.com/index.cfm?sidebar=lists
RE: Summary: A CF limitation in building a spider?
Does anyone know if a template will timeout if called from the command line (eg as a sceduled task). I'm thinking that it will only timeout when called from the browser, and then only because the browser stops waiting. Can someone confirm or deny this? Thanks -- Bruce > -Original Message- > From: Phill Gibson [mailto:[EMAIL PROTECTED]] > > > I will probably also eventually put the CFHTTP one to > work with . > > > Phill Gibson > Velawebs Web Designs > www.Velawebs.com > [EMAIL PROTECTED] > ~~ Structure your ColdFusion code with Fusebox. Get the official book at http://www.fusionauthority.com/bkinfo.cfm Archives: http://www.mail-archive.com/cf-talk@houseoffusion.com/ Unsubscribe: http://www.houseoffusion.com/index.cfm?sidebar=lists
