Re: web searching scripts
On Fri, 04 Aug 2006 18:11:18 +0200, Avell Diroll <[EMAIL PROTECTED]> wrote: > [EMAIL PROTECTED] wrote: >> Does anyone know of a freely available script that can take a given URL >> and follow every link within it? >> >> Ideally, I would like to start with this to build a quick application >> to grab all the content off a website to publish it to a CD. ... > If you just want to download websites (i.e. not necessarily writing a > program yourself to do that), you may try Httrack, it might suite your > needs. The well-known Gnu wget is what I always use. (IMHO, this is a situation where it's a /good/ idea to glue together existing software, rather than joining many bits of code to a mirror-from-http-to-cdr application.) /Jorgen -- // Jorgen Grahn R'lyeh wgah'nagl fhtagn! -- http://mail.python.org/mailman/listinfo/python-list
Re: web searching scripts
[EMAIL PROTECTED] wrote: > Does anyone know of a freely available script that can take a given URL > and follow every link within it? > > Ideally, I would like to start with this to build a quick application > to grab all the content off a website to publish it to a CD. > > Thanks, > > jul > If you just want to download websites (i.e. not necessarily writing a program yourself to do that), you may try Httrack, it might suite your needs. http://www.httrack.com/ There even seem to be some sort of python bindings ... http://www.satzbau-gmbh.de/staff/abel/httrack-py/ But there might be some more pythonic solution around ... i would start looking at twisted or cherrypy, but i never used them myself ... HIH regards Avell -- http://mail.python.org/mailman/listinfo/python-list
web searching scripts
Does anyone know of a freely available script that can take a given URL and follow every link within it? Ideally, I would like to start with this to build a quick application to grab all the content off a website to publish it to a CD. Thanks, jul -- http://mail.python.org/mailman/listinfo/python-list
