On 27 Jun 2001, at 9:43, "Bazuka" <[EMAIL PROTECTED]> wrote:
> I have modified Wget slightly so it writes the URL and file info to a
> database and then deletes the actual file (with --delete-after). Can I
> use -nc option in that case ?
Yes, but it download the deleted files again.
> The `-nc' option should do what you want.
My understanding of the -nc option is that it doesn't overwrite the existing
files. What does it do when downloading "index.html" files ? There might be
many copies of that file...so does it not overwrite the previous copy ?
I have modified Wget slight
"Bazuka" <[EMAIL PROTECTED]> writes:
> If I am running Wget overnight to crawl some sites (say about 50,000
> URLs) and it crashes/hangs up for some reason after retrieving half
> of them, is it possible to restart it from the point where it
> crashed (instead of downloading everything again ) ?
If I am running Wget overnight to crawl some sites (say about 50,000 URLs)
and it crashes/hangs up for some reason after retrieving half of them, is it
possible to restart it from the point where it crashed (instead of
downloading everything again ) ?
