Thanks!
I see, but then, how to exclude from being downloaded per file-basis?

-----Original Message-----
From: Hrvoje Niksic [mailto:[EMAIL PROTECTED]]
Sent: 22 November 2001 14:45
To: Wget List; Tomas Hjelmberg
Subject: Re: bug?


Tomas Hjelmberg <[EMAIL PROTECTED]> writes:

> I want to exclude /var/www/html/tomas.html from being indexed.
> It looks like:
[...]
>       <meta name="robots" content="noindex">
>       <title>Tomas</title>
[...]
> 
> I invoke wget with:
> wget -r http://localhost
> And tomas.html is unfourtnuately downloaded anyway...

Wget doesn't reallly "index" anything, so it pretty much ignores
`noindex'.  You can specify "nofollow" in which case Wget will refuse
to recurse into the document.

Reply via email to