"Aaron S. Hawley" <[EMAIL PROTECTED]> writes: > The HTML of those pages contains the meta-tag > > <meta name="robots" content="noindex,nofollow" /> > > and Wget listened, and only downloaded the first page. > > Perhaps Wget should give a warning message that the file contained a > meta-robots tag, so that people aren't quite so dumb-founded.
Good point. A message would be easy to add, and in this case enormously useful.
