According to Geoff Hutchison:
> On 30 Aug 2000 [EMAIL PROTECTED] wrote:
> > This directory doesn't contain any index.htm. so i point htdig to this
> > directory, and the html pages are indexed fine. however, as search results
> > also the filenames of the html pages are considered. if you click such a
> > link, you get the directory listing.
>
> Sure. That's because the page generated for the directory index includes
> the filenames. If you don't want the text of links indexed, set
> description_factor: 0 in your config file:
> <http://www.htdig.org/attrs.html#description_factor>
That will cause a search for words in the file names not to point to the
pages of those names, but it won't prevent the directory listing itself
from popping up in the search results, which I think is what he wanted to
eliminate.
> > Now, if i put an index.html in this directory, all the other pages
> > (butcher.htm, etc..) are NOT indexed. I guess because there is no reference
> > in index.htm to butcher.htm.
>
> Right, you got it--htdig follows links. Put in an index.html and the
> automatic directory-generation by the server stops. You could, of course,
> use the index.html for links to the files or whatever you want and that
> would also solve your question.
Yes, and if maintaining static index.html files is not an option, there's
also the trick we talked about on Aug 2:
Acording to Geoff Hutchison:
> On Wed, 2 Aug 2000, Gilles Detillieux wrote:
> > If you can coax your web server into inserting a tag like the following
> > into the head section of the pages it generates for indexes, that would
> > be the solution.
> >
> > <meta name="robots" content="noindex,follow">
> >
> > Unfortunately, I don't believe there's any way to do so.
>
> That's not quite true. If you're using Apache, you can use HeaderName and
> IndexOptions (in particular SupressHTMLPreamble) to insert anything you
> want into the headers:
> <http://www.apache.org/docs/mod/mod_autoindex.html#headername>
> <http://www.apache.org/docs/mod/mod_autoindex.html#indexoptions>
>
> You'd still need to make some sort of header file with the appropriate
> headers.
--
Gilles R. Detillieux E-mail: <[EMAIL PROTECTED]>
Spinal Cord Research Centre WWW: http://www.scrc.umanitoba.ca/~grdetil
Dept. Physiology, U. of Manitoba Phone: (204)789-3766
Winnipeg, MB R3E 3J7 (Canada) Fax: (204)789-3930
------------------------------------
To unsubscribe from the htdig3-dev mailing list, send a message to
[EMAIL PROTECTED]
You will receive a message to confirm this.
