At 10:03 PM -0500 2/9/00, bob wrote:
>The site is a directory site of related sites. What do I do to stop
>ht://Dig from following links to other sites? Put "?" at the end of
You can easily take care of this using the limit_urls_to attribute.
It is initially set to ${start_url} which will normally restrict it
to one server.
>Should I put "?" after the URL of the home page? Or, should I start
>an index page (for ht://Dig to start at) of all the URLs of the
>pages of the site that I want ht://Dig to index? And, not have a
>robot meta tag on the home page?
I don't see what the "?" would do unless you set that as a pattern in
exclude_urls. Besides, as a user I would find it really annoying. If
you really don't want ht://Dig to index the starting page but you do
want other search engines to do this, you will probably have to at
least put the links from the starting page into a separate file.
-Geoff Hutchison
Williams Students Online
http://wso.williams.edu/
------------------------------------
To unsubscribe from the htdig mailing list, send a message to
[EMAIL PROTECTED]
You will receive a message to confirm this.
