At 10:16 AM -0500 10/24/98, Wayne Spivak wrote:
>I can't for the life of me get Htdig to dig though the entire tree.
>These are the pertinent parts of my config file:
I would run "htdig -vvv" and redirect this to a file. Then look through
this file to see why it is rejecting certain URLs. My first impression is
that since you're using the local_urls, that it's rejecting files that
don't end in .html since it can't parse .cgi files.
-Geoff Hutchison
Williams Students Online
http://wso.williams.edu/
>database_dir: /www/htdig/db_com
>start_url: http://com.sb.usps.org/
>limit_urls_to: com.sb.usps.org
>exclude_urls: /cgi-bin/
>max_head_length: 60000
>search_algorithm: exact:1 synonyms:0.5 endings:0.1 soundex:0.5
>metaphone:0.5
>maintainer: [EMAIL PROTECTED]
>allow_numbers: true
>allow_virtual_hosts: true
>local_urls: http://com.sb.usps.org=/nat/
>create_url_list: true
>url_list: /www/htdig/url_list_com
>timeout: 42
----------------------------------------------------------------------
To unsubscribe from the htdig mailing list, send a message to
[EMAIL PROTECTED] containing the single word "unsubscribe" in
the body of the message.
