On 1 Oct 2001 at 8:30, Geoff Hutchison wrote: > As to whether messages inside these archive directories are indexed, you > don't mention what sort of config options you're using. Are you > restricting by hopcount?
> Are you sure you've made the max_doc_size is large enough? etc. I can boost that easily. BTW: the documentation at http://www.htdig.org/attrs.html#max_doc_size does not mention units. I'm assuming bytes. It was set at 200000. The missing document is only 3719. > Try running htdig once with extra debugging flags (say -vvv) and then use > less to find the URL in question and look to see what happens after the > redirect. [root@m20:/usr/tmp] # htdig -vvv -a -i -s -c /usr/local/etc/htdig-unixathome.org -adsl.conf > htdig.out # htmerge -vvv -a -s -c /usr/local/etc/htdig-unixathome.org- adsl.conf > htmerge.out [dan@m20:/usr/local/share/htdig] $ grep 2001_05 /usr/tmp/htdig.out +A tag: pos = 2, position = ="archives/2001_05"> href: http://unixathome.org/adsl/archives/2001_05 (may) resolving 'http://unixathome.org/adsl/archives/2001_05' pushing http://unixathome.org/adsl/archives/2001_05 5:5:1:http://unixathome.org/adsl/archives/2001_05: Retrieval command for http://unixathome.org/adsl/archives/2001 _05: GET /adsl/archives/2001_05 HTTP/1.0 Header line: Location: http://www.unixathome.org/adsl/archives/2001_05/ redirect: http://www.unixathome.org/adsl/archives/2001_05/ [dan@m20:/usr/local/share/htdig] $ grep 2001_05 /usr/tmp/htmerge.out Deleted, no excerpt: 5/http://unixathome.org/adsl/archives/2001_05 [dan@m20:/usr/local/share/htdig] $ I can't find anything in htmerge.out for 05-09 of 2001. But I have a clue. If you look at http://www.unixathome.org/adsl/ you'll see that the links from that page to the archives differ: http://www.unixathome.org/adsl/archives/2001_04/ http://www.unixathome.org/adsl/archives/2001_05 http://www.unixathome.org/adsl/archives/2001_06 etc. I suspect the lack of a trailing / is causing the redirect as mentioned above and then htdig never retries the redirected URL. Does that sound plausible? -- Dan Langille The FreeBSD Diary - http://freebsddiary.org/ - practical examples _______________________________________________ htdig-general mailing list <[EMAIL PROTECTED]> To unsubscribe, send a message to <[EMAIL PROTECTED]> with a subject of unsubscribe FAQ: http://htdig.sourceforge.net/FAQ.html
