According to Pub Litics: > Thanks for the helpful and prompt response. > > Have read 4.24 and 5.29 and supporting material. I have ruled out > blocking the extensions, because there are no discernible extensions. > For example, here are a few the multifarious search results: > > >/forum/viewtopic.php?t=18&start=0&postdays=0&postorder=asc&highlight=&sid=981ab2115a733110a6f9753da88aa73f > > > /forum/index.php?sid=a7fe32da1d54d5e820d29cef03db11b5 > > I already have /forum listed under URLs to be blocked, in the CONFIG > file, but apparently that is not halting the searching process in > that respect. > > I, of course, would like to exclude all of such types, per your > suggestion. So, I am focusing on bad query string as the likely > culprit. Only problem is, here is your example: > > bad_querystr: forum=private section=topsecret&passwd=required > > Pardon if I sound dense, but I cannot find in the instructions where > to place this. Would it go in the CONFIG file? If so, where abouts?
I think you're confusing the CONFIG file, which is used for setting certain paths and other settings at compile time, and htdig.conf, which is used for setting config attributes at run time. See http://www.htdig.org/FAQ.html#q4.18 http://www.htdig.org/config.html#htdig.conf http://www.htdig.org/confindex.html > Would it be changed? I assume that "forum" would be the name of my > forum. But, not sure what "section" refers to or "topsecret" means. > I assume that "password" would refer to my password? Or, should the word > "required" stay as it is? I think you're taking the example in the documentation far too literally. It's just an example of usage, not a suggestion taylor-made for your site. The general idea is to include any portions of query strings that appear in your URLs, upon which you'd want htdig to stop going any deeper and not index those URLs. I think before you can use bad_querystr or exclude_urls effectively, you need to step back and get an idea of how the whole hierarchy of links from document to document fits together on your site - i.e. what does this tree look like, and where are the best places to prune it? -- Gilles R. Detillieux E-mail: <[EMAIL PROTECTED]> Spinal Cord Research Centre WWW: http://www.scrc.umanitoba.ca/ Dept. Physiology, U. of Manitoba Winnipeg, MB R3E 3J7 (Canada) ------------------------------------------------------- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf _______________________________________________ htdig-general mailing list <[EMAIL PROTECTED]> To unsubscribe, send a message to <[EMAIL PROTECTED]> with a subject of unsubscribe FAQ: http://htdig.sourceforge.net/FAQ.html
