Anyone have a solution for excluding "section fronts" "menu pages" "index pages" from search results?
For the most part, ht://dig is returning great search results. But section front pages rank very high and really don't have the actual content users are searching for. I have set the backlink_factor to zero which seems to have been somewhat helpful in solving the problem. There are too many section front pages to add to "exclude_url" list. (Plus it seems that this would eliminate the entire section). Excluding all files named *index* is not a solution either. Ideally a htdig-noindex-follow meta tag would be great for me, since pages are dynamic. This question was brought up before, http://www.htdig.org/mail/1999/10/0069.html A simple <meta name="robots" content="noindex,follow"> is not a solution. These pages are very important for external search engines. (Think of the NYTimes.com international news page http://www.nytimes.com/pages/world/ - good content, but probably not what you are looking for with an intrasite search). I think what i'll end up doing is "cloaking" my pages for ht://dig. If the user agent is htdig then I'll output a robots meta tag for it: <meta name="robots" content="noindex,follow"> Thanks much, Josh __________________________________ Do you Yahoo!? Yahoo! Small Business $15K Web Design Giveaway http://promotions.yahoo.com/design_giveaway/ ------------------------------------------------------- This SF.Net email is sponsored by: IBM Linux Tutorials Free Linux tutorial presented by Daniel Robbins, President and CEO of GenToo technologies. Learn everything from fundamentals to system administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click _______________________________________________ ht://Dig general mailing list: <[EMAIL PROTECTED]> ht://Dig FAQ: http://htdig.sourceforge.net/FAQ.html List information (subscribe/unsubscribe, etc.) https://lists.sourceforge.net/lists/listinfo/htdig-general
