On Wed, 31 Mar 2004, Rick wrote: > Just installed htdig-3.2.0b5, but it has some > difficulty reading this robots.txt file ... > > User-agent: grub-client > Disallow: / > User-agent: * > Disallow: > > I believe that the robots.txt syntax is correct. > Release 3.1.6 has no problem with it. > > Any ideas/solutions/workarounds ?
There is at least one known bug involving robots.txt in 3.2.0b5. It sounds like this might be the same bug that you are running into. A patch for the problem has already been generated and is available from the patch archive in the contributed work section of the site. The direct link is provided below. ftp://ftp.ccsf.org/htdig-patches/3.2.0b5/robots.0 Jim ------------------------------------------------------- This SF.Net email is sponsored by: IBM Linux Tutorials Free Linux tutorial presented by Daniel Robbins, President and CEO of GenToo technologies. Learn everything from fundamentals to system administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click _______________________________________________ ht://Dig general mailing list: <[EMAIL PROTECTED]> ht://Dig FAQ: http://htdig.sourceforge.net/FAQ.html List information (subscribe/unsubscribe, etc.) https://lists.sourceforge.net/lists/listinfo/htdig-general
