On Friday, January 24, 2003, at 02:08 PM, Gilles Detillieux wrote:
So I was thinking that it was my custom 404 pages, so I eliminated them in httpd.conf and ran the dig again. No such luck. What is even stranger is that in my testing, I killed a dig while it was running and when I got the report, it had a few errors listed... I thought I had finally got the config right so I immediately ran the dig again, thinking that if I let it go I would get a full report... the report came back with no errors. This brings me back to thinking that somewhere in its processing it is dropping that information...According to Conrad Schilbe:OK, then it must be that the server is never returning any 404 status codes.It isn't running in update mode. I even added `remove_bad_urls: false' to the configuration file.
Are you sure this site has links to non-existant URLs?
No, the logic in the code is as follows...If that doesn't help, have a look at how 404 errors are dealt with on that site. It may be that htdig is never seeing that status code there, but is instead getting some other document (e.g. an error page), with a normal status code, for any unresolvable URL on that site.Even if it is not seeing any bad URLs possibly caused by the way 404s are handled, it should still output `Errors to take note of:' in the report. That text should be there even when there are no errors... I have seen it in my tests. Which makes me believe that something is failing.
if (notFound.length() > 0)
{
cout << "\n" << name << ": Errors to take note of:\n";
cout << notFound;
}
so if "notFound" is never set to anything, it won't put out the "Errors
to take note of" message either. notFound is only set (i.e. appended
to) when there is one of the following errors occurs for a given URL:
"Not found", "Unknown host", "Unable to contact server". The latter two
are detected internally by htdig, if the name lookup fails or the attempt
to open the connection fails. The first one, "Not found", only occurs
if the HTTP server returns a status code other than 200, 30*, or 401.
If none of these errors occurs, htdig shouldn't tell you to take note
of them.
When I use the -t option it produces a document list that includes information about failed requests, correct? I will cross my lack of errors in the report with this database in hopes that I can eliminate my server configuration from the cause.
Conrad.
-------------------------------------------------------
This SF.NET email is sponsored by:
SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
http://www.vasoftware.com
_______________________________________________
htdig-general mailing list <[EMAIL PROTECTED]>
To unsubscribe, send a message to <[EMAIL PROTECTED]> with a subject of unsubscribe
FAQ: http://htdig.sourceforge.net/FAQ.html
