Hi, According to my understanding:
1- default behavior is to enable persistent_connection; 2- htdig tries to get /robots.txt on every server in start_url; 3- In the same time, htdig detects wether the remote server support persistent connection mechanism. If the previouses sentences are true then there is a problem with some HTTP servers. A few of them add a 'Connection: close' HTTP header when returning HTTP status code 404. So if /robots.txt doesn't exist, and the HTTP server is 'bogus', htdig thinks the persistent connections mechanism is not supported. But it false. If you get another document (which exists) it's ok (remote server waits for another request in persistent mode). I've found that Microsoft-IIS/4.0 and 5.0 have this problem. Apache doesn't have it. Does someone could confirm this behavior ? Best regards, Alain FORCIOLI http://www.VigiNews.com/ http://www.risc.fr/ [EMAIL PROTECTED] http://www.april.org/ [EMAIL PROTECTED] GPG Public Key: http://www.risc.fr/~af/gpg.txt -- Alain FORCIOLI http://www.VigiNews.com/ http://www.risc.fr/ [EMAIL PROTECTED] http://www.april.org/ [EMAIL PROTECTED] GPG Public Key: http://www.risc.fr/~af/gpg.txt _______________________________________________________________ Have big pipes? SourceForge.net is looking for download mirrors. We supply the hardware. You get the recognition. Email Us: [EMAIL PROTECTED] _______________________________________________ htdig-dev mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/htdig-dev
