On Thu, 29 Mar 2007, Joe Auty wrote: > Date: Thu, 29 Mar 2007 17:05:55 -0400 > From: Joe Auty <[EMAIL PROTECTED]> > To: Jim Cole <[EMAIL PROTECTED]> > Cc: [email protected] > Subject: Re: [htdig] Status of htdig support of crawling SSL sites? > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Cool, that sounds exactly like what I've been experiencing. > > Do you think somebody could send me a copy of the patch? I expect that > some characters were garbled with my copy and paste of the list > archive, as I'm getting the following (where "patch" is a file > containing the paste of the patch included within this thread)
Here it is on the patch site: ftp://ftp.ccsf.org/htdig-patches/3.2.0b6/SSL_pending.0 Regards, Joe -- _/ _/_/_/ _/ ____________ __o _/ _/ _/ _/ ______________ _-\<,_ _/ _/ _/_/_/ _/ _/ ......(_)/ (_) _/_/ oe _/ _/. _/_/ ah [EMAIL PROTECTED] > > # patch -p0 <patch > patching file htnet/SSLConnection.cc > patch: **** malformed patch at line 4: { > > > Thanks in advance! > > > > Jim Cole wrote: > > Hi - There is a bug with SSL handling in the 3.2.0b6 code. While a > > patch has been applied to what is in CVS, it will not be in the > > version you are working with unless the RPM producer added it on > > their own. I suspect that it is this bug that is causing the > > problems you report below. The reason it seems to take an > > unexpectedly long time to generate the output you provided is due to > > the fact that the connection is not functioning properly and has to > > wait on a 30 second timeout three times before htdig gives up. > > > > Here are a couple links regarding the SSL bug, including a patch. > > > > > > http://sourceforge.net/mailarchive/message.php?msg_id=Pine.LNX.4.60.0408262141580.7827%40loki.yggdrasill.net > > > > > > http://sourceforge.net/mailarchive/forum.php?thread_name=Pine.LNX.4.63.0505230007240.26726%40loki.yggdrasill.net&forum_name=htdig-dev > > > > > > Jim > > > > > > On Mar 29, 2007, at 11:08 AM, Joe Auty wrote: > > > >> I've rebuilt htdig with --with-ssl, and it appears as if I have SSL > >> support, but I'm having the following problem now... I'm wondering if > >> htdig has difficulty with self-signed SSL certs? As it stands, it > >> takes quite a while to produce the following output: > >> > >> (I've replaced the real URL of my site with mysite.com) > >> > >> > >> > >> # ./htdig -vvvvv > >> ht://dig Start Time: Thu Mar 29 12:34:34 2007 > >> 1:1:https://mysite.com/assetdb > >> New server: mysite.com, 443 > >> - Persistent connections: enabled > >> - HEAD before GET: enabled > >> - Timeout: 30 > >> - Connection space: 0 > >> - Max Documents: -1 > >> - TCP retries: 1 > >> - TCP wait time: 5 > >> - Accept-Language: > >> Trying to retrieve robots.txt file > >> Creating an HtHTTPSecure object > >> Making HTTPS request on https://mysite.com/robots.txt > >> Making a HEAD call before the GET > >> Try to get through to host mysite.com (port 443) > >> 1 - Open of the connection ok > >> Assigned the remote host mysite.com > >> Assigned the port 443 > >> 1 - Connection fell down ... let's close it > >> Request time: 30 secs > >> Making a HEAD call before the GET > >> Try to get through to host mysite.com (port 443) > >> 2 - Open of the connection ok > >> Assigned the remote host mysite.com > >> Assigned the port 443 > >> 2 - Connection fell down ... let's close it > >> Request time: 30 secs > >> . Making a HEAD call before the GET > >> Try to get through to host mysite.com (port 443) > >> 3 - Open of the connection ok > >> Assigned the remote host mysite.com > >> Assigned the port 443 > >> 3 - Connection fell down ... let's close it > >> Request time: 30 secs > >> . pushed > >> pick: mysite.com, # servers = 1 > >>> mysite.com supports HTTP persistent connections (infinite) > >> ht://dig End Time: Thu Mar 29 12:36:04 2007 > > > > > - -- > Joe Auty > UITS Messaging > Indiana University > [EMAIL PROTECTED] > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.6 (Darwin) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > iD8DBQFGDCozIGoilq3QRWsRAkQXAKDNvT2B43wn4rI3Qg62p+ET5RO+UgCfeD5e > dGib3fpIJ3LYc+7B+ZrRxYw= > =mFVu > -----END PGP SIGNATURE----- > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys-and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > ht://Dig general mailing list: <[email protected]> > ht://Dig FAQ: http://htdig.sourceforge.net/FAQ.html > List information (subscribe/unsubscribe, etc.) > https://lists.sourceforge.net/lists/listinfo/htdig-general ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ ht://Dig general mailing list: <[email protected]> ht://Dig FAQ: http://htdig.sourceforge.net/FAQ.html List information (subscribe/unsubscribe, etc.) https://lists.sourceforge.net/lists/listinfo/htdig-general
