-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Yazeed Hamid wrote: > Hi to all. > Thank you very much for your efforts and a happy new year to all. > > I have been using wget in a bash script to measure website response > times over different proxy configurations. > > /usr/bin/time "%e Seconds" /usr/bin/wget -pEkq > --delete-after --proxy=$switch $URL[$index] > > I'm using wget version 1.10.2 on cygwin running on Windows Vista (v > 6.0.6000 Build 6000). > When there is a proxy to go through, the corresponsent proxy > address:port is exported to the environment (export http_proxy=$proxy") > > The problem is, when wget is working through a proxy, it doesn't seem to > reuse the existing > http connection like it does when --proxy=off is set. When there is no > proxy, all objects > referenced in an html file are fetched over the same connection. On the > other hand, when I am > going through a proxy, each object is fetched in a new connection > although the Connection: Keep-Alive header > is both in the http request and response messages. > > As a result, the measured response time through a proxy is very > much greater than that through direct connection <no proxy>. > > $ /usr/bin/time -f "%e Seconds" wget -pdEk --delete-after --proxy=off -o > log-no-proxy-w-debug www.mcafee.com <http://www.mcafee.com> > 3.56 Seconds > > $ /usr/bin/time -f "%e Seconds" wget -pdEk --delete-after --proxy=on -o > log-thru-proxy-w-debug www.mcafee.com <http://www.mcafee.com> > 20.42 Seconds > > How can I make wget make sense of the Connection: Keep-Alive message > when going through > a proxy and thus reuse the existing connection like it does when > directly connecting to the web server? > In an early post, I read something about modifying the source of wget, > is this the only solution?
Probably. It does look like you're right that Wget is dropping the connection; it'd be more certain if i could see tcpdump output indicating which side closed the connection first; but IIRC the debug messages are different when the remote side closes the connection first (I haven't time to check the source to see for myself just now). I'll file an issue for this, but it's not likely to be addressed anytime soon. You mentioned another message that talked about modifying the source; do you have a reference to it? If someone has already found an appropriate patch for this, it can be in much quicker than if you wait for me to get around to it. ;) - -- Micah J. Cowan Programmer, musician, typesetting enthusiast, gamer... http://micah.cowan.name/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHgm377M8hyUobTrERAoYqAJ9j8M6BNbzjE3oD8iMNCAdjU1eh8gCfdhlj 8Q7itdzlklBbyazbI3wS5iQ= =KlHm -----END PGP SIGNATURE-----
