Is anyone keeping a tally of these kinds of questions? <<[PATCH] HTTPS and PROXY fix>> -----Original Message----- From: Tobias Manthey [mailto:[EMAIL PROTECTED]] Sent: Tuesday, October 17, 2000 6:08 AM To: [EMAIL PROTECTED] Subject: Method CONNECT Problem Hi all, I am using LWP 5.48 and SSLeavy. What I want: Connect to an https URL, via proxy, using the CONNECT method. So far so good: My testroutine sends the requests and suceeds. Im getting the following response: $res->message => OK $res->code => 200 $res->headers => HTTP::Headers=HASH(0x2028fb18) $res->content; => EMPTY $res->{client_socket} => IO::Socket::INET=GLOB(0x202781a0) So I got the Sockethandle........ and now I have a problem. What I am going to do with it? Can somebody provide a small example in how to deal with https requests using the CONNECT Method? TIA Tobias -- Sent through GMX FreeMail - http://www.gmx.net
I dug through the archives, and didn't find a patch that would apply to 5.48 to fix these problems. So I decided to dig in, and see what was broken. As mentioned before, one problem was that it was trying to do a GET https://some.example.com/ to the proxy, instead of the CONNECT syntax that Netscape and IE use. After fixing that, I discovered another bug, that prevented POST's from working properly. It was sending headers and such on the CONNECT, which were confusing the web server when it was expecting SSL negotiation. This patch contains modified versions of http.pm and https.pm as applied to libwww-perl-5.48. It enables HTTPS connections to work through a proxy server. I've tested it using an Apache proxy server. GETs and POSTs both work correctly for the things I needed them for. I don't know if my hacks break anything for direct connections, or if it breaks anything like client certificates. I'm not much of a Perl hacker yet, so I've probably done things suboptimally... But it works for what I am using it for. I hope someone fixes this problem properly, and gets it put into the next version of libwww. Chris Hiner [EMAIL PROTECTED] It is partially based on a patch by: WORENKLEIN, David, GCM ([EMAIL PROTECTED]) titled: [PATCH] LWP 5.47, Keep-Alive, and HTTPS through a proxy server on Wed, 5 Apr 2000 16:12:22 -0400 Which is in turn based on a patch by Scott Gifford <[EMAIL PROTECTED]>diff -N -c5 -r -w libwww-perl-5.48/lib/LWP/Protocol/http.pm libwww-perl-5.48.mine/lib/LWP/Protocol/http.pm *** libwww-perl-5.48/lib/LWP/Protocol/http.pm Sun Apr 9 15:06:46 2000 --- libwww-perl-5.48.mine/lib/LWP/Protocol/http.pm Sun Aug 6 23:34:24 2000 *************** *** 111,123 **** $port = $url->port; $fullpath = $url->path_query; $fullpath = "/" unless length $fullpath; } # connect to remote site ! my $socket = $self->_new_socket($host, $port, $timeout); $self->_check_sock($request, $socket); my $sel = IO::Select->new($socket) if $timeout; my $request_line = "$method $fullpath HTTP/1.0$CRLF"; --- 111,128 ---- $port = $url->port; $fullpath = $url->path_query; $fullpath = "/" unless length $fullpath; } + my $socket; # connect to remote site ! if(defined ($request->{client_socket})) { ! $socket=$request->{client_socket}; ! } else { ! $socket = $self->_new_socket($host, $port, $timeout); $self->_check_sock($request, $socket); + } my $sel = IO::Select->new($socket) if $timeout; my $request_line = "$method $fullpath HTTP/1.0$CRLF"; *************** *** 142,157 **** --- 147,171 ---- $self->_fixup_header($h, $url); my $buf = $request_line . $h->as_string($CRLF) . $CRLF; my $n; # used for return value from syswrite/sysread + + # CH This seems to fix the HTTPS PROXY POST problems + if ($method eq "CONNECT") { + $buf = $request_line . $CRLF . $CRLF; + } + die "write timeout" if $timeout && !$sel->can_write($timeout); $n = $socket->syswrite($buf, length($buf)); die $! unless defined($n); die "short write" unless $n == length($buf); LWP::Debug::conns($buf); + # CH This seems to fix the HTTPS PROXY POST problems + if ($method ne "CONNECT") { + if ($ctype eq 'CODE') { while ( ($buf = &$cont_ref()), defined($buf) && length($buf)) { die "write timeout" if $timeout && !$sel->can_write($timeout); $n = $socket->syswrite($buf, length($buf)); die $! unless defined($n); *************** *** 164,174 **** $n = $socket->syswrite($$cont_ref, length($$cont_ref)); die $! unless defined($n); die "short write" unless $n == length($$cont_ref); LWP::Debug::conns($buf); } ! # read response line from server LWP::Debug::debug('reading response'); my $response; $buf = ''; --- 178,188 ---- $n = $socket->syswrite($$cont_ref, length($$cont_ref)); die $! unless defined($n); die "short write" unless $n == length($$cont_ref); LWP::Debug::conns($buf); } ! } # read response line from server LWP::Debug::debug('reading response'); my $response; $buf = ''; diff -N -c5 -r -w libwww-perl-5.48/lib/LWP/Protocol/https.pm libwww-perl-5.48.mine/lib/LWP/Protocol/https.pm *** libwww-perl-5.48/lib/LWP/Protocol/https.pm Sun Apr 9 15:06:07 2000 --- libwww-perl-5.48.mine/lib/LWP/Protocol/https.pm Sun Aug 6 23:35:07 2000 *************** *** 68,73 **** --- 68,130 ---- $res->header("Client-SSL-Cert-Issuer" => $cert->issuer_name); } $res->header("Client-SSL-Warning" => "Peer certificate not verified"); } + # proxy connect patch + sub request + { + my $self = shift; + my($request, $proxy, $arg, $size, $timeout) = @_; + LWP::Debug::trace('()'); + + return $self->SUPER::request(@_) + unless ($proxy); + + # We need to get through the proxy server with a regular + # http connection, then we transmogrify the connection into + # a secure socket. + + my $url = $request->url; + my $host = $url->host; + my $port = $url->port; + + my $socket; + + my $proxy_request = $request->clone; + $proxy_request->method("CONNECT"); + # $proxy_request->uri(new URI("$host:$port")); + + my $proxy_protocol = LWP::Protocol::create('http'); + + LWP::Debug::trace("Trying to CONNECT through the proxy server"); + + my $proxy_response = $proxy_protocol->request($proxy_request, $proxy, $arg, +$size, $timeout); + + # Did we get through the proxy server? + return $proxy_response + unless($proxy_response->code == 200); + + $socket=$proxy_response->{client_socket}; + + # Turn the socket into an SSL socket + LWP::Debug::trace("Transmogrifying our socket"); + bless $socket, "Net::SSL"; + *$socket->{'ssl_ctx'} = Net::SSL::_default_context(); + my $ssl = Crypt::SSLeay::Conn->new(*$socket->{'ssl_ctx'}, $socket); + if ($ssl->connect <= 0) { + # XXX should obtain the real SSLeay error message + LWP::Debug::trace("SSL error:".$ssl->connect); + $socket->_error("SSL negotiation failed"); + return new HTTP::Response &HTTP::Status::RC_INTERNAL_SERVER_ERROR, + 'SSL negotiation failed.'; + } + *$socket->{'ssl_ssl'} = $ssl; + + $request->{client_socket} = $socket; # so it can be picked up + + return $self->SUPER::request($request, undef, $arg, $size, $timeout); + } + + + 1;
