You should make sure you switch the filter AFTER the flush of all the
html data. You can do this by adding a switch_filter inline state
that looks like this and yield to it with the request obj where you
now just switch the filter:
(I assume some things here...like the shutdown event as an example.)
$kernel->yield('switch_filter' => $request);
switch_filter => sub {
my ($kernel, $heap,$request) = @_[KERNEL, HEAP,ARG0];
unless ($heap->{client}) {
# client went away
return $kernel->yield('shutdown');
}
# check if ALL has been flushed
my $pending_bytes = $heap->{client}->get_driver_out_octets();
return $kernel->yield('switch_filter' => $request) if ($pending_bytes > 0);
$heap->{client}->set_output_filter(POE::Filter::Stream->new());
$kernel->post( "ua" => "request", "got_response", $request );
}
I do this with my upgrade of trickster, an mp3 streamer. I was
actually thinking about writing a poe http proxy component yesterday.
I ended up just using HTTP::Proxy for a windows pc on my lan. I
didn't want to give it full internet access, just a few domains of my
choosing. I may end up writing one, but I've got my hands in too many
other projects.
I'm xantus on irc.perl.org in #poe.
--
David Davis
Perl Programmer
http://teknikill.net/
On 3 Oct 2004 18:55:58 -0000, PerlDiscuss - Perl Newsgroups and
mailing lists <[EMAIL PROTECTED]> wrote:
> Hello,
>
> I am working on creating a proxy server based off the example given on the
> poe.perl.org website for a HTTP Proxy.
>
> I have tried to change it to use streaming, however, I am finding that
> often the responses are incomplete and the wheels in
> POE::Component::Client::HTTP report read errors. This problem seems to be
> particularily bad when multiple requests are being made at the same time.
> However, when I do not use streaming, everything works fine.
> Interestingly, even without streaming there are still some read errors,
> but they do not seem to affect the content returned in the response
> object. I am making this proxy for a Windows system.
>
> I would be very happy if you could quickly take a look at the code to try
> and see why this is not working. The problem may be in the way
> POE::Component::Client::HTTP handles streaming (I am using v 1.56
> 2004/07/13 18:02:37 rcaputo), but more likely its the way I am interfacing
> with-it.
>
> The only routine changed heavily is handle_http_response from the example
> off the POE website.
>
> Thanks in advance,
> George Pabst
> [EMAIL PROTECTED]
>
> #!/usr/bin/perl
>
> use warnings;
> use strict;
>
> use POE;
> use POE::Component::Server::TCP;
> use POE::Component::Client::HTTP;
> use POE::Filter::HTTPD;
>
> use HTTP::Response;
> use Compress::Zlib;
>
> sub DUMP_REQUESTS () { 0 }
> sub DUMP_RESPONSES () { 0 }
> sub LISTEN_PORT () { 8088 }
>
> ### Spawn a web client to fetch requests through.
>
> our $HTTP_VER = '1.0'; # Version of HTTP to report to servers and clients
> our $COMPRESS_TEXT = 1; # GZIP compress HTML and text
> our $CRLF = "\015\012";
> our $COMPRESS_TEXT = 0;
>
> POE::Component::Client::HTTP->spawn(Protocol => "HTTP/$HTTP_VER", Alias =>
> 'ua', Agent => 'Mozilla/4.0 (compatible;)', Streaming => 4096, FollowRedirects
> => 0);
>
> ### Spawn a web server.
>
> # The ClientInput function is called to deal with client input.
> # ClientInput's callback function will receive entire HTTP requests
> # because this server uses POE::Filter::HTTPD to parse its input.
> #
> # InlineStates let us attach our own events and handlers to a TCP
> # server. Here we attach a handler for the got_response event, which
> # will be sent to us by Client::HTTP when it has fetched something.
>
> POE::Component::Server::TCP->new
> ( Alias => "web_server",
> Port => LISTEN_PORT,
> ClientFilter => 'POE::Filter::HTTPD',
>
> ClientInput => \&handle_http_request,
> InlineStates => { got_response => \&handle_http_response, },
> );
>
> ### Run the proxy until it is done, then exit.
>
> POE::Kernel->run();
> exit 0;
>
> ### Handle HTTP requests from the client. Pass them to the HTTP
> ### client component for further processing. Optionally dump the
> ### request as text to STDOUT.
>
> sub handle_http_request {
> my ( $kernel, $heap, $request ) = @_[ KERNEL, HEAP, ARG0 ];
>
> # If the request is really a HTTP::Response, then it indicates a
> # problem parsing the client's request. Send the response back so
> # the client knows what's happened.
> if ( $request->isa("HTTP::Response") ) {
> $heap->{client}->put($request);
> $kernel->yield("shutdown");
> return;
> }
>
> # Client::HTTP doesn't support keep-alives yet.
> $request->header( "Connection", "close" );
> $request->header( "Proxy-Connection", "close" );
> $request->remove_header("Keep-Alive");
>
> display_thing( $request->as_string() ) if DUMP_REQUESTS;
> $heap->{client}->set_output_filter(POE::Filter::Stream->new() ) if
> (defined($heap->{client}));
> $kernel->post( "ua" => "request", "got_response", $request );
> }
>
> ### Handle HTTP responses from the POE::Component::Client::HTTP we've
> ### spawned at the beginning of the program. Send each response back
> ### to the client that requested it. Optionally display the response
> ### as text.
>
> sub handle_http_response{
> my ( $kernel, $heap ) = @_[ KERNEL, HEAP ];
> my $http_response = $_[ARG1]->[0];
> my $chunk = $_[ARG1]->[1];
> return if ((!$http_response) && (!$chunk));
> our($sent_headers);
> our($CRLF,$is_text);
> unless ( ($sent_headers) ) {
> $sent_headers = 1;
> if ($http_response->content_type =~ /text|html/i){
> $is_text = 1;
> print "Document is text\n";
> }
> else{
> # unless ($heap->{request}->header("X-IO-Error")){
> $http_response->protocol('HTTP/1.0');
>
> $http_response->remove_header("Content-Length") unless
> ($http_response->content_type);
>
> $heap->{client}->put($http_response->protocol . " " .
> $http_response->code . " (" . $http_response->message . ") " . $CRLF) if
> (defined($heap->{client}));
> print $http_response->code . " (" .
> $http_response->message . ") " .
> $http_response->protocol . $CRLF;
>
> $heap->{client}->put($http_response->headers_as_string($CRLF) . $CRLF) if
> (defined($heap->{client}));
> print
> $http_response->headers_as_string("\n") . "\n";
> $is_text = 0;
> # }
> # else{ print "Continuing disrupted connection for " .
> $heap->{request}->uri . "\n"; }
> }
>
> }
>
> our $chunksent;
> our ($content, $totlen);
> $totlen = 0 if (not defined $totlen);
> if ((defined($chunk) && $chunk ne '-1')){
> if (length($chunk) > 0 && $chunk ne '-1'){
> $heap->{client}->put($chunk) if ((not $is_text) &&
> defined($heap->{client}));
> print "Sent chunk of length " . length($chunk) . " bytes\n";
> $totlen += length($chunk);
> if (!defined($heap->{client})){
> $kernel->yield("shutdown");
> }
> $content .= $chunk if ($is_text == 1 && $chunk ne '-1');
> $totlen += length($chunk);
> }
> }
> else{
>
> my $clen = length($content);
> #if ($http_response->header("Content-Length") > $totlen){ # Response
> was not fully received
> # $heap->{request}->header("Range","bytes=$clen-" .
> $http_response->header("Content-Length"));
> # $heap->{request}->header("X-IO-Error",1);
> # print "Resending request with range header (Content-Length = $clen)
> " . $heap->{request}->header("Range") . " for " . $heap->{request}->uri .
> "\n";
> # $kernel->post( "streamua" => "request", "got_stream",
> $heap->{request} ); # Resubmit the request
> # return;
> #}
> $http_response->content($content);
> if ($http_response->header('Content-Encoding') =~ /gzip/i){
>
>
> $http_response->content(Compress::Zlib::memGunzip($http_response->content));
> $http_response->remove_header("Content-Encoding");
>
> }
> if ($COMPRESS_TEXT && ($http_response->code == 200) &&
> (lc($http_response->content_type) eq 'text/html') ||
> (lc($http_response->content_type) eq 'text/plain') &&
> ($heap->{request}->header("Accept-Encoding") =~ /gzip/i) &&
> ($http_response->header("Content-Encoding") !~ /gzip/i)){
> print "Length of content before gzip is " .
> length($http_response->content) . "\n";
>
>
> $http_response->content(Compress::Zlib::memGzip($http_response->content));
> $http_response->header("Content-Encoding","gzip");
> }
> if ($is_text == 1){
> $http_response->protocol('HTTP/1.0');
>
> if (($http_response->code == 200) && ($http_response->content_type)){
> use bytes ();
>
> $http_response->header('Content-Length',bytes::length($http_response->content));
> }
>
> $heap->{client}->put($http_response->protocol . " "
> . $http_response->code
> " (" . $http_response->message . ") " . $CRLF) if (defined($heap->{client}));
> print $http_response->protocol . " " .
> $http_response->code . " (" .
> $http_response->message . ") " . $CRLF;
> $heap->{client}->put($http_response->headers_as_string($CRLF) .
> $CRLF) if (defined($heap->{client}));
> print $http_response->headers_as_string($CRLF) . $CRLF;
>
> $heap->{client}->put($http_response->content) if
> (defined($heap->{client}));
> print "Length of content at end of handle_http_response is " .
> length($http_response->content), "\n";
> }
> $content = '';
> $sent_headers = 0;
> $is_text = 0;
> $totlen = 0;
> $kernel->yield("shutdown");
> }
> }
> ### Display requests and responses with brackets around them so they
> ### stand apart.
> sub display_thing {
> my $thing = shift;
> $thing =~ s/^/| /mg;
> print ",", '-' x 78, "\n";
> print $thing;
> print "`", '-' x 78, "\n";
> }
>
>
