On 1/5/07, Bill Moseley <[EMAIL PROTECTED]> wrote:
$response->headers->as_string shows this:

    [...]
    Content-Length: 17604
    Content-Type: text/html
    Content-Type: text/html;charset=UTF-8
    Last-Modified: Mon, 23 Oct 2006 14:32:14 GMT
    [...]

Is there a better way to grab the charset than this?

    my $charset;
    for ( $response->header('content-type') ){
        $charset = $1 if /\bcharset=([^;]+)/;
    }
    my $content = Encode::decode( $charset, $response->content )
        if $charset;

LWP already provide a method to decode the content for you:

  my $content = $response->decoded_content;

This method will also undo various Content-Encodings for you.  If you
look at the source for decoded_content you will notice that it parse
the Content-Type header using:

       if (my @ct =
HTTP::Headers::Util::split_header_words($self->header("Content-Type")))
{
           ($ct, undef, %ct_param) = @{$ct[-1]};
           $ct = lc($ct);
       }

I'm assuming that the last Content-type header found is the charset to
use.

The code I quoted from decoded_content picks the last one.


    my ($ct, $other) = $response->content_type;

is only returning the first Content-Type header.


How do other's decode content fetched with LWP?  Do you parse out
charset and then decode it like above?


Doesn't that seem like something that should be part of LWP (or some
sub-class)?

    use LWP::Simple;
    $content = get( $url );

Shouldn't $content be decoded there?

That would probably be an improvement, but it might also break some
code that depends on just receving the raw binary content asis.

--
Gisle Aas

Reply via email to