* Dominic Mitchell <[EMAIL PROTECTED]> [12/09/04 01:53]:
> Moshe Kaminsky wrote:
> >It appears that HTML::Parser modifies some unicode characters while
> >parsing. The following program gives an example:
> >
> >#########
> >
> >#!/usr/bin/perl
> >use HTML::Parser;
> >use utf8;
> >open TEST, '>:utf8', 'word.txt';
> >my $p = new HTML::Parser text_h => [sub {print TEST shift}, 'text'];
> >$p->parse("zespoÅÃw\n");
> >close TEST;
> >
> >#########
> >
> >After running it, 'word.txt' contains "zespoÅÃw" with the funny l and
> >the funny o following it transformed to something else. What am I doing
> >wrong?
> >I'm running: perl 5.8.5, HTML::Parser version 3.36 on linux.
>
> It looks like HTML::Parser is losing the UTF-8 flag. XS modules have a
> nasty tendency to do this. :(
>
> Thankfully the workaround is fairly simple. Add "use Encode" to the top
> of the script, and change the callback slightly:
>
> sub { print TEST decode_utf8(shift) }
>
> seems to work ok here.
Thanks! That actually works. However, my real situation is that I'm
using HTML::FormatText, which uses, eventually, HTML::TreeBuilder and
HTML::Parser. So to fix the problem, it appears that the only way is to
modify HTML::TreeBuilder? I wonder if the maintainers of HTML::Parser
are aware of this problem, and if so, why don't they do this
automatically (or at least add an option to do it automatically) before
giving the text to the handler?
Anyway, thanks again.
Moshe
>
> -Dom
>
--
I love deadlines. I like the whooshing sound they make as they fly by.
-- Douglas Adams
Moshe Kaminsky <[EMAIL PROTECTED]>
Home: 08-9456841
pgpnQBBkpJSCH.pgp
Description: PGP signature
