Francis Turner wrote:
>
> David Busby wrote:
> > 
> >     I'm having trouble with the following:
> > 
> > 
> > my $tf = HTML::TagFilter->new(strip_comments => 1);
> > $tf->allow_tags(undef);
> > my $clean_html = $tf->filter($res->content);
> > 
> > But now my $clean_html has soooo many line feeds, tabs and spaces, wasted
> > space.
> > I don't know enough regex to make it remove all this extra whitespace.
> > I basically want all white space thats not a single space to be removed.
> > Samples please?
> > 
> A one line output with all whitespace (including linefeeds)  converted 
> to spaces is
> $clean_html =~ s/\s+/ /sg;
> 
> I use this a lot but it may be removing too much. in particulalr you may 
> want the linebreaks.
> 
> A more advanced trick is to
> 1) replace \n with something unlikely to be in the doc
> local $_ = $clean_html;
> my $n=chr(254); # unlikely to be present - you can check with /$n/
> s/\n/$n/sg;
> 2) replace all \s+ with spaces
> s/\s+/ /sg;
> 3) recover the \ns
> s/$n/\n/sg;
> 4) replace \n\s*\n with \n to remove blank lines
> s/\n\s*\n/\n/sg;
> $clean_html = $_;
> 
> I have not actually run this precise code so it may have some error in it.

A neater way to substiture all mutltiple whitespace that isn't a
newline is to form a character class out of all non-whitespace
characters plus newline and then negate it:

  s/[^\S\n]+/ /g

It may also be useful to collapse multiple newlines afterwards:

  s/\n{2,}/\n/g

Cheers,

Rob

Reply via email to