Bug#869240: ikiwiki: decoding of strftime() result is not needed

[Alexander Galanin]

23.07.2017 18:10, Simon McVittie пишет:
> Control: tags 869240 + pending
> 
> On Fri, 21 Jul 2017 at 23:48:36 +0300, Alexander Galanin wrote:
>>> Cannot decode string with wide characters at 
>>> /usr/lib/x86_64-linux-gnu/perl/5.24/Encode.pm line 202.
>>
>> I think that the real problem is double-decoding of strftime() result
>> that is already a proper widestring. See IkiWiki.pm at line 1300:
>>
>>> Encode::decode($strftime_encoding, POSIX::strftime(@_))
> 
> This decoding used to be necessary 
> ()
> so we probably can't just drop it. It appears to have become (at least
> conditionally) unnecessary in Perl 5.21.1, commit
> .
> 
> What we *can* do is to test the result of calling Encode::is_utf8() on
> the result, and only decode it if it isn't already a wide string, like
> safe_decode_utf8 in IkiWiki::CGI. Done in
> .
> Please try that patch; I did a quick test in the ru_RU.utf8 locale, and
> it seemed to fail as you describe without the patch but work with it.

Works for me.

-- 
Alexander Galanin



signature.asc
Description: OpenPGP digital signature

Bug#869240: ikiwiki: decoding of strftime() result is not needed

[Simon McVittie]

Control: tags 869240 + pending

On Fri, 21 Jul 2017 at 23:48:36 +0300, Alexander Galanin wrote:
> > Cannot decode string with wide characters at 
> > /usr/lib/x86_64-linux-gnu/perl/5.24/Encode.pm line 202.
> 
> I think that the real problem is double-decoding of strftime() result
> that is already a proper widestring. See IkiWiki.pm at line 1300:
> 
> > Encode::decode($strftime_encoding, POSIX::strftime(@_))

This decoding used to be necessary ()
so we probably can't just drop it. It appears to have become (at least
conditionally) unnecessary in Perl 5.21.1, commit
.

What we *can* do is to test the result of calling Encode::is_utf8() on
the result, and only decode it if it isn't already a wide string, like
safe_decode_utf8 in IkiWiki::CGI. Done in
.
Please try that patch; I did a quick test in the ru_RU.utf8 locale, and
it seemed to fail as you describe without the patch but work with it.

Regards,
S

Bug#869240: ikiwiki: decoding of strftime() result is not needed

[Alexander Galanin]

Package: ikiwiki
Version: 3.20170111
Severity: important
Tags: patch l10n

I am using ru_RU.UTF-8 locale. Ikiwiki build is failed with the
following error:

> Cannot decode string with wide characters at 
> /usr/lib/x86_64-linux-gnu/perl/5.24/Encode.pm line 202.

I think that the real problem is double-decoding of strftime() result
that is already a proper widestring. See IkiWiki.pm at line 1300:

> Encode::decode($strftime_encoding, POSIX::strftime(@_))

The obvious fix:

--- a/IkiWiki.pm 2017-01-11 00:00:00.0 +0300
+++ b/IkiWiki.pm 2017-07-21 23:47:24.017845918 +0300
@@ -1286,19 +1286,7 @@
$format=$config{timeformat};
}
 
-   return strftime_utf8($format, localtime($time));
-}
-
-my $strftime_encoding;
-sub strftime_utf8 {
-   # strftime doesn't know about encodings, so make sure
-   # its output is properly treated as utf8.
-   # Note that this does not handle utf-8 in the format string.
-   ($strftime_encoding) = POSIX::setlocale(&POSIX::LC_TIME) =~ m#\.([^@]+)#
-   unless defined $strftime_encoding;
-   $strftime_encoding
-   ? Encode::decode($strftime_encoding, POSIX::strftime(@_))
-   : POSIX::strftime(@_);
+   return POSIX::strftime($format, localtime($time));
 }
 
 sub date_3339 ($) {


-- System Information:
Debian Release: 9.0
  APT prefers stable
  APT policy: (990, 'stable'), (650, 'testing'), (500, 'oldstable-updates'), 
(500, 'oldstable')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.9.0-3-amd64 (SMP w/2 CPU cores)
Locale: LANG=ru_RU.UTF-8, LC_CTYPE=ru_RU.UTF-8 (charmap=UTF-8), 
LANGUAGE=ru_RU.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: sysvinit (via /sbin/init)

Versions of packages ikiwiki depends on:
ii  libhtml-parser-perl 3.72-3
ii  libhtml-scrubber-perl   0.15-1
ii  libhtml-template-perl   2.95-2
ii  libjson-perl2.90-1
ii  libtext-markdown-discount-perl  0.11-1+b3
ii  liburi-perl 1.71-1
ii  libyaml-libyaml-perl0.63-2
ii  perl5.24.1-3

Versions of packages ikiwiki recommends:
ii  gcc [c-compiler] 4:6.3.0-4
ii  gcc-6 [c-compiler]   6.3.0-18
ii  git [git-core]   1:2.11.0-3
pn  libauthen-passphrase-perl
ii  libc6-dev [libc-dev] 2.24-11+deb9u1
ii  libcgi-formbuilder-perl  3.10-1
ii  libcgi-pm-perl   4.35-1
ii  libcgi-session-perl  4.48-3
ii  libcrypt-ssleay-perl 0.73.04-2
pn  libgravatar-url-perl 
pn  liblwpx-paranoidagent-perl   
pn  libmail-sendmail-perl
pn  libnet-openid-consumer-perl  
pn  librpc-xml-perl  
ii  libterm-readline-gnu-perl1.35-1
ii  libtimedate-perl 2.3000-2
ii  libxml-simple-perl   2.22-1
ii  mercurial4.0-1
ii  subversion   1.9.5-1

Versions of packages ikiwiki suggests:
ii  dvipng 1.14-2+b3
ii  file   1:5.30-1
ii  gettext0.19.8.1-2
ii  ghostscript9.20~dfsg-3.2
ii  graphviz   2.38.0-17
pn  libfile-mimeinfo-perl  
ii  libhighlight-perl  3.18-3+b5
ii  libhtml-tree-perl  5.03-2
ii  libimage-magick-perl [perlmagick]  8:6.9.7.4+dfsg-11+deb9u1
ii  liblocale-gettext-perl 1.07-3+b1
pn  libmagickcore-extra
pn  libmailtools-perl  
pn  libnet-amazon-s3-perl  
pn  libnet-inet6glue-perl  
pn  libsearch-xapian-perl  
ii  libsort-naturally-perl 1.03-1
pn  libsparkline-php   
ii  libtext-csv-perl   1.33-2
ii  libtext-multimarkdown-perl 1.35-1
ii  libtext-textile-perl   2.13-2
pn  libtext-typography-perl
pn  libtext-wikicreole-perl
ii  libtext-wikiformat-perl0.79-1
pn  libxml-feed-perl   
ii  libxml-writer-perl 0.625-1
pn  po4a   
pn  polygen
ii  python 2.7.13-2
ii  python-docutils0.13.1+dfsg-2
ii  texlive2016.20170123-5
ii  tidy   1:5.2.0-2
pn  viewvc | gitweb | viewcvs  
pn  xapian-omega   

-- no debconf information