Bug#959474: Issues with Chinese language (all variants) when building some pages in buster

2021-01-27 Thread Changwoo Ryu
Korean is affected too and I added the "-O1" option workaround also to Korean.

Bug#959474: Issues with Chinese language (all variants) when building some pages in buster

2020-06-09 Thread Boyuan Yang
在 2020-06-07星期日的 21:23 +0200,Laura Arjona Reina写道: > Hi > > El 7/6/20 a las 16:02, Axel Beckert escribió: > > > Just ot be sure: I should still provide a stable update for buster, > > right? > > > > I don't know if the type of bug qualifies for a stable update. If I were the maintainer, I woul

Bug#959474: Issues with Chinese language (all variants) when building some pages in buster

2020-06-07 Thread Laura Arjona Reina
Hi El 7/6/20 a las 16:02, Axel Beckert escribió: > Just ot be sure: I should still provide a stable update for buster, > right? > I don't know if the type of bug qualifies for a stable update. For www.debian.org, we'll be using the -O1 workaround for building the Chinese pages, and that's abou

Bug#959474: Issues with Chinese language (all variants) when building some pages in buster

2020-06-07 Thread Axel Beckert
Hi, Laura Arjona Reina wrote: > I have compared the results of builds in stretch and buster both with > and without the option, and there are no changes in stretch, and the > UTF-8 issues are fixed in buster with the option Thanks for these tests. > So, I think that Bug#959474 can be closed, but

Bug#959474: Issues with Chinese language (all variants) when building some pages in buster

2020-06-07 Thread Laura Arjona Reina
Hi all As a workaround for the Debian website, until wml 2.12.2~ds1-3 or higher arrives to stable, I have added the option "-O1" to the options passed to wml for Chinese, in the /chinese/Make.lang file: +# Add "-O1" to wml to be passed to htmlstrip, to avoid malformed UTF-8 +# see bug #959474 +

Bug#959474: Bug#959761: Bug#959474: Issues with Chinese language (all variants) when building some pages in buster

2020-05-05 Thread gregor herrmann
On Tue, 05 May 2020 10:53:29 +0200, Axel Beckert wrote: > > Perhaps the strings in wml need to be decoded from UTF-8 so that they > > aren't treated as a sequence of independent bytes? > ... and would have expect "use feature unicode_strings;" already > activates all of this. (I haven't read the

Bug#959474: Bug#959761: Bug#959474: Issues with Chinese language (all variants) when building some pages in buster

2020-05-05 Thread Axel Beckert
Hi Damyan, Damyan Ivanov wrote: > (not a Perl maintainer here) Did help nevertheless. Just didn't want to spam the whole Perl Team with potential Perl bugs. ;-) > -=| Axel Beckert, 05.05.2020 03:34:28 +0200 |=- > > → echo 包 | perl -pe 's|\s+\n|\n|sg;' > > 包 > > → echo 包 | perl -M"feature unicode

Bug#959474: Issues with Chinese language (all variants) when building some pages in buster

2020-05-05 Thread Jakub Wilk
* Damyan Ivanov , 2020-05-05, 08:45: Strangely, replacing -CS with a call to STDIN->binmode("UTF-8") doesn't help: echo 包 | perl -E 'STDIN->binmode("UTF-8"); while(<>) { s|\s+\n|\n|sg; print }' � That's because "UTF-8" is not a valid argument for binmode(). You want: $ echo 包 | perl -E 'ST

Bug#959474: Issues with Chinese language (all variants) when building some pages in buster

2020-05-04 Thread Damyan Ivanov
(not a Perl maintainer here) -=| Axel Beckert, 05.05.2020 03:34:28 +0200 |=- > → echo 包 | perl -pe 's|\s+\n|\n|sg;' > 包 > → echo 包 | perl -M"feature unicode_strings" -pe 's|\s+\n|\n|sg;' > � > > Which kinda sounds like a Perl bug. Cc'ing the maintainers of Debian's > perl package (not the whole D

Bug#959474: Issues with Chinese language (all variants) when building some pages in buster

2020-05-04 Thread Yao Wei
On Mon, May 04, 2020 at 10:19:02PM -0400, Boyuan Yang wrote: > Mwei (https://nm.debian.org/person/mwei/) just talked to me saying that it > could be a bug with isSPACE_L1 macro in perl's pp.c. He will be replying the > email soon. > Hi, (I used reportbug to handle reply of this thread, and I mis

Bug#959474: Issues with Chinese language (all variants) when building some pages in buster

2020-05-04 Thread Boyuan Yang
Hi, 在 2020-05-05星期二的 03:34 +0200,Axel Beckert写道: > → echo 包 | perl -pe 's|\s+\n|\n|sg;' > 包 > → echo 包 | perl -M"feature unicode_strings" -pe 's|\s+\n|\n|sg;' > � > > Which kinda sounds like a Perl bug. Cc'ing the maintainers of Debian's > perl package (not the whole Debian Perl Team), maybe they

Bug#959474: Issues with Chinese language (all variants) when building some pages in buster

2020-05-04 Thread Yao Wei (魏銘廷)
Package: www.debian.org Followup-For: Bug #959474 Hi, After a bit of investigation of Perl source code (5.31.11 downloaded from upstream) I found the they have weird handling of whitespace when `feature unicode_strings` turned on. I am not a perl person and I haven't executed the source code yet

Bug#959474: Issues with Chinese language (all variants) when building some pages in buster

2020-05-04 Thread Axel Beckert
Hi, found the culprit quicker than expected. I'm though no more sure if it's really a WML issue or if sits even deeper: Axel Beckert wrote: > → echo 包 | /usr/share/wml/exec/wml_p8_htmlstrip -O 1 > 包 > → echo 包 | /usr/share/wml/exec/wml_p8_htmlstrip -O 2 > � Level 2 actually only consists of thes

Bug#959474: Issues with Chinese language (all variants) when building some pages in buster

2020-05-04 Thread Axel Beckert
Control: clone -1 -2 Control: reasign -2 wml 2.12.2~ds1-2 Control: retitle -2 wml: Regression in "htmlstrip -O2" (default) with Chinese language Hi, Boyuan Yang wrote: > Thanks for raising this issue. Thanks from me, too. I wasn't aware of such a regression, sorry. > These build errors might h

Bug#959474: Issues with Chinese language (all variants) when building some pages in buster

2020-05-04 Thread Boyuan Yang
Hi all, (with my Debian Chinese Team hat on) (see bottom...) 在 2020-05-03星期日的 22:57 +0200,Holger Wansing写道: > Hi, > > Laura Arjona Reina wrote: > > There are some issues with some Chinese pages when they are built in a > > buster machine. > > We need to fix those issues (at least the "Malforme

Bug#959474: Issues with Chinese language (all variants) when building some pages in buster

2020-05-03 Thread Holger Wansing
Hi, Laura Arjona Reina wrote: > There are some issues with some Chinese pages when they are built in a > buster machine. > We need to fix those issues (at least the "Malformed UTF-8 character > [...] at ../../bin/tocn.pl [...]" ones) so DSA can upgrade the > www-master machine to buster. See the

Bug#959474: Issues with Chinese language (all variants) when building some pages in buster

2020-05-02 Thread Laura Arjona Reina
Package: www.debian.org Severity: normal User: www.debian@packages.debian.org Usertags: scripts X-Debbugs-CC: debian-l10n-chin...@lists.debian.org X-Debbugs-CC: debian-i...@lists.debian.org Hi all, TL;DR There are some issues with some Chinese pages when they are built in a buster machine. W