Control: retitle -1 lintian: UTF-8 filenames cause internal errors
Hi Dmitry,
On Wed, Apr 8, 2020 at 10:30 AM Dmitry Shachnev wrote:
>
> Can't open
> '/tmp/lintian-pool-z2LddsoVG9/pool/s/sphinx/sphinx_2.4.3-2+salsaci_source/unpacked/tests/roots/test-images/testimäge.png'
> with mode '<:raw': 'No such file or directory' at
> /usr/share/lintian/checks/cruft.pm line 992
This error is about UTF-8 filenames. We have a solution in Lintian,
but it does not work reliably due to a Perl bug.
In my opinion, the bug is serious enough to patch all versions of Perl
shipped in Debian, as explained below. In lieu of what would have been
an upcoming email to debian-perl, they are hereby copied.
In a nutshell, file tests such as '-f' do not work reliably with
strings that were internally "upgraded" by Perl (i.e. utf8::upgrade).
More details about this dangerous bug are available here [1] and [2].
Other system calls such as 'stat' and 'open' are likewise affected.
This bug shows the problem with 'open'.
It is amazing that the bug has been open for so long. It is literally
the well-known "Unicode bug", but for file names. Apparently there is
no common solution for all Perl platforms. (The use of UTF-8 filenames
also appears to be uncommon outside Linux, or is implemented
differently, i.e. MacOS.)
In my view, many Perl scripts in Debian, including those for
installation and security purposes, depend on file tests working
reliably. Perhaps our Perl interpreters should be patched with a
Debian-specific fix.
Related questions about file name mangling or matching in modules,
such as Path::Tiny and File::Find::Rule, may have to be explored or
mitigated before this bug can be considered completely closed.
The credit for identifying the bug in Lintian (and saving my sanity)
goes to Grinnz on #debian-perl.
Kind regards,
Felix Lechner
[1] https://github.com/Perl/perl5/issues/10550
[2] https://github.com/Perl/perl5/issues/9674
