Branch: refs/heads/blead
Home: https://github.com/Perl/perl5
Commit: f101e19ae74be4d1d912e4d176a6e7a305bee770
https://github.com/Perl/perl5/commit/f101e19ae74be4d1d912e4d176a6e7a305bee770
Author: Karl Williamson <k...@cpan.org>
Date: 2022-11-29 (Tue, 29 Nov 2022)
Changed paths:
M embed.fnc
M ext/POSIX/POSIX.xs
M ext/POSIX/lib/POSIX.pm
M ext/POSIX/lib/POSIX.pod
M locale.c
M pod/perldelta.pod
M proto.h
Log Message:
-----------
Fix POSIX::strxfrm()
This commit does two things.
Most simply it extends strxfrm() to handle strings containing NUL
characters. Previously the transformation stopped at the first NUL
encountered.
Second, it combines the implementation of this with the existing
implementation used for the 'cmp' operator, eliminating existing
discrepancies and preventing future ones.
This function takes an SV containing a PV. The encoding of that
PV is based on the locale of the LC_CTYPE locale. It really doesn't
make sense to collate based off of the sequencing of a different locale,
which prior to this commit it would do (but not for 'cmp') if the
LC_COLLATION locale were different.
As an example, consider the string:
my $string = quotemeta join "", map { chr } (1..255);
and with LC_TYPE=8859-1 (Latin-1, used for several Western European
languages), LC_COLLATE set to ja_JP.utf8. This doesn't make much sense,
outside of specialty uses such as a lazy implementation of a
Japanese/French dictionary, or for quoting snippets in one language in a
document written in the other. ('lazy' because such text should really
be changing locales to the language of the snippet currently being
worked on.) Nevertheless Perl should do something as sensible as
possible. and this commit changes POSIX::strxfrm() to use the method
already in use by the code implementing 'cmp'. Prior to this commit,
POSIX::strxfrm($string) yielded on glibc 12.1:
^\3^\4^\5^\6^\a^\b^\t^\n^\13^\f^\r^\16^\17^\20^\21^\22^\23^\24^\25^\26^\27^\30^\31^\32^\e^\34^\35^\36^\37^
^!^\"^#^\$^%^&^'^(^)^*^+^,^-^.^/^0^123456789:;^<^=^>^?^\@^A^BCDEFGHIJKLMNOPQRSTUVWXYZ[\\^]^^^_^`a^bcdefghijklmnopqrstuvwxyz{|^}^~^\177^\302\200^\302\201^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3^\3
These are effectively a sorting order, and it is not meant to be human
understandable. But it is clear that most of the characters had the
same weight of 3, so a libc sort would mark them as ties in sorting
order.
And after,
^\3^\4^\5^\6^\a^\b^\t^\n^\13^\f^\r^\16^\17^\20^\21^\22^\23^\24^\25^\26^\27^\30^\31^\32^\e^\34^\35^\36^\37^
^!^\"^#^\$^%^&^'^(^)^*^+^,^-^.^/^0^123456789:;^<^=^>^?^\@^A^BCDEFGHIJKLMNOPQRSTUVWXYZ[\\^]^^^_^`a^bcdefghijklmnopqrstuvwxyz{|^}^~^\177^\302\200^\302\201^\302\202^\302\203^\302\204^\302\205^\302\206^\302\207^\302\210^\302\211^\302\212^\302\213^\302\214^\302\215^\302\216^\302\217^\3\3^\3\3^\302\220^\302\221^\302\222^\302\223^\302\224^\302\225^\302\226^\302\227^\302\230^\302\231^\302\232^\302\233^\302\234^\302\235^\302\236^\302\237^\3\3^\341\257\211^\304\257^\304\260^\341\257\221^\3\3^\341\257\212^\304\266^\303\255^\341\257\216^\341\257\215^\3\3^\305\225^\3\3^\341\257\217^\341\257\203^\304\251^\304\234^\3\3^\3\3^\303\253^\3\3^\305\260^\3\3^\341\257\200^\3\3^\341\257\214^\3\3^\3\3^\3\3^\3\3^\341\257\213^\341\260\236^\341\260\235^\341\260\240^\341\260\246^\341\260\237^\341\260\245^\341\260\202^\341\260\252^\341\260\256^\341\260\255^\341\260\260^\341\260\257^\341\260\273^\341\260\272^\341\260\275^\341\260\274^\3\3^\341\261\213^\341\261\215^\341\261\214^\341\261\217^\341\261\223^\341\261\216^\304\235^\341\260\211^\341\261\236^\341\261\235^\341\261\240^\341\261\237^\341\261\255^\341\260\214^\341\260\232^\341\261\264^\341\261\263^\341\261\266^\341\261\274^\341\261\265^\341\261\273^\341\260\215^\341\262\200^\341\262\204^\341\262\203^\341\262\206^\341\262\205^\341\262\221^\341\262\220^\341\262\223^\341\262\222^\341\260\217^\341\262\240^\341\262\242^\341\262\241^\341\262\244^\341\262\250^\341\262\243^\304\236^\341\260\230^\341\262\263^\341\262\262^\341\262\265^\341\262\264^\341\263\202^\341\260\234^\341\263\203
which shows that most of the ties have been resolved, and hence the
results are more sensible
