Branch: refs/heads/smoke-me/khw-threads Home: https://github.com/Perl/perl5 Commit: b24024b161f666228c8c4bb314728376d6fb50a8 https://github.com/Perl/perl5/commit/b24024b161f666228c8c4bb314728376d6fb50a8 Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022)
Changed paths: M embed.fnc M ext/POSIX/POSIX.xs M ext/POSIX/lib/POSIX.pm M ext/POSIX/lib/POSIX.pod M locale.c M pod/perldelta.pod M proto.h Log Message: ----------- Fix POSIX::strxfrm() This function takes an SV containing a PV. The encoding of that PV is based on the locale of the LC_CTYPE locale. It really doesn't make sense to collate based off of the sequencing of a different locale, which prior to this commit it would do if the LC_COLLATION locale were different. Commit: 1d41c6fa109e66f03b4162409e3314a283639b70 https://github.com/Perl/perl5/commit/1d41c6fa109e66f03b4162409e3314a283639b70 Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M locale.c Log Message: ----------- locale.c Move some code to more appropriate place Indentation fixes will come in a later commit Commit: 3f282381a647c83e57f28b3975af45a1b0bc871f https://github.com/Perl/perl5/commit/3f282381a647c83e57f28b3975af45a1b0bc871f Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M locale.c Log Message: ----------- locale.c white-space, comment, pod, C99 only Previous commits, in trying to minimize the difference listings, have left some indentations wrong, and comments out of order. This commit rectifies that. It rewords some pod, and moves some declarations closer to first use (now that we are using C99). Commit: 9c118c070418f41aae6f77ab96b20d43fef7921d https://github.com/Perl/perl5/commit/9c118c070418f41aae6f77ab96b20d43fef7921d Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M locale.c M perl.h Log Message: ----------- XXX flesh out msg: Add STDIZED_MUTEX_LOCK Commit: 68da38f2ca1e73602f8dbc7f1399e0572ab3c2da https://github.com/Perl/perl5/commit/68da38f2ca1e73602f8dbc7f1399e0572ab3c2da Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M locale.c Log Message: ----------- use mvrtowc lock Commit: f1296f2af9ded5be14d499d3eaff36a878e00dd4 https://github.com/Perl/perl5/commit/f1296f2af9ded5be14d499d3eaff36a878e00dd4 Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M perl.h Log Message: ----------- XXX perl.h: Debugging mutex lock' Commit: 3abf04d9eda78564858708725a4966ef7d53ce83 https://github.com/Perl/perl5/commit/3abf04d9eda78564858708725a4966ef7d53ce83 Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M locale.c Log Message: ----------- wrap Commit: a728a587f5070f706c164d8356cc0f32e635d71f https://github.com/Perl/perl5/commit/a728a587f5070f706c164d8356cc0f32e635d71f Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M embed.fnc M embed.h M locale.c M proto.h Log Message: ----------- locale.c: Rmv unnecessary parameter from static function This dates from an earlier implementation Commit: 69aa0e8460de79a9b414cd9a9e6c22f57e932575 https://github.com/Perl/perl5/commit/69aa0e8460de79a9b414cd9a9e6c22f57e932575 Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M embed.fnc M locale.c M proto.h Log Message: ----------- locale.c: Reorder parameters to static function Move the least important parameters (that can be NULL to indicate unused) to the end of the parameter list, thereby moving the required ones to the beginning. This makes it clear what is important. Commit: dc45e01d3c793b0a196c01ff584ed4259c932c6b https://github.com/Perl/perl5/commit/dc45e01d3c793b0a196c01ff584ed4259c932c6b Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M locale.c M perlvars.h Log Message: ----------- locale.c: Fix memory leak PL_C_locale_obj is a global variable, and should be allocated just once per program. Prior to this commit it could leak under MULTIPLICITY. I was unable to get LSAN to notice this leak. Commit: 6a19edda5509394f4cae432ad291e058fbfc3903 https://github.com/Perl/perl5/commit/6a19edda5509394f4cae432ad291e058fbfc3903 Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M embed.fnc M proto.h M util.c Log Message: ----------- XXX memlog Commit: d833ab9be12e8e7d5194c9fb971f06eb83e172c0 https://github.com/Perl/perl5/commit/d833ab9be12e8e7d5194c9fb971f06eb83e172c0 Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M README.cygwin Log Message: ----------- README.cygwin: Note parallel compiling/testing is available Commit: 95919dda01d9fd0d5695667673ec9e1634e8eeca https://github.com/Perl/perl5/commit/95919dda01d9fd0d5695667673ec9e1634e8eeca Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M util.c Log Message: ----------- util.c: Remove unnecessary #ifdef putenv now always expands to PelEnv_putenv; no need to explicitly state so. Commit: 2c43c780b8be71c4b3ea6c216f14691adb191bfd https://github.com/Perl/perl5/commit/2c43c780b8be71c4b3ea6c216f14691adb191bfd Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M t/loc_tools.pl Log Message: ----------- loc_tools.pl: Add sub to classify locales UTF-8 vs non The new sub taks a list of locales available on the platform and separates the UTF-8 ones from the non-UTF-8 ones, returning two sub-lists Commit: cb62ef01a62e7c16c1fd879983d3f66b09f1027b https://github.com/Perl/perl5/commit/cb62ef01a62e7c16c1fd879983d3f66b09f1027b Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M ext/POSIX/t/posix.t M locale.c Log Message: ----------- POSIX::localeconv: Return empty values This function returns a hash allowing Perl access to the localeconv() data structure, with the keys being the structure's field names, and the values being their corresponding value in the current locale. Prior to this commit, it did not populate the hash with any keys whose values are the empty string. But this is wrong. Those fields exist, and are explicitly allowed to be empty, except for the decimal point. For example, the symbol indicating a number is positive is empty in many locales. Someone might want to look up that value, and discover it is undefined. Or they might want to iterate over all the keys in the hash. I couldn't find a reason in the history why these are omitted, and it seems to me to be wrong to omit them. Commit: debc51b957fd9eb9f395d3b385c9df542580b592 https://github.com/Perl/perl5/commit/debc51b957fd9eb9f395d3b385c9df542580b592 Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M t/loc_tools.pl Log Message: ----------- loc_tools.pl: Add ability to skip known bad locales Some platforms have locales that shouldn't be used. This adds code to avoid using such when looking at all the locales on a platform. The next commit will add the first use. Commit: fd7e3639eb5a601bf58445f282b05a74a9a99244 https://github.com/Perl/perl5/commit/fd7e3639eb5a601bf58445f282b05a74a9a99244 Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M t/loc_tools.pl Log Message: ----------- Don't use Vietnamese locale on Solaris It immediately causes a segfault Commit: c6d709ec64d83c7cfeae5945f898dad8ac5c671d https://github.com/Perl/perl5/commit/c6d709ec64d83c7cfeae5945f898dad8ac5c671d Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M t/loc_tools.pl Log Message: ----------- loc_tools.pl: Accept dashless UTF8 besides to 'UTF-8' The dash is sometimes omitted Commit: 66c746976a5dd5094bb76952dbffb13aac6d1d08 https://github.com/Perl/perl5/commit/66c746976a5dd5094bb76952dbffb13aac6d1d08 Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M t/loc_tools.pl Log Message: ----------- loc_tools.pl: Always do normalized locale name check Because of variances in capitilization and use of dashes, we should only compare locale names that have been normalized to a common syntax. This was the remaining outlier, and caused issues on some platforms. Commit: 8d3ebdfa1ce78d9c8c5cd47159ab50d4c2f5ea5c https://github.com/Perl/perl5/commit/8d3ebdfa1ce78d9c8c5cd47159ab50d4c2f5ea5c Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M locale.c Log Message: ----------- locale.c: Prepare get_locale_string_utf8ness_i for no locales This changes this function so that when compiled without using locales, it still works: returning that anything passed to it isn't UTF-8. This is trivailly true. Commit: 0c56472717ab9bb70e420fbff92f868e3f090033 https://github.com/Perl/perl5/commit/0c56472717ab9bb70e420fbff92f868e3f090033 Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M locale.c Log Message: ----------- locale.c: Create #define for a particular #if combo This takes this #ifdef combination and creates a single symbol that means essentially the same thing. This is in preparation for it being used in additional places. And it makes it clearer what is meant. It also adds another condition which isn't currently tested for, but has the same meaning, and doesn't affect the current usage, but will future ones. Commit: 6f73953eeedc4fb39a3b68047e0f949449b2a54c https://github.com/Perl/perl5/commit/6f73953eeedc4fb39a3b68047e0f949449b2a54c Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M embed.fnc M embed.h M locale.c M proto.h Log Message: ----------- locale.c: Move 2 functions elsewhere in the code This is in preparation for them to be called on platforms where locale handling is not enabled. Commit: f2326bb9d8f79ffd3b0c9f567c90ae811c0757ed https://github.com/Perl/perl5/commit/f2326bb9d8f79ffd3b0c9f567c90ae811c0757ed Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M locale.c Log Message: ----------- locale.c: Remove use of localeconv_l() Use plain localeconv() always. When I wrote this code several years ago, I was under the impression that localeconv_l() was thread-safe. Alas, stress testing on various platforms indicate that it isn't, nor do the current man pages I found indicate it is. I don't now know how I came to that (mistaken) belief. The reason to use localeconv_l() was that no locking was used. But now that I know locking is required anyway, that advantage is gone, and it is just a separate code path that would need to be retained. Commit: d423f4059c09562eb6bd780f9f66759c08f4e282 https://github.com/Perl/perl5/commit/d423f4059c09562eb6bd780f9f66759c08f4e282 Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M locale.c Log Message: ----------- locale.c: Save to proper buffer This was using a particular buffer to save to, rather than the one it was supposed to, passed into the function. This bug only manifests itself in Windows Visual Studio prior to 2015, and to platforms (unlikely to exist) without the localeconv() system call. Commit: 93b8af2127420189bda81282d7f733eda014cbe8 https://github.com/Perl/perl5/commit/93b8af2127420189bda81282d7f733eda014cbe8 Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M embed.fnc M embed.h M locale.c M perl.h M proto.h Log Message: ----------- locale.c: Rewrite localeconv() handling localeconv() returns a structure contaiing fields that are associated with two different categories: LC_NUMERIC and LC_MONETARY. Perl via POSIX::localeconv() reutrns a hash containing all the fields. Testing on Windows showed that if LC_CTYPE is not the same locale as LC_MONETARY for the monetary fields, or isn't the same as LC_NUMERIC for the numeric ones, mojibake can result. The solution to similar situations elsewhere in the code is to toggle LC_CTYPE into being the same locale as the one for the returned fields. But those situations only have a single locale that LC_CTYPE has to match, so it doesn't work here when LC_NUMERIC and LC_MONETARY are different locales. Unlike Schrödinger's cat, LC_CTYPE has to be one or the other, not both at the same time. The previous implementation did not consider this possibility, and wasn't easily changeable to work. Therefore, this rewrites a bunch of it. The solution used is to call localeconv() twice, once for the LC_NUMERIC locale (with LC_CTYPE corresponding), and once for LC_MONETARY. If those locales are the same, the second call is collapsed into the first one, resulting in just one call. This likely scenario complicated the code so as to avoid unnecessary calls. Another complication is that on platforms that lack nl_langinfo(), (Windows, for example), localeconv() is used to emulate portions of it. Previously there was a separate function to handle this, using an SV() cast as an HV() to avoid using a hash that wasn't actually necessary. That proved to lead to extra duplicated code under the new scheme, so that function was collapsed into a single one and a real hash is used in all circumstances, but is only populated with the one or two fields needed for the emulation. The only part of this commit that I thought could be split off from the rest concerns the fact that localeconv()'s return is not thread-safe, and so must be copied to a safe place (the hash) while in a critical section, locking out all other threads. Before this commit, that copying was accompanied by determining if each string field needed to be marked as UTF-8. That determination isn't necessarily trivial, and in fact can be deferred until out of the critical section. This commit does that. And, with some effort, that part could have been split into a separate commit. but I didn't think it was worth the effort. Commit: 2173442d84db6274df0c78649b8a9c4acb66ea6b https://github.com/Perl/perl5/commit/2173442d84db6274df0c78649b8a9c4acb66ea6b Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M locale.c Log Message: ----------- XXX win workaround Commit: 6987dacb3ab1342576041b587e6341bcb050cb58 https://github.com/Perl/perl5/commit/6987dacb3ab1342576041b587e6341bcb050cb58 Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M embed.fnc M embed.h M locale.c M proto.h Log Message: ----------- XXX querylocale Commit: c68a4713cc97226da09f8b167fb41b1e2a369a93 https://github.com/Perl/perl5/commit/c68a4713cc97226da09f8b167fb41b1e2a369a93 Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M locale.c Log Message: ----------- locale.c: Silence compiler warning This as only emitted on Windows VS before 2015. Commit: 4655003ad160f558e42ac3b0bef61c65c2f99aa3 https://github.com/Perl/perl5/commit/4655003ad160f558e42ac3b0bef61c65c2f99aa3 Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M locale.c Log Message: ----------- locale.c: If not compiling locales, can't have UTF-8 ones If Perl is compiled to not pay attention at all to locales, everything effectively becomes the C locale, which isn't a UTF-8 one. So we know that at compile time. Commit: 6f0aeb51ffddbbc06c3c7615883a01600fd6c717 https://github.com/Perl/perl5/commit/6f0aeb51ffddbbc06c3c7615883a01600fd6c717 Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M lib/locale_threads.t Log Message: ----------- locale_threads Commit: 170758183e22c426c1a2609b1da65099bfcf434d https://github.com/Perl/perl5/commit/170758183e22c426c1a2609b1da65099bfcf434d Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M dist/threads/lib/threads.pm M dist/threads/threads.xs M perl.h Log Message: ----------- XXX threads NON_tTHX leak maybe fix breakage Commit: f53d71e876c5fc2eb5f2caa7b20362c113bd2a90 https://github.com/Perl/perl5/commit/f53d71e876c5fc2eb5f2caa7b20362c113bd2a90 Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M perl.h Log Message: ----------- help Commit: 6c690cc51da4b63137aeff4148d1432ebb4326aa https://github.com/Perl/perl5/commit/6c690cc51da4b63137aeff4148d1432ebb4326aa Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M hints/darwin.sh Log Message: ----------- no langinfo on darwin Commit: c85e5d3422d13aa5cdca94da19689356c690d57c https://github.com/Perl/perl5/commit/c85e5d3422d13aa5cdca94da19689356c690d57c Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M lib/locale_threads.t Log Message: ----------- locale_threads Commit: 9cd1b212c323ebffb2ee853b630b97d1ed50b8ea https://github.com/Perl/perl5/commit/9cd1b212c323ebffb2ee853b630b97d1ed50b8ea Author: Karl Williamson <k...@cpan.org> Date: 2022-11-28 (Mon, 28 Nov 2022) Changed paths: M lib/locale_threads.t Log Message: ----------- sleep Compare: https://github.com/Perl/perl5/compare/c28ecf13bcfa...9cd1b212c323