On Jul 31 16:06, Bruno Haible via Cygwin wrote: > Corinna Vinschen wrote: > > I have a problem with the c32isalpha function. > > > > c32isalpha fails for the character U+FF11 FULLWIDTH DIGIT ONE, > > because it expects the character to be an alphabetic character. > > This is not a big problem. You can see in the test-c32isalpha.c file > that this test is disabled for many platforms, in particular glibc.
Which is interesting, because I actually tried that today on glibc, and for iswalpha (0xff11) it returns 1. So it actually behaves as the testcase expects. > There's no problem with disabling it on Cygwin as well. I'd rather make Cygwin do the same as glibc. > > The Cygwin unicode information is automatically generated from the > > Unicode data file UnicodeData.txt, fresh from their homepage. iswalpha > > in newlib is checking for the Unicode categories, using the expression: > > > > return cat == CAT_LC || cat == CAT_Lu || cat == CAT_Ll || cat == CAT_Lt > > || cat == CAT_Lm || cat == CAT_Lo > > || cat == CAT_Nl // Letter_Number > > ; > > > > with CAT_foo being equivalent to Unicode category foo. > > > > Per UnicodeData.txt, ff11 is of category Nd, so it's a digit, not an > > alphabetic character. > > This is not wrong. However, see the comments in the generator of the > gnulib tables: > > https://git.savannah.gnu.org/gitweb/?p=gnulib.git;a=blob;f=lib/gen-uni-tables.c;h=0dceedc06cd72f886807fd575a2c4dba99cd147a;hb=HEAD#l5789 > > /* Consider all the non-ASCII digits as alphabetic. > ISO C 99 forbids us to have them in category "digit", > but we want iswalnum to return true on them. */ > > Likewise in the generator of the glibc tables: > > https://sourceware.org/git/?p=glibc.git;a=blob;f=localedata/unicode-gen/unicode_utils.py;h=5af03113a2f1f063769752ea426fcaf6f6ba9e95;hb=HEAD#l274 > > The original comment (from 2000) was: > > /* SUSV2 gives us some freedom for the "digit" category, but ISO C 99 > takes it away: > 7.25.2.1.5: > The iswdigit function tests for any wide character that corresponds > to a decimal-digit character (as defined in 5.2.1). > 5.2.1: > the 10 decimal digits 0 1 2 3 4 5 6 7 8 9 > */ > return (ch >= 0x0030 && ch <= 0x0039); > > The question is: In which category do you put these non-ASCII digits? > "print" and "graph", sure. But other than that? "punct" or "alnum"? > "punct" seems wrong. If you, like me, decide to put them in "alnum", > then you they need to be in "alpha" or "digit" (per POSIX > https://pubs.opengroup.org/onlinepubs/9699919799/functions/iswalnum.html ). > But ISO C 23 § 7.4.1.5 + § 5.2.1 does not allow them in category "digit". Thanks for the description. It was clear to me that they don't belong into the ISO C digit category, but other than that... So, if we change the expression in iswalpha_l to something like return cat == CAT_LC || cat == CAT_Lu || cat == CAT_Ll || cat == CAT_Lt || cat == CAT_Lm || cat == CAT_Lo || cat == CAT_Nl // Letter_Number /* Also all digits not allowed to be called digits per ISO C 99 */ || (cat == CAT_Nd && !(c >= (wint_t)'0' && c <= (wint_t)'9')); ; we're good? Thanks, Corinna -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation: https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple