Hi! I believe cpp_check_xid_property (function added to libcpp for the Rust FE) is incorrect. The bits are documented /* Valid in a C++23 identifier? */ CXX23 = 32, /* Valid in a C++23 identifier, but not as the first character? */ NXX23 = 64, So, characters which have both the Unicode XID_Start and XID_Continue derived properties have just CXX23 bit set, and characters which have just XID_Continue derived property and not XID_Start have CXX23|NXX23 bits set. So, I believe the function (except for the ASCII fast path which is correct) incorrectly returns CPP_XID_START | CPP_XID_CONTINUE for any characters which have XID_Continue derived property, and only ever returns just CPP_XID_CONTINUE in the ASCII [0-9_] cases.
The following untested patch should fix that, I guess it would be nice to have a testcase, pick some character in XID_Continue and not in XID_Start, random choice e.g. SUNDANESE CONSONANT SIGN PASANGAN WA, and check if it can be used in the second+ character of identifier (probably it can and should continue to do so) and as the first character of identifier (likely incorrectly accepted right now). 2026-06-11 Jakub Jelinek <[email protected]> * charset.cc (cpp_check_xid_property): Return CPP_XID_START | CPP_XID_CONTINUE only if CXX23 bit is set and NXX23 is not. If both are set, return CPP_XID_CONTINUE. --- libcpp/charset.cc.jj 2026-04-21 18:24:44.122033404 +0200 +++ libcpp/charset.cc 2026-06-11 13:33:51.356096060 +0200 @@ -1366,9 +1366,9 @@ cpp_check_xid_property (cppchar_t c) unsigned short flags = ucnranges[mn].flags; - if (flags & CXX23) + if ((flags & (CXX23 | NXX23)) == CXX23) return CPP_XID_START | CPP_XID_CONTINUE; - if (flags & NXX23) + if ((flags & (CXX23 | NXX23)) == (CXX23 | NXX23)) return CPP_XID_CONTINUE; return 0; } Jakub
