In perl.git, the branch blead has been updated <https://perl5.git.perl.org/perl.git/commitdiff/d713f9d937268132d1244ae0ea17496e08c82194?hp=5d26d78791c18cfb2ce66f44cbf8e9679dcd23ec>
- Log ----------------------------------------------------------------- commit d713f9d937268132d1244ae0ea17496e08c82194 Author: Karl Williamson <[email protected]> Date: Sun Aug 4 21:00:36 2019 -0600 perlapi: Reword some isFOO char classification docs The input to these isn't restricted to a byte, and some minor clarifications. commit 8b4b30c5d389983c3df51b7ff3b38e5608c7c2e2 Author: Karl Williamson <[email protected]> Date: Sat Aug 3 09:17:43 2019 -0600 perlapi: 5.30 promise not met; change to 5.32 We delayed this change, but I forgot to change this documentation commit 31532982b04c20a43aa9c3d26780e3591c524fbc Author: Karl Williamson <[email protected]> Date: Thu Jun 27 15:39:11 2019 -0600 regcomp.c: Don't read off the end of buffer Until this commit, it was possible that \p{nv=3/} would cause the right brace to be considered part of the property name. Spotted by Hugo van der Sanden ----------------------------------------------------------------------- Summary of changes: handy.h | 100 ++++++++++++++++++++++++++++++++------------------------------ regcomp.c | 4 ++- 2 files changed, 54 insertions(+), 50 deletions(-) diff --git a/handy.h b/handy.h index 24c028a638..685591e6f3 100644 --- a/handy.h +++ b/handy.h @@ -569,9 +569,9 @@ each class. (Not all macros have all variants; each item below lists the ones valid for it.) None are affected by C<use bytes>, and only the ones with C<LC> in the name are affected by the current locale. -The base function, e.g., C<isALPHA()>, takes an octet (either a C<char> or a -C<U8>) as input and returns a boolean as to whether or not the character -represented by that octet is (or on non-ASCII platforms, corresponds to) an +The base function, e.g., C<isALPHA()>, takes any signed or unsigned value, +treating it as a code point, and returns a boolean as to whether or not the +character represented by it is (or on non-ASCII platforms, corresponds to) an ASCII character in the named class based on platform, Unicode, and Perl rules. If the input is a number that doesn't fit in an octet, FALSE is returned. @@ -585,16 +585,19 @@ since ASCII is a subset of Latin-1. But the non-ASCII code points are treated as if they are Latin-1 characters. For example, C<isWORDCHAR_L1()> will return true when called with the code point 0xDF, which is a word character in both ASCII and EBCDIC (though it represents different characters in each). +If the input is a number that doesn't fit in an octet, FALSE is returned. +(Perl's documentation uses a colloquial definition of Latin-1, to include all +code points below 256.) -Variant C<isI<FOO>_uvchr> is like the C<isI<FOO>_L1> variant, but accepts any UV code -point as input. If the code point is larger than 255, Unicode rules are used -to determine if it is in the character class. For example, +Variant C<isI<FOO>_uvchr> is exactly like the C<isI<FOO>_L1> variant, for +inputs below 256, but if the code point is larger than 255, Unicode rules are +used to determine if it is in the character class. For example, C<isWORDCHAR_uvchr(0x100)> returns TRUE, since 0x100 is LATIN CAPITAL LETTER A WITH MACRON in Unicode, and is a word character. Variant C<isI<FOO>_utf8_safe> is like C<isI<FOO>_uvchr>, but is used for UTF-8 -encoded strings. Each call classifies one character, even if the string -contains many. This variant takes two parameters. The first, C<p>, is a +encoded strings. Each call classifies the first character of the string. This +variant takes two parameters. The first, C<p>, is a pointer to the first byte of the character to be classified. (Recall that it may take more than one byte to represent a character in UTF-8 strings.) The second parameter, C<e>, points to anywhere in the string beyond the first @@ -609,73 +612,72 @@ future releases. Variant C<isI<FOO>_utf8> is like C<isI<FOO>_utf8_safe>, but takes just a single parameter, C<p>, which has the same meaning as the corresponding parameter does in C<isI<FOO>_utf8_safe>. The function therefore can't check if it is reading -beyond the end of the string. Starting in Perl v5.30, it will take a second +beyond the end of the string. Starting in Perl v5.32, it will take a second parameter, becoming a synonym for C<isI<FOO>_utf8_safe>. At that time every program that uses it will have to be changed to successfully compile. In the meantime, the first runtime call to C<isI<FOO>_utf8> from each call point in the program will raise a deprecation warning, enabled by default. You can convert your program now to use C<isI<FOO>_utf8_safe>, and avoid the warnings, and get an -extra measure of protection, or you can wait until v5.30, when you'll be forced +extra measure of protection, or you can wait until v5.32, when you'll be forced to add the C<e> parameter. -Variant C<isI<FOO>_LC> is like the C<isI<FOO>_A> and C<isI<FOO>_L1> variants, but the -result is based on the current locale, which is what C<LC> in the name stands -for. If Perl can determine that the current locale is a UTF-8 locale, it uses -the published Unicode rules; otherwise, it uses the C library function that -gives the named classification. For example, C<isDIGIT_LC()> when not in a -UTF-8 locale returns the result of calling C<isdigit()>. FALSE is always +Variant C<isI<FOO>_LC> is like the C<isI<FOO>_A> and C<isI<FOO>_L1> variants, +but the result is based on the current locale, which is what C<LC> in the name +stands for. If Perl can determine that the current locale is a UTF-8 locale, +it uses the published Unicode rules; otherwise, it uses the C library function +that gives the named classification. For example, C<isDIGIT_LC()> when not in +a UTF-8 locale returns the result of calling C<isdigit()>. FALSE is always returned if the input won't fit into an octet. On some platforms where the C library function is known to be defective, Perl changes its result to follow the POSIX standard's rules. -Variant C<isI<FOO>_LC_uvchr> is like C<isI<FOO>_LC>, but is defined on any UV. It -returns the same as C<isI<FOO>_LC> for input code points less than 256, and -returns the hard-coded, not-affected-by-locale, Unicode results for larger ones. +Variant C<isI<FOO>_LC_uvchr> acts exactly like C<isI<FOO>_LC> for inputs less +than 256, but for larger ones it returns the Unicode classification of the code +point. Variant C<isI<FOO>_LC_utf8_safe> is like C<isI<FOO>_LC_uvchr>, but is used for UTF-8 -encoded strings. Each call classifies one character, even if the string -contains many. This variant takes two parameters. The first, C<p>, is a -pointer to the first byte of the character to be classified. (Recall that it -may take more than one byte to represent a character in UTF-8 strings.) The -second parameter, C<e>, points to anywhere in the string beyond the first -character, up to one byte past the end of the entire string. The suffix -C<_safe> in the function's name indicates that it will not attempt to read -beyond S<C<e - 1>>, provided that the constraint S<C<s E<lt> e>> is true (this -is asserted for in C<-DDEBUGGING> builds). If the UTF-8 for the input -character is malformed in some way, the program may croak, or the function may -return FALSE, at the discretion of the implementation, and subject to change in -future releases. +encoded strings. Each call classifies the first character of the string. This +variant takes two parameters. The first, C<p>, is a pointer to the first byte +of the character to be classified. (Recall that it may take more than one byte +to represent a character in UTF-8 strings.) The second parameter, C<e>, +points to anywhere in the string beyond the first character, up to one byte +past the end of the entire string. The suffix C<_safe> in the function's name +indicates that it will not attempt to read beyond S<C<e - 1>>, provided that +the constraint S<C<s E<lt> e>> is true (this is asserted for in C<-DDEBUGGING> +builds). If the UTF-8 for the input character is malformed in some way, the +program may croak, or the function may return FALSE, at the discretion of the +implementation, and subject to change in future releases. Variant C<isI<FOO>_LC_utf8> is like C<isI<FOO>_LC_utf8_safe>, but takes just a single parameter, C<p>, which has the same meaning as the corresponding parameter does in C<isI<FOO>_LC_utf8_safe>. The function therefore can't check if it is reading -beyond the end of the string. Starting in Perl v5.30, it will take a second +beyond the end of the string. Starting in Perl v5.32, it will take a second parameter, becoming a synonym for C<isI<FOO>_LC_utf8_safe>. At that time every program that uses it will have to be changed to successfully compile. In the meantime, the first runtime call to C<isI<FOO>_LC_utf8> from each call point in the program will raise a deprecation warning, enabled by default. You can convert your program now to use C<isI<FOO>_LC_utf8_safe>, and avoid the warnings, -and get an extra measure of protection, or you can wait until v5.30, when +and get an extra measure of protection, or you can wait until v5.32, when you'll be forced to add the C<e> parameter. -=for apidoc Am|bool|isALPHA|char ch -Returns a boolean indicating whether the specified character is an -alphabetic character, analogous to C<m/[[:alpha:]]/>. +=for apidoc Am|bool|isALPHA|int ch +Returns a boolean indicating whether the specified input is one of C<[A-Za-z]>, +analogous to C<m/[[:alpha:]]/>. See the L<top of this section|/Character classification> for an explanation of variants C<isALPHA_A>, C<isALPHA_L1>, C<isALPHA_uvchr>, C<isALPHA_utf8_safe>, C<isALPHA_LC>, C<isALPHA_LC_uvchr>, and C<isALPHA_LC_utf8_safe>. -=for apidoc Am|bool|isALPHANUMERIC|char ch -Returns a boolean indicating whether the specified character is a either an -alphabetic character or decimal digit, analogous to C<m/[[:alnum:]]/>. +=for apidoc Am|bool|isALPHANUMERIC|int ch +Returns a boolean indicating whether the specified character is one of +C<[A-Za-z0-9]>, analogous to C<m/[[:alnum:]]/>. See the L<top of this section|/Character classification> for an explanation of variants C<isALPHANUMERIC_A>, C<isALPHANUMERIC_L1>, C<isALPHANUMERIC_uvchr>, C<isALPHANUMERIC_utf8_safe>, C<isALPHANUMERIC_LC>, C<isALPHANUMERIC_LC_uvchr>, and C<isALPHANUMERIC_LC_utf8_safe>. -=for apidoc Am|bool|isASCII|char ch +=for apidoc Am|bool|isASCII|int ch Returns a boolean indicating whether the specified character is one of the 128 characters in the ASCII character set, analogous to C<m/[[:ascii:]]/>. On non-ASCII platforms, it returns TRUE iff this @@ -860,7 +862,7 @@ an API that does allow every possible legal result to be returned.) Likewise no other function that is crippled by not being able to give the correct results for the full range of possible inputs has been implemented here. -=for apidoc Am|U8|toUPPER|U8 ch +=for apidoc Am|U8|toUPPER|int ch Converts the specified character to uppercase. If the input is anything but an ASCII lowercase character, that input character itself is returned. Variant C<toUPPER_A> is equivalent. @@ -897,13 +899,13 @@ implementation, and subject to change in future releases. =for apidoc Am|UV|toUPPER_utf8|U8* p|U8* s|STRLEN* lenp This is like C<L</toUPPER_utf8_safe>>, but doesn't have the C<e> parameter The function therefore can't check if it is reading -beyond the end of the string. Starting in Perl v5.30, it will take the C<e> +beyond the end of the string. Starting in Perl v5.32, it will take the C<e> parameter, becoming a synonym for C<toUPPER_utf8_safe>. At that time every program that uses it will have to be changed to successfully compile. In the meantime, the first runtime call to C<toUPPER_utf8> from each call point in the program will raise a deprecation warning, enabled by default. You can convert your program now to use C<toUPPER_utf8_safe>, and avoid the warnings, and get an -extra measure of protection, or you can wait until v5.30, when you'll be forced +extra measure of protection, or you can wait until v5.32, when you'll be forced to add the C<e> parameter. =for apidoc Am|U8|toFOLD|U8 ch @@ -944,13 +946,13 @@ implementation, and subject to change in future releases. =for apidoc Am|UV|toFOLD_utf8|U8* p|U8* s|STRLEN* lenp This is like C<L</toFOLD_utf8_safe>>, but doesn't have the C<e> parameter The function therefore can't check if it is reading -beyond the end of the string. Starting in Perl v5.30, it will take the C<e> +beyond the end of the string. Starting in Perl v5.32, it will take the C<e> parameter, becoming a synonym for C<toFOLD_utf8_safe>. At that time every program that uses it will have to be changed to successfully compile. In the meantime, the first runtime call to C<toFOLD_utf8> from each call point in the program will raise a deprecation warning, enabled by default. You can convert your program now to use C<toFOLD_utf8_safe>, and avoid the warnings, and get an -extra measure of protection, or you can wait until v5.30, when you'll be forced +extra measure of protection, or you can wait until v5.32, when you'll be forced to add the C<e> parameter. =for apidoc Am|U8|toLOWER|U8 ch @@ -999,13 +1001,13 @@ implementation, and subject to change in future releases. =for apidoc Am|UV|toLOWER_utf8|U8* p|U8* s|STRLEN* lenp This is like C<L</toLOWER_utf8_safe>>, but doesn't have the C<e> parameter The function therefore can't check if it is reading -beyond the end of the string. Starting in Perl v5.30, it will take the C<e> +beyond the end of the string. Starting in Perl v5.32, it will take the C<e> parameter, becoming a synonym for C<toLOWER_utf8_safe>. At that time every program that uses it will have to be changed to successfully compile. In the meantime, the first runtime call to C<toLOWER_utf8> from each call point in the program will raise a deprecation warning, enabled by default. You can convert your program now to use C<toLOWER_utf8_safe>, and avoid the warnings, and get an -extra measure of protection, or you can wait until v5.30, when you'll be forced +extra measure of protection, or you can wait until v5.32, when you'll be forced to add the C<e> parameter. =for apidoc Am|U8|toTITLE|U8 ch @@ -1047,13 +1049,13 @@ implementation, and subject to change in future releases. =for apidoc Am|UV|toTITLE_utf8|U8* p|U8* s|STRLEN* lenp This is like C<L</toLOWER_utf8_safe>>, but doesn't have the C<e> parameter The function therefore can't check if it is reading -beyond the end of the string. Starting in Perl v5.30, it will take the C<e> +beyond the end of the string. Starting in Perl v5.32, it will take the C<e> parameter, becoming a synonym for C<toTITLE_utf8_safe>. At that time every program that uses it will have to be changed to successfully compile. In the meantime, the first runtime call to C<toTITLE_utf8> from each call point in the program will raise a deprecation warning, enabled by default. You can convert your program now to use C<toTITLE_utf8_safe>, and avoid the warnings, and get an -extra measure of protection, or you can wait until v5.30, when you'll be forced +extra measure of protection, or you can wait until v5.32, when you'll be forced to add the C<e> parameter. =cut diff --git a/regcomp.c b/regcomp.c index 1117998fc8..cf9246473f 100644 --- a/regcomp.c +++ b/regcomp.c @@ -23092,7 +23092,9 @@ Perl_parse_uniprop_string(pTHX_ } /* Store the first real character in the denominator */ - lookup_name[j++] = name[i]; + if (i < name_len) { + lookup_name[j++] = name[i]; + } } } -- Perl5 Master Repository
