andrei Tue Sep 27 15:56:39 2005 EDT
Modified files:
/php-src README.UNICODE-UPGRADES
Log:
strrev() walkthrough
http://cvs.php.net/diff.php/php-src/README.UNICODE-UPGRADES?r1=1.5&r2=1.6&ty=u
Index: php-src/README.UNICODE-UPGRADES
diff -u php-src/README.UNICODE-UPGRADES:1.5 php-src/README.UNICODE-UPGRADES:1.6
--- php-src/README.UNICODE-UPGRADES:1.5 Fri Sep 23 17:24:31 2005
+++ php-src/README.UNICODE-UPGRADES Tue Sep 27 15:56:39 2005
@@ -274,24 +274,24 @@
This functions returns part of a string based on offset and length
parameters.
- void *str;
- int32_t str_len, cp_len;
- zend_uchar str_type;
-
- if (zend_parse_parameters(ZEND_NUM_ARGS() TSRMLS_CC, "tl|l", &str,
&str_len, &str_type, &f, &l) == FAILURE) {
- return;
- }
+ void *str;
+ int32_t str_len, cp_len;
+ zend_uchar str_type;
+
+ if (zend_parse_parameters(ZEND_NUM_ARGS() TSRMLS_CC, "tl|l", &str,
&str_len, &str_type, &f, &l) == FAILURE) {
+ return;
+ }
The first thing we notice is that the incoming string specifier is 't',
which means that we can accept all 3 string types. The 'str' variable is
declared as void*, because it can point to either UChar* or char*.
The actual type of the incoming string is stored in 'str_type' variable.
- if (str_type == IS_UNICODE) {
- cp_len = u_countChar32(str, str_len);
- } else {
- cp_len = str_len;
- }
+ if (str_type == IS_UNICODE) {
+ cp_len = u_countChar32(str, str_len);
+ } else {
+ cp_len = str_len;
+ }
If the string is a Unicode one, we cannot rely on the str_len value to tell
us the number of characters in it. Instead, we call u_countChar32() to
@@ -300,12 +300,12 @@
The next several lines normalize start and length parameters to fit within the
string. Nothing new here. Then we locate the appropriate segment.
- if (str_type == IS_UNICODE) {
- int32_t start = 0, end = 0;
- U16_FWD_N((UChar*)str, end, str_len, f);
- start = end;
- U16_FWD_N((UChar*)str, end, str_len, l);
- RETURN_UNICODEL((UChar*)str + start, end-start, 1);
+ if (str_type == IS_UNICODE) {
+ int32_t start = 0, end = 0;
+ U16_FWD_N((UChar*)str, end, str_len, f);
+ start = end;
+ U16_FWD_N((UChar*)str, end, str_len, l);
+ RETURN_UNICODEL((UChar*)str + start, end-start, 1);
Since codepoint (character) #n is not necessarily at offset #n in Unicode
strings, we start at the beginning and iterate forward until we have gone
@@ -314,13 +314,84 @@
of codepoints specified by the offset. Once that's done, we can return the
segment as a Unicode string.
- } else {
- RETURN_STRINGL((char*)str + f, l, 1);
- }
+ } else {
+ RETURN_STRINGL((char*)str + f, l, 1);
+ }
For native and binary types, we can return the segment directly.
+strrev()
+--------
+
+Let's look at strrev() which requires somewhat more complicated upgrade.
+While one of the guidelines for upgrades is that combining sequences are not
+really taken into account during processing -- substr() can break them up,
+for example -- in this case, we actually should be concerned, because
+reversing combining sequence may result in a completely different string. To
+illustrate:
+
+ a (U+0061 LATIN SMALL LETTER A)
+ o (U+006f LATIN SMALL LETTER O)
+ + ' (U+0301 COMBINING ACUTE ACCENT)
+ + _ (U+0320 COMBINING MINUS SIGN BELOW)
+ l (U+006C LATIN SMALL LETTER L)
+
+Reversing this would result in:
+
+ l (U+006C LATIN SMALL LETTER L)
+ + _ (U+0320 COMBINING MINUS SIGN BELOW)
+ + ' (U+0301 COMBINING ACUTE ACCENT)
+ o (U+006f LATIN SMALL LETTER O)
+ a (U+0061 LATIN SMALL LETTER A)
+
+All of a sudden the combining marks are being applied to 'l' instead of 'o'.
+To avoid this, we need to treat combininig sequences as a unit, by checking
+the combining character class of each character with u_getCombiningClass().
+
+strrev() obtains its single argument, a string, and unless the string is of
+Unicode type, processes it exactly as before, simply swapping bytes around.
+For Unicode case, the magic is like this:
+
+ int32_t i, x1, x2;
+ UChar32 ch;
+ UChar *u_s, *u_n, *u_p;
+
+ u_n = eumalloc(Z_USTRLEN_PP(str)+1);
+ u_p = u_n;
+ u_s = Z_USTRVAL_PP(str);
+
+ i = Z_USTRLEN_PP(str);
+ while (i > 0) {
+ U16_PREV(u_s, 0, i, ch);
+ if (u_getCombiningClass(ch) == 0) {
+ u_p += zend_codepoint_to_uchar(ch, u_p);
+ } else {
+ x2 = i;
+ do {
+ U16_PREV(u_s, 0, i, ch);
+ } while (u_getCombiningClass(ch) != 0);
+ x1 = i;
+ while (x1 <= x2) {
+ U16_NEXT(u_s, x1, Z_USTRLEN_PP(str), ch);
+ u_p += zend_codepoint_to_uchar(ch, u_p);
+ }
+ }
+ }
+ *u_p = 0;
+
+The basic idea is to walk the string backwards from the end, using
+U16_PREV() macro. If the combining class of the current character is 0,
+meaning it's a base character and not a combining mark, we simply append it
+to the new string. Otherwise, we save the location of the index and do a run
+over the characters until we get to the next one with combining class 0. At
+that point we append the sequence as is, without reversing, to the new
+string. Voila.
+
+Note that the code uses zend_codepoint_to_uchar() to convert full Unicode
+characters (UChar32 type) to 1 or 2 UTF-16 code units (UChar type).
+
+
References
==========
--
PHP CVS Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
