This can lead to subtle bugs, cf. length of random and e_one. You have to convert everything to dstring to get the "expected" result. However, this is not always desirable.

There are three things that you need to be aware of when handling unicode: code units, code points and graphems.

In general the length of one guarantees anything about the length of the other, except for utf32, which is a 1:1 mapping between code units and code points.

In this thread, we were discussing the relationship between code points and graphemes. You're examples however apply to the relationship between code units and code points.

To measure the columns needed to print a string, you'll need the number of graphemes. (d|)?string.length gives you the number of code units.

If you normalize a string (in the sequence of characters/codepoints sense, not object.string) to NFC, it will decompose every precomposed character in the string (like é, single codeunit), establish a defined order between the composite characters and then recompose a selected few graphemes (like é). This way é always ends up as a single code unit in NFC. There are dozens of other combinations where you'll still have n:1 mapping between code points and graphemes left after normalization.

Example given already in this thread: putting an arrow over an latin letter is typical in math and always more than one codepoint.

Reply via email to