I'm puzzled by a statement in UAX #29 Unicode Text Segmentation: "In particular, the characters with the Line_Break property values of Contingent_Break (CB), Complex_Context (SA/Southeast Asian), and Unknown (XX) are assigned word boundary property values based on criteria outside of the scope of this annex. That means that satisfactory treatment of languages like Chinese or Thai requires special handling."
Is 'Contingent_Break (CB)' an error for 'Ideographic (ID)'? That would make sense for Chinese, for some applications needs to group ideographs into words. While I am on the topic, does anyone know of character level mechanisms used to advise alogrithms of the word boundaries (or lack of boundaries) in Chinese text? Richard.
