On Friday 28 February 2003 09:03 pm, John Kilbourne wrote: > I am a beginner as well, with the task of finding and counting the > non-ascii characters in a utf-8 text. How do I do this?
That depends on what you want to accomplish. Counting Unicode code points is easy. ASCII characters have the form 0x0bbbbbbb in UTF-8. Initial bytes of non-ASCII character encodingss have the form 10bbbbbb. All other bytes in UTF-8 streams have the form 11bbbbbb. So matching the range 10000000-10111111 (hex 80-BF) will suffice. If you want to count text characters while ignoring control characters and undefined code points, or to count base character +modifier sequences as single characters, or to count glyphs in the rendering, you need to have a precise set of definitions suited to your application, and know a good deal about the details of Unicode. -- Edward Cherlin Generalist & activist--Linux, languages, literacy and more "A knot! Oh, do let me help to undo it!" --Alice in Wonderland
