> 
> Maybe "lousy" is too strong a word, but aside from
> compatibility with other 
> libs/software that use it (which I'll address separately),
> UTF-16 is not 
> particularly useful compared to UTF-8 and UTF-32:
...
> 

I tried to avoid commenting this because I am afraid we'll stray away from the 
main point (which is not discussion about which Unicode is better). But in 
short I would say: "Not quite right". UTF-16 as already mentioned is generally 
faster for non-Latin letters (as reading 2 bytes of aligned data takes the same 
time as reading 1 byte). Although, I am not familiar with Asian languages, I 
believe that UTF-16 requires just 2 bytes instead of 3 for most of symbols. 
That is one of the reason they don't like UTF-8. UTF-32 doesn't have any 
advantage except for being fixed length. It has a lot of unnecessary memory, 
cache, etc. overhead (the worst case scenario for both UTF8/16) which is not 
justified for any language.

> 
> First of all, it's not exactly unheard of for big projects
> to make a 
> sub-optimal decision.

I would say, the decision was quite optimal for many reasons, including that 
"lousy programming" will not cause too many problems as in case of UTF-8.

> 
> Secondly, Java and Windows adapted 16-bit encodings back
> when many people 
> were still under the mistaken impression that would allow
> them to hold any 
> character in one code-unit. If that had been true, then it

I doubt that it was the only reason. UTF-8 was already available before Windows 
NT was released. It would be much easier to use UTF-8 instead of ANSI as 
opposed to creating parallel API. Nonetheless, UTF-16 has been chosen. In 
addition, C# has been released already when UTF-16 became variable length. I 
doubt that conversion overhead (which is small compared to VM) was the main 
reason to preserve UTF-16.


Concerning why I say that it's good to have conversion to UTF-32 (you asked 
somewhere):

I think you did not understand correctly what I meant. This a very common 
practice, and in fact - required, to convert from both UTF-8 and UTF-16 to 
UTF-32 when you need to do character analysis (e.g. mbtowc() in C). In fact, it 
is the only place where UTF-32 is commonly used and useful.



      

Reply via email to