Autrijus, welcome to the club :)
On 2002.02.19, at 09:43, Autrijus Tang wrote: > Being a native Big5/GB (and HKSCS, Big5+, etc) user, I'm extremely > happy to > see Dan's work on Encode.pm. :-) Actually more credit should go to Nick Ing-Simmons; Encode::XX is so far all based upon his work on EUC_JP. > At jhi's bidding, I did some rudimentary test using the standard Big5 > encoding > range, with iconv 2.0 as the reference point. Within the > [A1-F9][40-7E,A1-FE] > range, the result was like this: > > 1. In the A140 - A3BF range (punctuations and phonetic symbols), iconv > parsed > without errors; Encode, however, does not agree with it in 3 places: > > * big5(A150) doesn't get mapped properly. > * it has an off-by-one error in range big5(A15A..A17D); it mapped > big5(A15A) as ucs2(big5(A15B)), big5(A15B) as ucs2(big5(A15C)), > etc. > * it cannot parse the range big5(A17E..A3BF). > > 2. In the A440 - C67E range (widely-used characters), both iconv and > Encode > worked perfectly. > > 3. In the C6A1 - C8D3 range (word parts, japanese characters, and > assorted > symbols), both Encode and iconv doesn't work beyond big5(C7FC), which > is expected. > > 4. In the C940 - F9D5 range (rarely used characters), both iconv and > Encode > worked perfectly. > > 5. In the F9D6 - F9FE range (addendum, table-drawing characters), both > of them > doesnt work, which is expected. I also found Encode::TW "compiling" somewhat noisy. Would you also test it on Encode::Tcl module? Encode::TW is in a way just compiled version of Encode::Tcl. > 6. I didn't test utf8=>big5 much, but they seem to work alright. > > Note that the Big5+ spec at <http://www.cmex.org.tw/download-b5.html> > specified > a rather comprehensive set of official big5<=>ucs2 mappings, the > relevant > part of it are available at <http://autrijus.org/big5-ucs.tar.gz>. > Their format > should be self-descriptory; I wonder if it's possible to use that table > to > fill in the missing codepoints, or should we add a 'big5p' encoding? Other major codings that are missing is obviously CNS11643. I don't know much about it but so far as I know CNS11643 is ISO-2022 compliant and CNS11643-1 and CNS11643-2 covers Big5. But as you see Encode:XX is so far dependent on Tcl encoding and there is no CNS11643 there yet.... > Anyway, I'll get some more tests (and get GB working) when I wake up. > > Hope that helps, It does! Now we are looking for testers of KR and CN as well. Anyone? Dan the Man with Too Many Languages to Learn
