Re: Chinese LyX

[Abdelrazak Younes]

Enrico Forestieri wrote:

I have compiled a Cygwin version of LyX/Qt4 using the native GUI (no X11)
and now I see the chinese characters mentioned by Abdel when I load
an old document. I also get a lot of messages on the console:

Error returned from iconv
EILSEQ An invalid multibyte sequence has been encountered in the input.
When converting from UCS-4 to UCS-2.

Input: 0xff 0xff 0xff 0xa2

This was not the case with Qt3 where the characters were simply hollow
squares on screen. However, when I start a new document the characters
are shown correctly.

The attached patch at least make LyX show normal text correctly.

LyX also crashes when quitting, but I don't know if this is related to
the patches I applied to Qt4 or if it is a LyX problem. Anyway, all
Qt4 examples compile and run just fine.

Backtrace?


Index: unicode.C
===================================================================
--- unicode.C   (revision 14808)
+++ unicode.C   (working copy)
@@ -177,12 +177,13 @@
 
 std::vector<boost::uint32_t> utf8_to_ucs4(std::vector<char> const & utf8str)
 {
-       //lyxerr << "Buff = " << string(utf8str.begin(), utf8str.end())
-       //       << " (" << utf8str.size() << ")" << endl;
-       //lyxerr << "Res = " << string(res.begin(), res.end())
-       //       << " (" << res.size() << ")" << endl;
+       std::vector<char> res = iconv_convert("UCS-4LE", "UTF-8", utf8str);
 
-       std::vector<char> res = iconv_convert("UCS-4", "UTF-8", utf8str);
+       lyxerr << "Buff = " << string(utf8str.begin(), utf8str.end())
+              << " (" << utf8str.size() << ")" << endl;
+       lyxerr << "Res = " << string(res.begin(), res.end())
+              << " (" << res.size() << ")" << endl;
+
        return bytes_to_ucs4(res);
 }
 
@@ -203,7 +204,7 @@
                lyxerr << std::setw(2) << std::setfill('0') << ((s & 0xff00) >> 
8) << endl;
        }
 
-       std::vector<char> res = iconv_convert("UCS-4", "UCS-2", in);
+       std::vector<char> res = iconv_convert("UCS-4LE", "UCS-2LE", in);
        return bytes_to_ucs4(res);
 }
 
@@ -221,7 +222,7 @@
                in.push_back(static_cast<char>((s & 0x0000ff00) >> 8));
                in.push_back(static_cast<char>(s & 0x000000ff));
        }
-       std::vector<char> res = iconv_convert("UCS-2", "UCS-4", in);
+       std::vector<char> res = iconv_convert("UCS-2LE", "UCS-4LE", in);
        return bytes_to_ucs2(res);
 }
 
@@ -236,7 +237,7 @@
                in.push_back(static_cast<char>((s[i] & 0x0000ff00) >> 8));
                in.push_back(static_cast<char>(s[i] & 0x000000ff));
        }
-       std::vector<char> res = iconv_convert("UCS-2", "UCS-4", in);
+       std::vector<char> res = iconv_convert("UCS-2LE", "UCS-4LE", in);
        return bytes_to_ucs2(res);
 }
 
@@ -249,7 +250,7 @@
        in.push_back(static_cast<char>((c & 0x00ff0000) >> 16));
        in.push_back(static_cast<char>((c & 0x0000ff00) >> 8));
        in.push_back(static_cast<char>(c & 0x000000ff));
-       std::vector<char> res = iconv_convert("UCS-2", "UCS-4", in);
+       std::vector<char> res = iconv_convert("UCS-2LE", "UCS-4LE", in);
        std::vector<unsigned short> us = bytes_to_ucs2(res);
        if (!us.empty())
                return us[0];
@@ -270,7 +271,7 @@
                in.push_back(static_cast<char>((s & 0x0000ff00) >> 8));
                in.push_back(static_cast<char>(s & 0x000000ff));
        }
-       std::vector<char> res = iconv_convert("UTF-8", "UCS-4", in);
+       std::vector<char> res = iconv_convert("UTF-8", "UCS-4LE", in);
        return res;
 }
 
@@ -282,6 +283,6 @@
        in.push_back(static_cast<char>((c & 0x00ff0000) >> 16));
        in.push_back(static_cast<char>((c & 0x0000ff00) >> 8));
        in.push_back(static_cast<char>(c & 0x000000ff));
-       std::vector<char> res = iconv_convert("UTF-8", "UCS-4", in);
+       std::vector<char> res = iconv_convert("UTF-8", "UCS-4LE", in);
        return res;
 }