Re: [fltk.bugs] [LOW] STR #2505: Xft backend doesn't filter bad UTF-8 characters
[STR Closed w/Resolution]
Link: http://www.fltk.org/str.php?L2505
Version: 1.3.0
Fix Version: 1.3.0 (r8208)
Fixed in Subversion repository.
Thanks Albrecht for this lesson of Cygwin that's was very useful
for me.
Could Xft be processed under Cygwin as it is under X11 ?
That is, if one does, in fl_utf.c:
unsigned fl_utf8towc(const char* src, unsigned srclen,
wchar_t* dst, unsigned dstlen)
{
#if defined(WIN32) !defined(__CYGWIN__)
return fl_utf8toUtf16(src, srclen, (unsigned short*)dst, dstlen);
#else
...
#endif
}
and removes the cygwin-special cases in Fl_Xlib_Graphics_Driver::draw
and utf8extents() of fl_font_xft.cxx,
would that run OK on cygwin ?
Link: http://www.fltk.org/str.php?L2505
Version: 1.3.0
Fix Version: 1.3.0 (r8208)
___
fltk-bugs mailing list
fltk-bugs@easysw.com
http://lists.easysw.com/mailman/listinfo/fltk-bugs
Re: [fltk.bugs] [LOW] STR #2505: Xft backend doesn't filter bad UTF-8 characters
[STR Closed w/Resolution] Link: http://www.fltk.org/str.php?L2505 Version: 1.3.0 Fix Version: 1.3.0 (r8211) For the record: the above-mentioned commit was in r8208. Further improvement (removed double string conversion) in svn r 8211. Link: http://www.fltk.org/str.php?L2505 Version: 1.3.0 Fix Version: 1.3.0 (r8211) ___ fltk-bugs mailing list fltk-bugs@easysw.com http://lists.easysw.com/mailman/listinfo/fltk-bugs
Re: [fltk.bugs] [LOW] STR #2505: Xft backend doesn't filter bad UTF-8 characters
DO NOT REPLY TO THIS MESSAGE. INSTEAD, POST ANY RESPONSES TO THE LINK BELOW. [STR New] Link: http://www.fltk.org/str.php?L2505 Version: 1.3-current I propose to close this STR with the attached patch fl_font_xft.diff that's a simplification of Ossman's one: the UTF-8 decoding is done by fl_utf8towc() that always returns a 32-bit Unicode value under X11, so I don't think XftTextExtents16() and XftTextExtentsUtf8() are useful. In essence, it consists in decoding the UTF-8 string into Unicode (32-bit) characters for each fl_draw() and fl_width() call and drawing or computing the width of the text with an Xft function that accepts 32-bit Unicode values instead of a UTF-8 string. The decoding is done, ultimately, with function fl_utf8decode() that transforms any byte sequence into valid Unicode values, interpreting bad UTF-8 data as CP1252. Thus, any byte sequence will produce some output text. I suspect that UTF-8-accepting Xft functions decode the UTF-8 internally. Thus, doing this decode operation before hand may not slow down the process. I have found it runs well under Linux with test non-UTF-8 data. Link: http://www.fltk.org/str.php?L2505 Version: 1.3-current ___ fltk-bugs mailing list fltk-bugs@easysw.com http://lists.easysw.com/mailman/listinfo/fltk-bugs
Re: [fltk.bugs] [LOW] STR #2505: Xft backend doesn't filter bad UTF-8 characters
DO NOT REPLY TO THIS MESSAGE. INSTEAD, POST ANY RESPONSES TO THE LINK BELOW. [STR New] Link: http://www.fltk.org/str.php?L2505 Version: 1.3.0 Taking assignment of this STR. Link: http://www.fltk.org/str.php?L2505 Version: 1.3.0 ___ fltk-bugs mailing list fltk-bugs@easysw.com http://lists.easysw.com/mailman/listinfo/fltk-bugs
Re: [fltk.bugs] [LOW] STR #2505: Xft backend doesn't filter bad UTF-8 characters
DO NOT REPLY TO THIS MESSAGE. INSTEAD, POST ANY RESPONSES TO THE LINK BELOW. [STR New] Link: http://www.fltk.org/str.php?L2505 Version: 1.3.0 Posts overlapped: please go ahead, apply and close ;-) Link: http://www.fltk.org/str.php?L2505 Version: 1.3.0 ___ fltk-bugs mailing list fltk-bugs@easysw.com http://lists.easysw.com/mailman/listinfo/fltk-bugs
Re: [fltk.bugs] [LOW] STR #2505: Xft backend doesn't filter bad UTF-8 characters
DO NOT REPLY TO THIS MESSAGE. INSTEAD, POST ANY RESPONSES TO THE LINK BELOW.
[STR New]
Link: http://www.fltk.org/str.php?L2505
Version: 1.3.0
Manolo, unfortunately your patch doesn't work on Windows with Cygwin/X11
(configure --enable-x11).
fl_utf8towc() does /not/ convert to an array of unsigned, but wchar_t, and
this is only 2 bytes on Windows. Therefore we need the 16-bit variants of
the Xft-specific functions. I modified your patch (see attached file
fl_font_xft_as.diff), so that it takes care of this (I used wchar_t
instead of unsigned). Instead of the OP's sizeof() check I used
__CYGWIN__, because this is the only platform that (currently?) supports
X11 on Windows (and we can't use WIN32 here).
There's another case in utf8reformat() where your patch returns NULL, if
n==0. I don't think that we should return NULL here, because the returned
pointer would be used later (although this might never happen, but
anyway). My patch returns a pointer to an empty wchar_t string instead.
I tested my patch both on Windows (Cygwin with --enable-x11) and Linux,
and it worked for me. Please check if this is okay (at least it's better
than before).
Link: http://www.fltk.org/str.php?L2505
Version: 1.3.0Index: src/fl_font_xft.cxx
===
--- src/fl_font_xft.cxx (revision 8203)
+++ src/fl_font_xft.cxx (working copy)
@@ -358,6 +358,38 @@
// XftFontClose(fl_display, font);
}
+/* returns the string decoded into a series of Unicode characters.
+ n is set to the number of characters upon return.
+ Don't deallocate the returned memory.
+ */
+static const wchar_t *utf8reformat(const char *str, int n)
+{
+ static const wchar_t empty[] = {0};
+ static wchar_t *buffer;
+ static int lbuf = 0;
+ int newn;
+ if (n == 0) return empty;
+ newn = fl_utf8towc(str, n, (wchar_t*)buffer, lbuf);
+ if (newn = lbuf) {
+lbuf = newn + 100;
+if (buffer) free(buffer);
+buffer = (wchar_t*)malloc(lbuf * sizeof(wchar_t));
+}
+ n = fl_utf8towc(str, n, (wchar_t*)buffer, lbuf);
+ return buffer;
+}
+
+static void utf8extents(const char *str, int n, XGlyphInfo *extents)
+{
+ memset(extents, 0, sizeof(XGlyphInfo));
+ const wchar_t *buffer = utf8reformat(str, n);
+#ifdef __CYGWIN__
+XftTextExtents16(fl_display, current_font, (XftChar16 *)buffer, n,
extents);
+#else
+XftTextExtents32(fl_display, current_font, (XftChar32 *)buffer, n,
extents);
+#endif
+}
+
int fl_height() {
if (current_font) return current_font-ascent + current_font-descent;
else return -1;
@@ -371,7 +403,7 @@
double fl_width(const char *str, int n) {
if (!current_font) return -1.0;
XGlyphInfo i;
- XftTextExtentsUtf8(fl_display, current_font, (XftChar8 *)str, n, i);
+ utf8extents(str, n, i);
return i.xOff;
}
@@ -397,7 +429,7 @@
return;
}
XGlyphInfo gi;
- XftTextExtentsUtf8(fl_display, current_font, (XftChar8 *)c, n, gi);
+ utf8extents(c, n, gi);
w = gi.width;
h = gi.height;
@@ -585,8 +617,13 @@
color.color.green = ((int)g)*0x101;
color.color.blue = ((int)b)*0x101;
color.color.alpha = 0x;
-
- XftDrawStringUtf8(draw_, color, current_font, x, y, (XftChar8 *)str, n);
+
+ const wchar_t *buffer = utf8reformat(str, n);
+#ifdef __CYGWIN__
+ XftDrawString16(draw_, color, current_font, x, y, (XftChar16 *)buffer, n);
+#else
+ XftDrawString32(draw_, color, current_font, x, y, (XftChar32 *)buffer, n);
+#endif
}
void Fl_Xlib_Graphics_Driver::draw(int angle, const char *str, int n, int x,
int y) {
___
fltk-bugs mailing list
fltk-bugs@easysw.com
http://lists.easysw.com/mailman/listinfo/fltk-bugs
Re: [fltk.bugs] [LOW] STR #2505: Xft backend doesn't filter bad UTF-8 characters
DO NOT REPLY TO THIS MESSAGE. INSTEAD, POST ANY RESPONSES TO THE LINK BELOW. [STR New] Link: http://www.fltk.org/str.php?L2505 Version: 1.3-current Quite right, the gist of it is that it converts the encoding to wchar_t to exploit the side effect of sanitizing the string. For the issues: 1. Dynamic arrays I'll have a look at this and post an updated patch. 2. sizeof() checks Yeah, I'm not particularly proud of this one. The basic problem is that wchar_t is not well defined when it comes to size, so there needed to be something. Preferably I'd have a preprocessor check, but I didn't have any defines to work with. Things can be improved by adding a check in configure for this. 3. Performance As you say, the performance hit is already there for Windows so as far as the project goes, it has been deemed acceptable. Improvements might be possible, but I suspect they're small. The conversion from UTF-8 to UCS-4 is a fairly simple one, and the extra cycles needed compared to only parsing and validating the UTF-8 string are probably negligable. The big performance hog will most likely be the malloc(). But let's not kid ourselves. This is C++, so there are stray malloc():s hidden all over the place when objects get flung around. So unless someone can show me numbers that this is causing any tangible slowdown, I'd consider this a case of premature optimization. :) Link: http://www.fltk.org/str.php?L2505 Version: 1.3-current ___ fltk-bugs mailing list fltk-bugs@easysw.com http://lists.easysw.com/mailman/listinfo/fltk-bugs
Re: [fltk.bugs] [LOW] STR #2505: Xft backend doesn't filter bad UTF-8 characters
DO NOT REPLY TO THIS MESSAGE. INSTEAD, POST ANY RESPONSES TO THE LINK BELOW.
[STR New]
Link: http://www.fltk.org/str.php?L2505
Version: 1.3-current
Attached file fltk-1_v3.3.x-xft-check.patch...
Link: http://www.fltk.org/str.php?L2505
Version: 1.3-currentdiff -up ./src/fl_font_xft.cxx.xft ./src/fl_font_xft.cxx
--- ./src/fl_font_xft.cxx.xft 2010-11-29 19:18:27.0 +0100
+++ ./src/fl_font_xft.cxx 2011-01-05 09:47:42.191472266 +0100
@@ -358,6 +358,33 @@ Fl_Font_Descriptor::~Fl_Font_Descriptor(
// XftFontClose(fl_display, font);
}
+static void utf8extents(const char *str, int n, XGlyphInfo *extents)
+{
+
+ wchar_t wstr_buf[1024];
+ wchar_t *wstr;
+ int count;
+
+ memset(extents, 0, sizeof(XGlyphInfo));
+
+ wstr = wstr_buf;
+ count = fl_utf8towc(str, n, wstr, 1024);
+ if (count = 1024) {
+wstr = (wchar_t *)malloc((count + 1) * sizeof(wchar_t));
+count = fl_utf8towc(str, n, wstr, count + 1);
+ }
+
+ if (sizeof(wchar_t) == sizeof(FcChar32))
+XftTextExtents32(fl_display, current_font, (XftChar32 *)wstr, count,
extents);
+ else if (sizeof(wchar_t) == sizeof(FcChar16))
+XftTextExtents16(fl_display, current_font, (XftChar16 *)wstr, count,
extents);
+ else
+XftTextExtentsUtf8(fl_display, current_font, (XftChar8 *)str, n, extents);
+
+ if (wstr != wstr_buf)
+free(wstr);
+}
+
int fl_height() {
if (current_font) return current_font-ascent + current_font-descent;
else return -1;
@@ -371,7 +398,7 @@ int fl_descent() {
double fl_width(const char *str, int n) {
if (!current_font) return -1.0;
XGlyphInfo i;
- XftTextExtentsUtf8(fl_display, current_font, (XftChar8 *)str, n, i);
+ utf8extents(str, n, i);
return i.xOff;
}
@@ -397,7 +424,7 @@ void fl_text_extents(const char *c, int
return;
}
XGlyphInfo gi;
- XftTextExtentsUtf8(fl_display, current_font, (XftChar8 *)c, n, gi);
+ utf8extents(c, n, gi);
w = gi.width;
h = gi.height;
@@ -586,7 +613,26 @@ void Fl_Graphics_Driver::draw(const char
color.color.blue = ((int)b)*0x101;
color.color.alpha = 0x;
- XftDrawStringUtf8(draw_, color, current_font, x, y, (XftChar8 *)str, n);
+ wchar_t wstr_buf[1024];
+ wchar_t *wstr;
+ int count;
+
+ wstr = wstr_buf;
+ count = fl_utf8towc(str, n, wstr, 1024);
+ if (count = 1024) {
+wstr = (wchar_t *)malloc((count + 1) * sizeof(wchar_t));
+count = fl_utf8towc(str, n, wstr, count + 1);
+ }
+
+ if (sizeof(wchar_t) == sizeof(FcChar32))
+XftDrawString32(draw_, color, current_font, x, y, (XftChar32 *)wstr,
count);
+ else if (sizeof(wchar_t) == sizeof(FcChar16))
+XftDrawString16(draw_, color, current_font, x, y, (XftChar16 *)wstr,
count);
+ else
+XftDrawStringUtf8(draw_, color, current_font, x, y, (XftChar8 *)str, n);
+
+ if (wstr != wstr_buf)
+free(wstr);
}
void Fl_Graphics_Driver::draw(int angle, const char *str, int n, int x, int y)
{
___
fltk-bugs mailing list
fltk-bugs@easysw.com
http://lists.easysw.com/mailman/listinfo/fltk-bugs
Re: [fltk.bugs] [LOW] STR #2505: Xft backend doesn't filter bad UTF-8 characters
DO NOT REPLY TO THIS MESSAGE. INSTEAD, POST ANY RESPONSES TO THE LINK BELOW. [STR New] Link: http://www.fltk.org/str.php?L2505 Version: 1.3-current Updated patch that avoids dynamic arrays. I've used the same method as fl_utf8to_mb() that tries to use a statically allocated string first to avoid having to do two passes. Link: http://www.fltk.org/str.php?L2505 Version: 1.3-current ___ fltk-bugs mailing list fltk-bugs@easysw.com http://lists.easysw.com/mailman/listinfo/fltk-bugs
Re: [fltk.bugs] [LOW] STR #2505: Xft backend doesn't filter bad UTF-8 characters
DO NOT REPLY TO THIS MESSAGE. INSTEAD, POST ANY RESPONSES TO THE LINK BELOW. [STR New] Link: http://www.fltk.org/str.php?L2505 Version: 1.3-current Note that some of the other text handling functions do malloc the local string the first time they are called then hang onto the allocation and track its size, and realloc it later if a bigger string needs to be processed. I don't know if this is better or worse than having a large fixed string. Obvioulsy a fixed string might be subject to a buffer overflow, and allocation from the heap may be better than a static string or allocation from the stack... I don't know which I dislike least, actually... I don't *like* either option much! Link: http://www.fltk.org/str.php?L2505 Version: 1.3-current ___ fltk-bugs mailing list fltk-bugs@easysw.com http://lists.easysw.com/mailman/listinfo/fltk-bugs
Re: [fltk.bugs] [LOW] STR #2505: Xft backend doesn't filter bad UTF-8 characters
DO NOT REPLY TO THIS MESSAGE. INSTEAD, POST ANY RESPONSES TO THE LINK BELOW. [STR New] Link: http://www.fltk.org/str.php?L2505 Version: 1.3-current Textual data can enter FLTK widgets by three means: 1) as character strings expressed in the program code (e.g. button labels); 2) from keyboard (or character palette) input; 3) reading an external file with methods insertfile(), appendfile() or loadfile() of Fl_Text_Buffer. It's fair to assume that all strings from case (1) are correct UTF-8. The OS guarantees that strings from case (2) are correct UTF-8 also. All input from case (3) is systematically transformed into valid UTF-8 by the current FLTK 1.3 version regardless of the running OS. This transformation is performed assuming that non-UTF8 data is CP1252 data. Thus, I believe there's no need for further bad UTF-8 handling. Am I wrong ? Link: http://www.fltk.org/str.php?L2505 Version: 1.3-current ___ fltk-bugs mailing list fltk-bugs@easysw.com http://lists.easysw.com/mailman/listinfo/fltk-bugs
Re: [fltk.bugs] [LOW] STR #2505: Xft backend doesn't filter bad UTF-8 characters
DO NOT REPLY TO THIS MESSAGE. INSTEAD, POST ANY RESPONSES TO THE LINK BELOW. [STR New] Link: http://www.fltk.org/str.php?L2505 Version: 1.3-current Aside from the matter of plain ol' bugs, the text displayed in widgets might come from external sources like databases, network communication or read from files in one of the gazilion ways that doesn't have idiot proof encoding handling. There is also the problematic case of file names where there is always a fair amount of ambiguity when it comes to encoding. The simple fact is that UTF-8 has the possibility of being malformed. Allowing the toolkit to handle that and visibly present the presence of bad characters would be an improvement in my book. There is also the matter of consistent behaviour across FLTK platforms. Link: http://www.fltk.org/str.php?L2505 Version: 1.3-current ___ fltk-bugs mailing list fltk-bugs@easysw.com http://lists.easysw.com/mailman/listinfo/fltk-bugs
Re: [fltk.bugs] [LOW] STR #2505: Xft backend doesn't filter bad UTF-8 characters
DO NOT REPLY TO THIS MESSAGE. INSTEAD, POST ANY RESPONSES TO THE LINK BELOW. [STR New] Link: http://www.fltk.org/str.php?L2505 Version: 1.3-current Do you mean than instead of having a byte stream containing a series of valid UTF-8 sequences of header byte and correct number of trailing bytes, you have sequences that contain a header byte and the incorrect number of trailing bytes? Ie, a correct two byte sequence would be 110x 10xx but the stream contains 110x 110y 10yy so the first UTF-8 sequence has been truncated Link: http://www.fltk.org/str.php?L2505 Version: 1.3-current ___ fltk-bugs mailing list fltk-bugs@easysw.com http://lists.easysw.com/mailman/listinfo/fltk-bugs
Re: [fltk.bugs] [LOW] STR #2505: Xft backend doesn't filter bad UTF-8 characters
DO NOT REPLY TO THIS MESSAGE. INSTEAD, POST ANY RESPONSES TO THE LINK BELOW. [STR New] Link: http://www.fltk.org/str.php?L2505 Version: 1.3-current To be honest, I haven't checked the [already existing] code that validates the UTF-8 string. I just noticed it seems to do a good job at catching junk. I would assume that incorrect lengths is the most likely way errors are spotted though. Link: http://www.fltk.org/str.php?L2505 Version: 1.3-current ___ fltk-bugs mailing list fltk-bugs@easysw.com http://lists.easysw.com/mailman/listinfo/fltk-bugs
Re: [fltk.bugs] [LOW] STR #2505: Xft backend doesn't filter bad UTF-8 characters
DO NOT REPLY TO THIS MESSAGE. INSTEAD, POST ANY RESPONSES TO THE LINK BELOW.
[STR New]
Link: http://www.fltk.org/str.php?L2505
Version: 1.3-current
Attached file fltk-1_v2.3.x-xft-check.patch...
Link: http://www.fltk.org/str.php?L2505
Version: 1.3-currentdiff -up ./src/fl_font_xft.cxx.xft ./src/fl_font_xft.cxx
--- ./src/fl_font_xft.cxx.xft 2010-11-29 19:18:27.0 +0100
+++ ./src/fl_font_xft.cxx 2011-01-04 15:04:01.847304494 +0100
@@ -358,6 +358,26 @@ Fl_Font_Descriptor::~Fl_Font_Descriptor(
// XftFontClose(fl_display, font);
}
+static void utf8extents(const char *str, int n, XGlyphInfo *extents)
+{
+
+ wchar_t wstr[n + 1];
+ int count;
+
+ memset(extents, 0, sizeof(XGlyphInfo));
+
+ count = fl_utf8towc(str, n, wstr, sizeof(wstr));
+ if (count (int)sizeof(wstr))
+return;
+
+ if (sizeof(wchar_t) == sizeof(FcChar32))
+XftTextExtents32(fl_display, current_font, (XftChar32 *)wstr, count,
extents);
+ else if (sizeof(wchar_t) == sizeof(FcChar16))
+XftTextExtents16(fl_display, current_font, (XftChar16 *)wstr, count,
extents);
+ else
+XftTextExtentsUtf8(fl_display, current_font, (XftChar8 *)str, n, extents);
+}
+
int fl_height() {
if (current_font) return current_font-ascent + current_font-descent;
else return -1;
@@ -371,7 +391,7 @@ int fl_descent() {
double fl_width(const char *str, int n) {
if (!current_font) return -1.0;
XGlyphInfo i;
- XftTextExtentsUtf8(fl_display, current_font, (XftChar8 *)str, n, i);
+ utf8extents(str, n, i);
return i.xOff;
}
@@ -397,7 +417,7 @@ void fl_text_extents(const char *c, int
return;
}
XGlyphInfo gi;
- XftTextExtentsUtf8(fl_display, current_font, (XftChar8 *)c, n, gi);
+ utf8extents(c, n, gi);
w = gi.width;
h = gi.height;
@@ -586,7 +606,19 @@ void Fl_Graphics_Driver::draw(const char
color.color.blue = ((int)b)*0x101;
color.color.alpha = 0x;
- XftDrawStringUtf8(draw_, color, current_font, x, y, (XftChar8 *)str, n);
+ wchar_t wstr[n + 1];
+ int count;
+
+ count = fl_utf8towc(str, n, wstr, sizeof(wstr));
+ if (count (int)sizeof(wstr))
+return;
+
+ if (sizeof(wchar_t) == sizeof(FcChar32))
+XftDrawString32(draw_, color, current_font, x, y, (XftChar32 *)wstr,
count);
+ else if (sizeof(wchar_t) == sizeof(FcChar16))
+XftDrawString16(draw_, color, current_font, x, y, (XftChar16 *)wstr,
count);
+ else
+XftDrawStringUtf8(draw_, color, current_font, x, y, (XftChar8 *)str, n);
}
void Fl_Graphics_Driver::draw(int angle, const char *str, int n, int x, int y)
{
___
fltk-bugs mailing list
fltk-bugs@easysw.com
http://lists.easysw.com/mailman/listinfo/fltk-bugs
Re: [fltk.bugs] [LOW] STR #2505: Xft backend doesn't filter bad UTF-8 characters
DO NOT REPLY TO THIS MESSAGE. INSTEAD, POST ANY RESPONSES TO THE LINK BELOW. [STR New] Link: http://www.fltk.org/str.php?L2505 Version: 1.3-current Updated patch that also handles text measurements properly. Rendering of stuff like selections got a bit funky otherwise. Link: http://www.fltk.org/str.php?L2505 Version: 1.3-current ___ fltk-bugs mailing list fltk-bugs@easysw.com http://lists.easysw.com/mailman/listinfo/fltk-bugs
Re: [fltk.bugs] [LOW] STR #2505: Xft backend doesn't filter bad UTF-8 characters
DO NOT REPLY TO THIS MESSAGE. INSTEAD, POST ANY RESPONSES TO THE LINK BELOW. [STR New] Link: http://www.fltk.org/str.php?L2505 Version: 1.3-current I am impartial to this. On the one hand, we never check parameters for validity to keep the library fast. OTOH, we *do* put quite an effort into converting text in MSWindows and in OS X, so I guess doing this in X11 is just as fine. Any Unix users who could please verify and apply this change? Link: http://www.fltk.org/str.php?L2505 Version: 1.3-current ___ fltk-bugs mailing list fltk-bugs@easysw.com http://lists.easysw.com/mailman/listinfo/fltk-bugs
Re: [fltk.bugs] [LOW] STR #2505: Xft backend doesn't filter bad UTF-8 characters
DO NOT REPLY TO THIS MESSAGE. INSTEAD, POST ANY RESPONSES TO THE LINK BELOW. [STR New] Link: http://www.fltk.org/str.php?L2505 Version: 1.3-current Yes, I think that's right. The OP wrote: ... by doing a UTF-8 to UCS-2/UCS-4 conversion, which has the side effect of filtering the string. That's true, and I tested it with an application that uses ISO-8859-1 for all data (text) internally. The appearance is much more like it is with the Windows version. But there's a difference: on Windows we _must_ convert all strings from UTF-8 to the hosts wchar_t format to do Unicode rendering, on Linux we don't need to do it (basically). So what happens is that we lose speed for something we don't need to do if all text is correct (UTF-8). :-( OTOH, as mentioned above, we _do_ convert all strings on Windows, and this has the same speed loss, so it might be negligible. Maybe? My tests showed some problems with rendering of non-UTF-8 text (similar to misc/iso-8859-1.txt), so I'll need more time to test... Technical remarks to the provided patch: - As seen recently, we must not allocate dynamic arrays like wchar_t wstr[n + 1]; - we must malloc() or new (and free/delete) dynamically sized data for compatibility reasons with old compilers. - As Ian said, the runtime sizeof() comparisons are probably not needed, and we don't need to convert the string if we use it later in the else case as-is. That should be improved, if we agree to apply it. Link: http://www.fltk.org/str.php?L2505 Version: 1.3-current ___ fltk-bugs mailing list fltk-bugs@easysw.com http://lists.easysw.com/mailman/listinfo/fltk-bugs
[fltk.bugs] [LOW] STR #2505: Xft backend doesn't filter bad UTF-8 characters
DO NOT REPLY TO THIS MESSAGE. INSTEAD, POST ANY RESPONSES TO THE LINK BELOW.
[STR New]
Link: http://www.fltk.org/str.php?L2505
Version: 1.3-current
The Windows backend handles bad UTF-8 sequences by either replacing them
with the replacement character, or guesses that they are ISO
8859-1/CP1252. No such handling is done in the Xft backend though. It
would be nice if this is done for the features above, and for consistency
across platforms if nothing else.
The attached patch changes this by doing a UTF-8 to UCS-2/UCS-4
conversion, which has the side effect of filtering the string.
Link: http://www.fltk.org/str.php?L2505
Version: 1.3-currentdiff -up ./src/fl_font_xft.cxx.xft ./src/fl_font_xft.cxx
--- ./src/fl_font_xft.cxx.xft 2010-11-29 19:18:27.0 +0100
+++ ./src/fl_font_xft.cxx 2011-01-03 11:52:18.017946001 +0100
@@ -586,7 +586,19 @@ void Fl_Graphics_Driver::draw(const char
color.color.blue = ((int)b)*0x101;
color.color.alpha = 0x;
- XftDrawStringUtf8(draw_, color, current_font, x, y, (XftChar8 *)str, n);
+ wchar_t wstr[n + 1];
+ int count;
+
+ count = fl_utf8towc(str, n, wstr, sizeof(wstr));
+ if (count sizeof(wstr))
+return;
+
+ if (sizeof(wchar_t) == sizeof(FcChar32))
+XftDrawString32(draw_, color, current_font, x, y, (XftChar32 *)wstr,
count);
+ else if (sizeof(wchar_t) == sizeof(FcChar16))
+XftDrawString16(draw_, color, current_font, x, y, (XftChar16 *)wstr,
count);
+ else
+XftDrawStringUtf8(draw_, color, current_font, x, y, (XftChar8 *)str, n);
}
void Fl_Graphics_Driver::draw(int angle, const char *str, int n, int x, int y)
{
___
fltk-bugs mailing list
fltk-bugs@easysw.com
http://lists.easysw.com/mailman/listinfo/fltk-bugs
