Re[2]: [fpc-devel] UTF8Encode widestring encoding
Hello Sergei, Tuesday, August 26, 2008, 4:16:00 PM, you wrote: SG> Graeme Geldenhuys wrote: >> >> I couldn't find a single program or editor that could display that >> file correctly. Pretty amazing that one file can defeat all programs. >> :-) >> >> Programs I tried: >> Kate (kde editor) >> gEdit (Gnome editor) >> Firefox 3 (did the best out of the lot) >> vi (disastrous) >> OpenOffice 2.3 (probably the best out of the lot) >> AbiWord 2.4.6 >> >> >> Do you know of any program that passes that test? >> SG> I viewed that file with Opera 9.21 and it looked fine for me (all SG> out-of-range chars replaced by a triangle symbol), although the question SG> how it *should* look remains open. Opera 9.50 (Windows) fails for 5.3.x tests. Also it seens to fail at 2.2.2 eating the " simbol after the character, but it could be a render error due the carrier error at 2.1.1 rendering NULL character. Even I do not know it is should be considered OK the 2 chars render of 2.1.4 and 2.3.4, but again seens to be a render problem, not a decode error. -- Best regards, JoshyFun ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] UTF8Encode widestring encoding
On 8/26/08, Sergei Gorelkin <[EMAIL PROTECTED]> wrote: > > > I viewed that file with Opera 9.21 and it looked fine for me (all > out-of-range chars replaced by a triangle symbol), although the question how > it *should* look remains open. You are right. :-) Just tried Opera 9.5 (after setting my fonts to mono). It looks by far the best and only tests 2.1.x and 2.2.x fails - right hand column must be in line except for 2.1.1 and I think 2.2.1 Regards, - Graeme - ___ fpGUI - a cross-platform Free Pascal GUI toolkit http://opensoft.homeip.net/fpgui/ ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re[2]: [fpc-devel] UTF8Encode widestring encoding
Hello Graeme, Tuesday, August 26, 2008, 3:48:39 PM, you wrote: GG> That doesn't say much... :-) It's the usual status :) most ones are focused in speed instead given a bit of security, special against overlong sequences which can bypass security barriers... check the attached file and decode it with FPC UTF8Decode ;) and DO NOT RUN IT! ;) GG> I couldn't find a single program or editor that could display that GG> file correctly. Pretty amazing that one file can defeat all programs. GG> :-) GG> Programs I tried: GG> Kate (kde editor) GG> gEdit (Gnome editor) GG> Firefox 3 (did the best out of the lot) GG> vi (disastrous) GG> OpenOffice 2.3 (probably the best out of the lot) GG> AbiWord 2.4.6 GG> Do you know of any program that passes that test? Nope, I had checked some of them and Firefox is maybe the better. You can also try the UTF8ToUnicode that I'd posted in the bug filled. -- Best regards, JoshyFunrm "/home/userÀ¯..À¯..À¯*" À¯r ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] UTF8Encode widestring encoding
Graeme Geldenhuys wrote: I couldn't find a single program or editor that could display that file correctly. Pretty amazing that one file can defeat all programs. :-) Programs I tried: Kate (kde editor) gEdit (Gnome editor) Firefox 3 (did the best out of the lot) vi (disastrous) OpenOffice 2.3 (probably the best out of the lot) AbiWord 2.4.6 Do you know of any program that passes that test? I viewed that file with Opera 9.21 and it looked fine for me (all out-of-range chars replaced by a triangle symbol), although the question how it *should* look remains open. Regards, Sergei ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] UTF8Encode widestring encoding
On 8/26/08, JoshyFun <[EMAIL PROTECTED]> wrote: > > http://bugs.freepascal.org/view.php?id=11791 > > The LCL implementation of UTF8ToUTF16 (default one) does not pass the > UTF8 stress test. That doesn't say much... :-) I couldn't find a single program or editor that could display that file correctly. Pretty amazing that one file can defeat all programs. :-) Programs I tried: Kate (kde editor) gEdit (Gnome editor) Firefox 3 (did the best out of the lot) vi (disastrous) OpenOffice 2.3 (probably the best out of the lot) AbiWord 2.4.6 Do you know of any program that passes that test? Regards, - Graeme - ___ fpGUI - a cross-platform Free Pascal GUI toolkit http://opensoft.homeip.net/fpgui/ ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] UTF8Encode widestring encoding
Hello Felipe, Tuesday, August 26, 2008, 12:28:45 PM, you wrote: FMdC> I read the code for UTF8Encode and UTF8Decode routines and they seam FMdC> to suppose that the widestring encoding is UCS-2! Instead of UTF-16 FMdC> Is this the expected behavior or is it only partially implemented? FMdC> Someone added UTF8 <-> UTF16 conversion routines in the LCL, so they FMdC> could be used to repair UTF8Encode / Decode. http://bugs.freepascal.org/view.php?id=11791 The LCL implementation of UTF8ToUTF16 (default one) does not pass the UTF8 stress test. My version does not handle UTF16 intentionally as the proposed replace function can not make grow the widestring which comes allocated, but... hmmm... oh! it comes at 2*bytes in UTF8, so there must be space always to fit any character even the doubles ones. :) It's time to add the UTF16 to my code. -- Best regards, JoshyFun ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] UTF8Encode widestring encoding
Op Tue, 26 Aug 2008, schreef Felipe Monteiro de Carvalho: Hello, I read the code for UTF8Encode and UTF8Decode routines and they seam to suppose that the widestring encoding is UCS-2! Instead of UTF-16 Is this the expected behavior or is it only partially implemented? It is a broken implementation, it should be fixed. Daniël___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] UTF8Encode widestring encoding
Hello, I read the code for UTF8Encode and UTF8Decode routines and they seam to suppose that the widestring encoding is UCS-2! Instead of UTF-16 Is this the expected behavior or is it only partially implemented? Someone added UTF8 <-> UTF16 conversion routines in the LCL, so they could be used to repair UTF8Encode / Decode. thanks, -- Felipe Monteiro de Carvalho ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
