Re: [fpc-devel] Unicode support in RTL - Roadmap

2008-11-21 Thread Luiz Americo Pereira Camara
Graeme Geldenhuys escreveu: Hi, I have added a Roadmap section in the following wiki page. If you find anything missing or not 100% implemented, please add it to the wiki page. http://wiki.freepascal.org/FPC_Unicode_support#Roadmap_of_RTL_Unicode_support I started a wiki page to list the u

Re: [fpc-devel] Re: new 27 page document describing Unicode support in D2009

2008-11-21 Thread Luiz Americo Pereira Camara
Sergei Gorelkin escreveu: Well, with exclusion of the "class helper for TStrings" (notable is that they call it a hack themselves :) the design looks rather clean. Since each string stores its element size, both ansi and unicode strings are probably handled with common set of procedures, avoi

Re: [fpc-devel] Re: new 27 page document describing Unicode support in D2009

2008-11-21 Thread Sergei Gorelkin
Graeme Geldenhuys wrote: On Fri, Nov 21, 2008 at 11:08 PM, Graeme Geldenhuys <[EMAIL PROTECTED]> wrote: I thought you guys might find this interesting. It's a new 27 page document describing Unicode support in D2009. http://dn.codegear.com/article/38980 Seeing that I don't own D2009 and only

Re: [fpc-devel] Re: new 27 page document describing Unicode support in D2009

2008-11-21 Thread Luiz Americo Pereira Camara
Graeme Geldenhuys escreveu: On Fri, Nov 21, 2008 at 11:08 PM, Graeme Geldenhuys <[EMAIL PROTECTED]> wrote: I thought you guys might find this interesting. It's a new 27 page document describing Unicode support in D2009. http://dn.codegear.com/article/38980 Seeing that I don't own D200

Re: [fpc-devel] Re: new 27 page document describing Unicode support in D2009

2008-11-21 Thread Marco van de Voort
In our previous episode, Graeme Geldenhuys said: > > I thought you guys might find this interesting. It's a new 27 page > > document describing Unicode support in D2009. > > > > http://dn.codegear.com/article/38980 > > Seeing that I don't own D2009 and only read about it's Unicode support > I foun

[fpc-devel] Re: new 27 page document describing Unicode support in D2009

2008-11-21 Thread Graeme Geldenhuys
On Fri, Nov 21, 2008 at 11:08 PM, Graeme Geldenhuys <[EMAIL PROTECTED]> wrote: > > I thought you guys might find this interesting. It's a new 27 page > document describing Unicode support in D2009. > > http://dn.codegear.com/article/38980 Seeing that I don't own D2009 and only read about it's Unic

[fpc-devel] new 27 page document describing Unicode support in D2009

2008-11-21 Thread Graeme Geldenhuys
Hello, I thought you guys might find this interesting. It's a new 27 page document describing Unicode support in D2009. http://dn.codegear.com/article/38980 -- Abstract: Learn more about the new Unicode support in Delphi 2009 and CodeGear RAD Studio 2009 in this white

Re: [fpc-devel] wrong rtti default value in the fixes_2_2 branch (dont know about trunk)

2008-11-21 Thread Michael Van Canneyt
On Fri, 21 Nov 2008, Paul Ishenin wrote: > Michael Van Canneyt wrote: > > I fixed the bug in trunk. Please do some tests in Lazarus with the 12114 > > revision of the compiler. If all works still OK and the testsuites don't > > give any regressions, I'll merge it to the fix branch. > > > Here

Re: [fpc-devel] Unicode support in RTL - Roadmap

2008-11-21 Thread Martin Friebe
Felipe Monteiro de Carvalho wrote: On Fri, Nov 21, 2008 at 2:42 PM, Michael Schnell <[EMAIL PROTECTED]> wrote: And thus forces all users to "understand the full UTF-8 spec" and to rewrite their programs, even though the old code perfectly compiles and up to a certain extent seems to work. Th

Re: [fpc-devel] Unicode support in RTL - Roadmap

2008-11-21 Thread Felipe Monteiro de Carvalho
On Fri, Nov 21, 2008 at 2:42 PM, Michael Schnell <[EMAIL PROTECTED]> wrote: > And thus forces all users to "understand the full UTF-8 spec" and to rewrite > their programs, even though the old code perfectly compiles and up to a > certain extent seems to work. > > This is what I think is "not at al

Re: [fpc-devel] Unicode support in RTL - Roadmap

2008-11-21 Thread Yury Sidorov
From: "Florian Klaempfl" <[EMAIL PROTECTED]> Folks, before your waste your time again with endless discussions, have a look at Yury's work on an unicode rtl, test it and help with patches and suggestions, it's available in svn at http://svn.freepascal.org/svn/fpc/branches/unicodertl It is wor

Re: [fpc-devel] Unicode support in RTL - Roadmap

2008-11-21 Thread Michael Schnell
Legacy code that assumes ASCII can be used in UTF-8. Code that needs to deal with higher code points needs to be rewritten This is any Program that formerly used (ANSIS) String and now is automatically converted to use UTF8 and that is to be released in Germany, France With that of co

Re: [fpc-devel] Unicode support in RTL - Roadmap

2008-11-21 Thread Florian Klaempfl
Folks, before your waste your time again with endless discussions, have a look at Yury's work on an unicode rtl, test it and help with patches and suggestions, it's available in svn at http://svn.freepascal.org/svn/fpc/branches/unicodertl ___ fpc-devel ma

Re: [fpc-devel] Unicode support in RTL - Roadmap

2008-11-21 Thread Daniël Mantione
Op Fri, 21 Nov 2008, schreef Michael Schnell: If your point is that there is no way to allow for legacy code to be used with a "String" type that holds UTF8 code and that it is not possible (or desirable) to allow for code used in simple occasions that is understandable to someone who does n

Re: [fpc-devel] Unicode support in RTL - Roadmap

2008-11-21 Thread Michael Schnell
If your point is that there is no way to allow for legacy code to be used with a "String" type that holds UTF8 code and that it is not possible (or desirable) to allow for code used in simple occasions that is understandable to someone who does not want to go into the complete depth of the UTF8

Re: [fpc-devel] Unicode support in RTL - Roadmap

2008-11-21 Thread Jonas Maebe
On 21 Nov 2008, at 16:16, Michael Schnell wrote: So UTF8ElementlLength('Ü') would be 2 and UTF8PointLength('Ü') would be 1. Or 2, depending on whether it's predcomposed or decomposed. I seem to remember that we discussed this some time ago and the result was that the compose (MAC style ?)

Re: [fpc-devel] Unicode support in RTL - Roadmap

2008-11-21 Thread Michael Schnell
you also cannot freely change the return value of Pos() from elements to codepoints. Of course the counting needs to be consistent for all string functions. So changing it "on the fly" is dangerous (if you keep a count value in an integer variable). But this is up to the user. -Michael

Re: [fpc-devel] Unicode support in RTL - Roadmap

2008-11-21 Thread Michael Schnell
So UTF8ElementlLength('Ü') would be 2 and UTF8PointLength('Ü') would be 1. Or 2, depending on whether it's predcomposed or decomposed. I seem to remember that we discussed this some time ago and the result was that the compose (MAC style ?) characters in fact are a single code point (Unicode

Re: [fpc-devel] Unicode support in RTL - Roadmap

2008-11-21 Thread Sergei Gorelkin
Michael Schnell wrote: I don't really understand your question. I think would would need to have two different function UTF8ElementlLength(UTF8String) and UTF8PointLength(UTF8String), first giving the string length in code elements (byte) and second giving the length in code points (unicode

Re: [fpc-devel] Unicode support in RTL - Roadmap

2008-11-21 Thread Jonas Maebe
On 21 Nov 2008, at 14:50, Michael Schnell wrote: If Length() would return its value in chars, what length in *bytes* would the following call set: SetLength(utfstring_1), Length(utfstring_2)); I don't really understand your question. I think would would need to have two different function

Re: [fpc-devel] Unicode support in RTL - Roadmap

2008-11-21 Thread Michael Schnell
If Length() would return its value in chars, what length in *bytes* would the following call set: SetLength(utfstring_1), Length(utfstring_2)); I don't really understand your question. I think would would need to have two different function UTF8ElementlLength(UTF8String) and UTF8PointL

Re: [fpc-devel] Unicode support in RTL - Roadmap

2008-11-21 Thread Sergei Gorelkin
Michael Schnell wrote: I prefer it to be counted in bytes. If it is counted in Bytes then I can build a routine that counts in real chars. And we already have a lot of code to handle utf-8 inside ansisstring which depends on that. Counting the elements in real chars is very ineficient. This

Re: [fpc-devel] wrong rtti default value in the fixes_2_2 branch (dont know about trunk)

2008-11-21 Thread Michael Van Canneyt
On Fri, 21 Nov 2008, Paul Ishenin wrote: > Michael Van Canneyt wrote: > > I fixed the bug in trunk. Please do some tests in Lazarus with the 12114 > > revision of the compiler. If all works still OK and the testsuites don't > > give any regressions, I'll merge it to the fix branch. > > > Here

Re: [fpc-devel] wrong rtti default value in the fixes_2_2 branch (dont know about trunk)

2008-11-21 Thread Paul Ishenin
Michael Van Canneyt wrote: I fixed the bug in trunk. Please do some tests in Lazarus with the 12114 revision of the compiler. If all works still OK and the testsuites don't give any regressions, I'll merge it to the fix branch. Here nothing bad happen - at least I had not note. If you have no

Re: [fpc-devel] Unicode and Lazarus

2008-11-21 Thread Jonas Maebe
On 21 Nov 2008, at 10:51, Felipe Monteiro de Carvalho wrote: It would be really good if there was a guide, preferably in the wiki, to explain how to add a new test case to Free Pascal. It is documented here: http://svn.freepascal.org/cgi-bin/viewvc.cgi/trunk/tests/readme.txt?view=markup You

Re: [fpc-devel] Unicode and Lazarus

2008-11-21 Thread Graeme Geldenhuys
On Fri, Nov 21, 2008 at 12:47 PM, Mattias Gärtner <[EMAIL PROTECTED]> wrote: > > ... and a possibility to tell the compiler 'Thanks, I know. Don't bark about > this place any longer'. :-) Yes definately! Like the wish for "Parameter not being used" or "Sender not being user" etc... Those drive me

Re: [fpc-devel] Unicode and Lazarus

2008-11-21 Thread Mattias Gärtner
Zitat von Graeme Geldenhuys <[EMAIL PROTECTED]>: > On Fri, Nov 21, 2008 at 11:45 AM, Michael Schnell <[EMAIL PROTECTED]> wrote: > > > > In fact it should automatically convert ("as correctly as possible") > between > > all available string types (ANSI, UTF8, UTF16). > > And the compiler should pro

Re: [fpc-devel] Unicode support in RTL - Roadmap

2008-11-21 Thread Michael Schnell
I only added the roadmap section, the rest of the content existed before. You are welcome to amend the content. I'd rightfully be severely bashed by those who actually will be required to do the work ;) . -Michael ___ fpc-devel maillist - fpc-d

Re: [fpc-devel] Unicode and Lazarus

2008-11-21 Thread Graeme Geldenhuys
On Fri, Nov 21, 2008 at 11:51 AM, Felipe Monteiro de Carvalho <[EMAIL PROTECTED]> wrote: > > It would be really good if there was a guide, preferably in the wiki, > to explain how to add a new test case to Free Pascal. > > I have already some test cases in mind, like making sure static > methods co

Re: [fpc-devel] Unicode and Lazarus

2008-11-21 Thread Graeme Geldenhuys
On Fri, Nov 21, 2008 at 11:45 AM, Michael Schnell <[EMAIL PROTECTED]> wrote: > > In fact it should automatically convert ("as correctly as possible") between > all available string types (ANSI, UTF8, UTF16). And the compiler should produce a warning if you assign UTF8 or UTF16 string to a ANSI str

Re: [fpc-devel] Unicode support in RTL - Roadmap

2008-11-21 Thread Graeme Geldenhuys
On Fri, Nov 21, 2008 at 11:30 AM, Michael Schnell <[EMAIL PROTECTED]> wrote: > >> http://wiki.freepascal.org/FPC_Unicode_support#Roadmap_of_RTL_Unicode_support > > This page does not talk about UTF8Strings being counted in code elements vs > in code points. I only added the roadmap section, the re

Re: [fpc-devel] Unicode support - for the 20th time... ;-)

2008-11-21 Thread Marco van de Voort
In our previous episode, Felipe Monteiro de Carvalho said: > > When Marco said "yet another 2.2 fixes branch release", he meant 2.2.6. > > Ah, ok ... =) > > So my commend would then be changed to: > > Unicode is what is most discussed and needed at the moment. What is > the point in making a maj

Re: [fpc-devel] Unicode and Lazarus

2008-11-21 Thread Felipe Monteiro de Carvalho
On Fri, Nov 21, 2008 at 7:49 AM, Florian Klaempfl <[EMAIL PROTECTED]> wrote: > Big deal, I simply enable operator overloading for unique string types > to get this working, then everybody is happy and we've unicode support? Indeed that could work. But the operator overloading would need to overrid

Re: [fpc-devel] Unicode and Lazarus

2008-11-21 Thread Florian Klaempfl
Felipe Monteiro de Carvalho schrieb: > On Fri, Nov 21, 2008 at 7:30 AM, Florian Klaempfl > <[EMAIL PROTECTED]> wrote: >> This is easily said, please create examples and descriptions how fully >> working is defined. > > It would be really good if there was a guide, preferably in the wiki, > to expl

Re: [fpc-devel] Unicode support in RTL - Roadmap

2008-11-21 Thread Michael Schnell
I prefer it to be counted in bytes. If it is counted in Bytes then I can build a routine that counts in real chars. And we already have a lot of code to handle utf-8 inside ansisstring which depends on that. Counting the elements in real chars is very ineficient. This is commonly agreed, But

Re: [fpc-devel] Unicode and Lazarus

2008-11-21 Thread Felipe Monteiro de Carvalho
On Fri, Nov 21, 2008 at 7:30 AM, Florian Klaempfl <[EMAIL PROTECTED]> wrote: > This is easily said, please create examples and descriptions how fully > working is defined. It would be really good if there was a guide, preferably in the wiki, to explain how to add a new test case to Free Pascal. I

Re: [fpc-devel] Unicode and Lazarus

2008-11-21 Thread Florian Klaempfl
Felipe Monteiro de Carvalho schrieb: > On Fri, Nov 21, 2008 at 7:30 AM, Florian Klaempfl > <[EMAIL PROTECTED]> wrote: >> This is easily said, please create examples and descriptions how fully >> working is defined. > > // Should actually convert from widestring to utf-8 when using encoding utf-8 >

Re: [fpc-devel] Unicode and Lazarus

2008-11-21 Thread Michael Schnell
Of course correct automatic assigning of strings constants defined in the source file correctly regarding the coding of the source file (e.g. UTF8 or ANSI) Maybe here the coding of the source file needs actively be communicated to the compiler. Of course assigning a constant should not do a co

Re: [fpc-devel] Unicode support - for the 20th time... ;-)

2008-11-21 Thread Felipe Monteiro de Carvalho
On Fri, Nov 21, 2008 at 7:43 AM, Vincent Snijders <[EMAIL PROTECTED]> wrote: > When Marco said "yet another 2.2 fixes branch release", he meant 2.2.6. Ah, ok ... =) So my commend would then be changed to: Unicode is what is most discussed and needed at the moment. What is the point in making a m

Re: [fpc-devel] Unicode and Lazarus

2008-11-21 Thread Michael Schnell
// Should actually convert from widestring to utf-8 when using encoding utf-8 programa utf8test1; In fact it should automatically convert ("as correctly as possible") between all available string types (ANSI, UTF8, UTF16). Should provide appropriate char types for all available string type

Re: [fpc-devel] Unicode support in RTL - Roadmap

2008-11-21 Thread Felipe Monteiro de Carvalho
On Fri, Nov 21, 2008 at 7:30 AM, Michael Schnell <[EMAIL PROTECTED]> wrote: > This page does not talk about UTF8Strings being counted in code elements vs > in code points. > > I don't consider it understood that they in any case are counted in code > elements. IMHO this should be seriously discusse

Re: [fpc-devel] Unicode support - for the 20th time... ;-)

2008-11-21 Thread Vincent Snijders
Felipe Monteiro de Carvalho schreef: On Fri, Nov 21, 2008 at 7:01 AM, Marco van de Voort <[EMAIL PROTECTED]> wrote: Is it? Because that might mean yet another 2.2 fixes branch release to fix up the delay that this will cause to 2.4 Another 2.2 fixes branch release is a good idea, because it co

Re: [fpc-devel] Unicode support - for the 20th time... ;-)

2008-11-21 Thread Felipe Monteiro de Carvalho
On Fri, Nov 21, 2008 at 7:01 AM, Marco van de Voort <[EMAIL PROTECTED]> wrote: > Is it? Because that might mean yet another 2.2 fixes branch release to fix > up the delay that this will cause to 2.4 Another 2.2 fixes branch release is a good idea, because it contains a fix for static methods which

Re: [fpc-devel] Unicode and Lazarus

2008-11-21 Thread Felipe Monteiro de Carvalho
On Fri, Nov 21, 2008 at 7:30 AM, Florian Klaempfl <[EMAIL PROTECTED]> wrote: > This is easily said, please create examples and descriptions how fully > working is defined. // Should actually convert from widestring to utf-8 when using encoding utf-8 programa utf8test1; {$encoding utf-8} // or is

Re: [fpc-devel] Unicode and Lazarus

2008-11-21 Thread Daniël Mantione
Op Fri, 21 Nov 2008, schreef Felipe Monteiro de Carvalho: On Thu, Nov 20, 2008 at 9:05 AM, Daniël Mantione <[EMAIL PROTECTED]> wrote: There will be a real UTF8string, i.e. ansistring with UTF-8 encoding as part of type information, this will help Lazarus users to get rid of the utf8encode/utf

Re: [fpc-devel] UnicodeString and RTL

2008-11-21 Thread Michael Schnell
I'm new to this, but as far as I understand, WideString is for backward compatibility on Windows olestr type. It's not reference counted. Whereas UnicodeString type is reference counted on all platforms. I found the paragraph in the Wiki :) . I wonder how we should deal with not reference c

Re: [fpc-devel] Unicode and Lazarus

2008-11-21 Thread Felipe Monteiro de Carvalho
On Thu, Nov 20, 2008 at 9:05 AM, Daniël Mantione <[EMAIL PROTECTED]> wrote: > There will be a real UTF8string, i.e. ansistring with UTF-8 encoding as part > of type information, this will help Lazarus users to get rid of the > utf8encode/utf8decode. When? Is this planned for 2.4? thanks, -- Feli

Re: [fpc-devel] Unicode and Lazarus

2008-11-21 Thread Florian Klaempfl
Felipe Monteiro de Carvalho schrieb: > On Thu, Nov 20, 2008 at 9:05 AM, Daniël Mantione > <[EMAIL PROTECTED]> wrote: >> On the other hand Lazarus may want to move to a "string depending on >> platform" too, to attract both Delphi < 2009 and Delphi >= 2009 users. > > I don't see this change any tim

Re: [fpc-devel] Re: Unicode and Lazarus

2008-11-21 Thread Felipe Monteiro de Carvalho
On Thu, Nov 20, 2008 at 9:09 AM, Mattias Gärtner <[EMAIL PROTECTED]> wrote: > So the roadmap from LCL pov is: > - a RTL using unicode strings > - changing the string types in the lazarus code > - a fpc release with the unicode RTL >From what I've heard about the Unicode RTL fpc developers recomend

Re: [fpc-devel] Unicode support in RTL - Roadmap

2008-11-21 Thread Michael Schnell
http://wiki.freepascal.org/FPC_Unicode_support#Roadmap_of_RTL_Unicode_support This page does not talk about UTF8Strings being counted in code elements vs in code points. I don't consider it understood that they in any case are counted in code elements. IMHO this should be seriously disc

Re: [fpc-devel] Unicode and Lazarus

2008-11-21 Thread Felipe Monteiro de Carvalho
On Thu, Nov 20, 2008 at 9:05 AM, Daniël Mantione <[EMAIL PROTECTED]> wrote: > On the other hand Lazarus may want to move to a "string depending on > platform" too, to attract both Delphi < 2009 and Delphi >= 2009 users. I don't see this change any time soon, because it would break too much of exis

Re: [fpc-devel] Unicode support - for the 20th time... ;-)

2008-11-21 Thread Marco van de Voort
In our previous episode, Dani?l Mantione said: > >> If you want to help, we need to implement the Delphi 2009 encoding aware > >> string type, both runtime support as well as the compiler support. > > A previous discussion showed that this also breaks a lot of old code and is > > not really nice.

Re: [fpc-devel] Unicode support - for the 20th time... ;-)

2008-11-21 Thread Daniël Mantione
Op Fri, 21 Nov 2008, schreef Marco van de Voort: In our previous episode, Dani?l Mantione said: Full Unicode support is for FPC 2.4. If you need it today, widestrings are your best option. Is it? Because that might mean yet another 2.2 fixes branch release to fix up the delay that this will

Re: [fpc-devel] Unicode support - for the 20th time... ;-)

2008-11-21 Thread Marco van de Voort
In our previous episode, Florian Klaempfl said: > They add it only because they insist on using utf-8 :) That's perfectly normal on *nix. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel

Re: [fpc-devel] Unicode support - for the 20th time... ;-)

2008-11-21 Thread Marco van de Voort
In our previous episode, Dani?l Mantione said: > Full Unicode support is for FPC 2.4. If you need it today, widestrings are > your best option. Is it? Because that might mean yet another 2.2 fixes branch release to fix up the delay that this will cause to 2.4 _