Re: [fpc-devel] indexed property with default value for param?
Daniël Mantione wrote:
Op Thu, 11 Jun 2009, schreef Michael Van Canneyt:
On Thu, 11 Jun 2009, Jonas Maebe wrote:
I would think it is a natural extension of procedural overloading ?
It's not that simple. Properties can be both hidden variable access as
well as procedure access. Variable overloading is not allowed, because
it causes a lot of issues, i.e.:
var a:word;
a:char;
begin
a:=1;
a:='!';
writeln(a); {what does this mean??}
end;
Property overloading gets you the same issues, i.e. in the example
above you could replace both variables with properties and answer the
same question.
Considering this, I do not think it can considered a logical extension
of procedure overloading.
Good point.
However the example above (even so you use the setter [assign to the
property] instead of reading), is the same as overloading *functions* by
result.
You can not overload functions by result; still you can overload them by
arguments.
So consistency of overloading functions is *not* defined by being able
to use every bit of information on the function for the overloading; but
only selected (the parameters passed to the function).
On properties this question was about the index given to a property
(number and type of indexes). I do agree, it can not be about the type
of the property itself.
Martin
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] indexed property with default value for param?
Michael Van Canneyt wrote: On Wed, 10 Jun 2009, Martin wrote: However the follwing Syntax compiles fine: function GetFoo(Bar: integer; XY: Boolean = True): Integer; property Foo[Bar: integer; XY: Boolean] read GetFoo; a := Object.Foo[1]; But: Is that supposed to work? No, it is not. It is a compiler bug. Could you please create a bug report? Done. However it would be nice to have a feature like property Foo[Index: Integer]: Integer write SetFoo read GetFoo; property Foo[Index: Integer; Ask: Boolean]: Integer write SetFooAsk read GetFooAsk; The must have different Method Names, because the Setters(write SetFoo) last argument is the Value, so it' can not have a optional Ask element: procedure WrongSetFoo(Index: Integer; Ask: Boolean= True; AValue: Integer); But with different Setter/Getters that would not be a problem. In fact they can have the same name, if they are overloaded. Any idea if such a feature would have a chance? Best Regards Martin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] local "class of" error?
Not sure if this is intended, or just a wrong assumptions by the parser? Procedure foo; type TSomeClass = Class of TSome; . gives the error: Error: Local class definitions are not allowed But there is no Class declaration. (Not sure what you call "class of") TSome is declared elsewhere already. Hence my question: Is this intended to give an error? (or does the parser just see class and makes the assumption a new class will be declared?) Best Regards Martin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Delphi smart-linking perspective ( Language change and implementation ... )
Marc Weustink wrote: Marco van de Voort wrote: In our previous episode, Florian Klaempfl said: Also note that WPO can do little about published methods (since it must assume that these can all be called via RTTI). Maybe the IDE can feedback some info about used classes and methods? In time, a pascal analyser would be a good thing to have. I think analysis of smartlinking is more something for an analyzer than the compiler itself (though the compiler could generate certain datafiles obviously). This won't help lazarus. For lazarus, also the contents of the lfms is important. Yeah, I got it when I saw Jonas' reply. The IDE could list all visual components in forms in a file and hand it to the compiler. And a "custom" list could allow additional streaming types (for users code) All components are already sortof reported :) In order to get streamed, each component must have a member variable in the owning form. However what is needed is a list of properties touched in the lfm (setting control.caption or control.font.color for instance). Unfortunately, thanks to RTTI this isn't enough. Because any published property can be changed to any value, even: - if it is not referred in the code - if it is not in any LFM The code could read some kind of .ini file (or worse ask the user to input a property name and value (ouch)) and use RTTI to find and set the property. As a result, any code that can be triggered by a puplished property must be included. This means: - drag and dock manager (since they are enabled by setting a boolean property) - probably a lot of image code - many others The only way round this, is to break compatibility with the VCL. If drag/dock (just a random picked example) were components, that must be put on the form or otherwise can not be used, then they would not need to be included. With graphics a lot of image-format readers are registered, if the unit is used. Again the issue is to find if they could be used in some automated way... Martin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] bug in 2.3.1 with -O2 -Os
Just want to draw attention to http://bugs.freepascal.org/view.php?id=13484 I don't know any more than is said in the report (the problem still exists, despite it being closed). Maybe some one has any idea about it. Feel free to reopen and move the issue to the fpc section of mantis. Best regards Martin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] virtual methods and smart-linking?
Just a question. Can smart linking detect and remove unused virtual methods? I did a few tests and it seems it does not remove them. If I understand it correctly, then this could be because virtual methods are nver directly linked (they go via the VMT) so the linker would not know? Best Regards Martin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] unit imm.pp on windows
Hi,
does anyone know if a unit imm.pp (see below) was ever part of FPC?
(See http://bugs.freepascal.org/view.php?id=13140 )
If it was, what was the reason for it's removal?
Martin
head of imm.pp:
{
This file is part of the Free Pascal run time library.
Copyright (c) 2005 by the Free Pascal development team
Header translation by Alexey Barkovoy for Free Pascal Platform
Input Method Manager definitions
See the file COPYING.FPC, included in this distribution,
for details about the copyright.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
**}
(**)
(* imm.h - Input Method Manager definitions *)
(**)
(* Copyright (c) Microsoft Corporation. All rights reserved. *)
(**)
{$mode objfpc}
unit
Imm;
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] valgrind, parsing sym from exe
Hi, I am trying to use Valgrind on a executable build with fpc. I get the following error (the error was while using callgrind, with verbose output) ==18293== Reading syms from /data/OTHER/laz_trnc/lazarus (0x8048000) @@ unlikely looking definition in unparsed remains ";" @@ expected ',' at struct TYPE (remains="PCHAR:t166=*180") @@ parsing 185=s20length:87,0,32;mlength:87,32,32;parent:98,64,32;vm1_or_classname:183,96,32;__pfn:184,128,32; gave NULL type (s20length:87,0,32;mlength:87,32,32;parent:98,64,32;vm1_or_classname:183,96,32;__pfn:184,128,32; remains) --18293-- INTERNAL ERROR: Valgrind received a signal 11 (SIGSEGV) - exiting --18293-- si_code=0xC Fault EIP: 0xB8060318 (); Faulting address: 0x0 Are there any known restrictions? - I am using the fpc 22 fixes branch (checked out today) - compile with either "-g -gv" or "-gs -gv" (same error in both cases) - The application I am trying to run under valgrind is lazarus, but as for reading the symbol tables from the file, that shouldn't matter? - Ah yes, and the OS is FreeBSD (whic may be part of the problem...) - valgrind identifies itself as 2.1.2.cvs (I had to go for the cvs version or it wouldn't start at all) Any idea if there are fixes in later versions of FPC? Or other compiler settings? Thanks for any feedback Martin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] units not getting rebuild after changes in other used ("uses") units
Hi, I encountered a strange problem. I am not yet sure what causes it, but I have a suspicion. Before I go all the way, of trying to prove the cause, and report it as a bug, let nme ask here if it may already be a knonw issue, or maybe is by design. Normally if I change a unit (e.g. changes to the inteface of "unit Nested"), and then compile my project, fpc will recompile all other units that have a "uses Nested;" (new ppu files, new .o files) In my case for some reason this does not happen, and all I get is an error from the linker at the very end: "Error: Undefined symbol: VMT_SYNGUTTER_TSYNEDITMARK" Btw, anyone, is there a way to make the linker tell me which file is requiring this symbol? Anyway, this means some unit was compiled using the other unit declaring the symbol. Then when the symbol was removed, the unit was not recompiled. The package containing it was rebuild, so it must have been fpc believing that the file and all dependencies where unchanged; even so dependencies had changed) I have a suspicion why FPC may not see the change in the underlying unit. - TSYNEDITMARK is defined in "unit SynEditMarks" - "unit SynEdit" has a forward declaration (in it's interface) that is "unit SynEdit" uses SynEditMarks, and then has a "type TSYNEDITMARK = SynEditMarks.TSYNEDITMARK ;" - Whatever unit causes the error, hust uses "Unit SynEdit", but not "unit SynEditMarks" Should such a case be detected? Best Regards/Thanks Martin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Multithreading wait for
Mattias Gärtner wrote: Hi all, I'm looking for a high performance 'wait-for'. The situation: There are m threads working in parallel on n chunks of work (n>=m). The n chunks are indexed 0..n-1. Sometimes one thread needs the result of some of the lower indexed chunks. For example the chunk number 5 needs chunks 0..2. So I have a function WaitForIndex(LowerIndex: integer) which should wait until all chunks with less or equal to LowerIndex have finished. My current approach is this: EnterCriticalSection (try..finally) Check if all lower chunks have finished, if yes then exit LeaveCriticalSection RTLeventWaitFor(AnEventOfTheCurrentThread); And each time a thread has finished a chunk, it wakes all corresponding threads. The problem is the gap between LeaveCriticalSection and RTLeventWaitFor. During this time the other threads may finish. So there is no one left to wake up the waiting thread. Moving the RTLeventWaitFor into the CritialSection creates a deadlock. I can use the timeout of RTLeventWaitFor and check in intervals, but criticalsections and low response times don't fit together. "high performance 'wait-for'. " First think that came to mind was Spin-Locks, so you never enter a wait state. But that only works, if you know you will never have to wait for long. Another possibility is a set of Semaphores. Each thread needs one semaphore. You can use zero wait on all semaphores of "lower" threads. Each thread sets it sem to 1 while working and to zero once finished. You can reverse the logic: 0 eq thread is busy, 1 (via increment, not via set-value) means thread is done. Then the waitng thread tries to decrement all semaphores Best Regards Martin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Call initialisation of library automatically in unit-initialization?
Joost van der Sluis wrote: Op woensdag 03-12-2008 om 13:09 uur [tijdzone +0100], schreef Michael Van Canneyt: On Wed, 3 Dec 2008, Joost van der Sluis wrote: When you use the MySQL client, you have to call myslq_library_init to initialise the library, and mysql_library_end to de-initialize it again. When the library is dynamically loaded using mysqlXXdyn.pp, shall I call these functions automatically? Or should we leave it to the programmer? (I'm in favour of doing it automatically) Are there any options to these calls ? If there are, it is a no, I think ? Didn't thought about that. And yes, there are options. They are in practice only used with the embedded-mysql version. And iven in that case I guess they are rarely used: So, what to do? There is another option. You can keep track if the user called it. If the user did not call it, but any other function is called, then call it yourself procedure MaybeCallInit; begin if InitCalled then exit; DoCallInit(0, nil, nil); InitCalled := True; end; Well it will be more complex, as if this is in a multi threaded environment you have to keep track for each Thread. That is unless the library is loaded and initialized before any threads are spawned... Best Regards Martin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Unicode and UTF8String
Mattias Gaertner wrote: [...] Also it would be nice (so I do not know how) not to have to duplicate code, in order to archive this. Something like generics, maybe. The goal of RTLString is to avoid duplicate code in the RTL. Yes I acknowledged that this would be a problem. And also RTLString has the benefit, of allowing unicode fpc to be available far earlier than it would otherwise. The question remains, could it then be extended/optimized? Maybe a generic like template (for functions, instead of objects)? Which needs to be written only once, and the will be specialized for each string type? Yes. I wonder, how many are there that really need this? Can you estimate? You mean how many rttl functions would benefit. I do not know. But I do think it's the wrong question. In order to do it you would want something like function-templates (generics equivalent for functions). I do not believe such a template system would be developped for the sole purpose of doing the above rtl work. If however such a template system would be implemented anyway (because it would benefit everyone in the same way generics do (actually I never checked: can generics do simple functions or only classes?)), so if it was there, then why not use it for the rtl too? In such a scenario does it matter if you optimize 5 or 5000 functions? ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Unicode and UTF8String
Point taken. And point also taken that it's not a solution to to a "half job", meaning overload only methods based on parameters, but not results. This would be a false pretence, ending people n a believe they do the right thing where they do not. Another idea: The programmer should be able to set himself what RTLString means, (with the default being the OS depended). It isn't really a nice solution though. But I throw it into the discussion anyway. Otherwise I''ll have to see, If I can come up with something else. Jonas Maebe wrote: On 01 Dec 2008, at 22:33, Martin Friebe wrote: I suggested to have a rtl, that has overloaded functions for each string type. of course that sounds easier than in fact it will be. Florian pointed out a few issues, like overloading by result is not possible (yet?). No, it is fundamentally impossible. First of all, because a Pascal compiler does not look at how the result of an expression is used while parsing the expression (so while evaluating a function call, the compiler has no idea what the "expected" result type is). That's simply how Pascal works. And how could the compiler decide what to call here: function f: longint; begin result:=1; end; function f: string; begin result:='s'; end; begin writeln(f); end. There are an immense amount of similar examples possible (and if you add operator overloading in the mix, it increases exponentially), and you'd get a huge amount of "can't decide which overloaded function to call" errors even if the compiler were hacked to take the use of the function results into account. Jonas ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Unicode and UTF8String
Felipe Monteiro de Carvalho wrote: On Mon, Dec 1, 2008 at 7:33 PM, Martin Friebe <[EMAIL PROTECTED]> wrote: I suggested to have a rtl, that has overloaded functions for each string type. of course that sounds easier than in fact it will be. This is about the same as having all string routines in 3 flavours: RTLString, utf-8 and utf-16 If you have a function overloaded in utf6 and utf16, then You will not need to have it in RTLString. RTLString will be equal to either of the 2? ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Unicode and UTF8String
Marco van de Voort wrote: In our previous episode, Mattias Gaertner said: Florian Klaempfl wrote: [...] My opinion is that it should be the programmers choice. I a programmer wants or needs a simpler way (keeping all the strings in is application in one format, which will be known to him) then he/she should have that choice. And then on this type the person could perform any index or index-like operation. About: keeping all the strings in is application in one format, which will be known to him This is not possible, since you don't control OS + headers. Most stuff will come from the outside in the system encoding. This way you can do the whole app in the system encoding, and only face conversions when outputing to the GUI, which is (relatively) infinitely slow anyway. I never needed an argument for the introduction of RTLString. It's a great idea. It allows a programmer at hi/her choice to make applications much more portable. Even so, in fact it is not new. You could always just have introduced your own type, and set it OS depended with a few IFDEFs I argue against a RTL that *only* provides a RTLString interface. Instead of various overloaded methods (see other mails, for which set of functions/methods this actually makes sense). I have however so far got good arguments for starting with RTLString solution. - a solution with many overloads would defer release - overloads (currently) lead to code duplication (so that needs to be solved first) - overloads are not possible based on function results But on the long term (fpc 2.6 or maybe 2.8) surely those issues can be overcome (if wanted)? ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Unicode and UTF8String
Felipe Monteiro de Carvalho wrote: On Mon, Dec 1, 2008 at 10:13 AM, Florian Klaempfl <[EMAIL PROTECTED]> wrote: I assume that the new variable encoding type would be used for all unicode routines, am I right? No, it will be RTLString which type depends on the OS. The more I think about it the more I like this solution. I think it's better then the previous idea of a string with encode information inside it. That I agree with. A string with encode information in it (I read that as a type-data field at runtime ) is not good (IMHO) Best Regards Martin. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Unicode and UTF8String
Mattias Gaertner wrote: On Mon, 01 Dec 2008 15:06:45 + Martin Friebe <[EMAIL PROTECTED]> wrote That would mean that in order to avoid conversation, some functions of the RTL would be needed in overloaded versions for each string type. IMHO this applies only to those, which do not (or not always) make calls to the OS. Any other function does the conversation anyway. (It will be a case by case base) Sorry, I can't follow here. Please enlighten me, why an overloaded function with an internal conversion is better than an implicit conversion? I made a difference between rtl function which will call OS functions (the one that would need to internally convert the format) and those that do not make OS calls (or do not always make them) Yes I agree, and said so before: If a rtl function is going to pass on the data to the OS, and conversation is always needed, then no overloading is needed. Use RTLString. If an function does not use the OS (e.g extract file-path or name) then no internal conversion is needed. Therefore overloaded functions would give benefit to some programmers. (to name one group: most beginners, who have more than enough to worry about; and will be glad if strings are kept simple to use ( a fixed known type, not a type that looks somehow abstract, because you do not know its implementation at the time you write your code) [...] Also it would be nice (so I do not know how) not to have to duplicate code, in order to archive this. Something like generics, maybe. The goal of RTLString is to avoid duplicate code in the RTL. Yes I acknowledged that this would be a problem. And also RTLString has the benefit, of allowing unicode fpc to be available far earlier than it would otherwise. The question remains, could it then be extended/optimized? Maybe a generic like template (for functions, instead of objects)? Which needs to be written only once, and the will be specialized for each string type? Best Regards Martin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Unicode and UTF8String
Luiz Americo Pereira Camara wrote: Martin Friebe escreveu: All the code Widestring := RtlFunction; Utf8string := RtlFunction; will run, it may just perform badly. Yes and no. Let's assume the platforms windows and unix having UnicodeString (UTF-16) and UTF8String as native types respectively. You choose to use UnicodeString type in your app. Using the rtlstring approach you get: Under windows: the native string type of platform is the same as you are using no conversion is taken. Good. Under unix: the native string type of platform is NOT the same as you are using ONE conversion is taken. Bad. Now let's assume that fpc team decided to use a fixed unicode encoding for the RTL. Let's say a UnicodeString RTL. You choose to use UnicodeString type in your app. I never suggested the RTL to be in a fixed encoding. I fully agree that this would be far worse. I suggested to have a rtl, that has overloaded functions for each string type. of course that sounds easier than in fact it will be. Florian pointed out a few issues, like overloading by result is not possible (yet?). And code duplication would be a maintenance hell. But those limits can be overcome. Maybe not in full for the first Unicode fpc release. I can see that in order to get at least something (and in a way forward compatible) to all the waiting users of fpc, the RTLString solution is a good solution (or compromise: full function, limited optimization). The functions that can be overloaded with what fpc already has, could be written for the various types. Maybe a template system for plain functions (like generics for objects) could be found? So code would not be duplicated. Maybe fpc could be extended to allow overloading by result? (sure that has other uses too?) It's just a suggestion. Best Regards Martin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Unicode and UTF8String
Marco van de Voort wrote: In our previous episode, Martin Friebe said: Of course they are still there, to be used in the few parts of your code, where you specialize on whatever string type you deal with. But otherwise, using RTLString IMHO will abandon this part of pascal syntax. It removes ASCII legacy. I don't see you complaining about the fact that char is not 8 bit anymore, and that that abandons that part of the pascal syntax. It does not abandon the syntax. It only adds to it's meaning (*adds*, any existing meaning is unaltered.). I can still do: foo[1] for *any* type of string. (well yes even RTLstring, but see below) - If string happens to be an old ascii string, that still works as it always has - If string happens to be any unicode => that is still the same syntax, but with a new meaning. The new meaning doe snot break anything, because it only applies to new types. It is usable too, because I know, I am dealing with codepoints, or sub code points. And I know how they look, and how to identify them The introduction of RTLString is fine. I do say it is a good thing. RTLString does not interfere with the above. In fact even for RTLstring the syntax foo[1] does exist. Just it is not useful. If I tread it as utf8 sub code point, I can be wrong. If I tread it as ascii, I can be wrong. If I tread it as UTF16 I can be wrong My argument was not against RTLString. However it was my understanding that RTL functions will "enforce" RTLString. That they will only exist for RTLString, and they will *not* exist for other string types. That I would call enforcing RTLString, because of penalties on using other string types. I acknowledge, that if the end result of calling the RTL function, is an OS call, the conversation/penalty is always there. But not every RTL function ends up in an OS call. I admit that the Problem started (and that has been discussed more than enough) starts with UTF8string (yes even with utf16 string). But in this case those functions became a new, but predictable meaning. I can do utf8string[1], and I can use the result. Only I have to be aware what it means. Yes. As widestring[1] also requires interpretation. That's unicode. See above: Yes it requires interpretation. But it allows me to do so I can not see how I can interpret RtlString[1]. If the result is bigger than 128, then I must know what type it is. If it is ANSI, it is a single byte char. If it is utf8, it is a sub-codepoint which will be part of a codepoint. If it is widestring, well yes, here breaks my assumption that RtlString[1] returns a byte ouch I can *not* do rtlString[1], as at the time of code writing I can not be aware what it means. You don't have to. You carry it around as long as you can, and when you don't can, you assign it to your type of choice and bite the penalty. As I said in another mail. Every programmer starts as a beginner. And for many of those this is the last thing to think about. Best Regards Martin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Unicode and UTF8String
Florian Klaempfl wrote: Martin Friebe schrieb: In other words, write pascal code, just do not use some of the (imho) most common elements of pascal syntax? I acknowledge a language is a living thing, and needs to be adjusted to the new things, that come up over time. I only ask, if this is the best way? We're open to proposals, make one. Well, RTLString will have it's place, same as something like RawByteString would. It sounded to me, that this will be the only way of communicating with the RTL. My opinion is that it should be the programmers choice. I a programmer wants or needs a simpler way (keeping all the strings in is application in one format, which will be known to him) then he/she should have that choice. And then on this type the person could perform any index or index-like operation. That would mean that in order to avoid conversation, some functions of the RTL would be needed in overloaded versions for each string type. IMHO this applies only to those, which do not (or not always) make calls to the OS. Any other function does the conversation anyway. (It will be a case by case base) It is nothing urgent, as it can always be added to the RTL. In the end this is an optimization, not a feature. All the code Widestring := RtlFunction; Utf8string := RtlFunction; will run, it may just perform badly. And that is my point. Not everyone using pascal, is an expert. Pascal is also an attractive language to beginners. And not every bit of code will be written for a bg application; some may go into a small tool, for a very specific task at hand. There may be valid reasons to not keep every string in your code as RTLString. I believe FPC should do a good job for this kind of code too? Anyway. I know "Patches are welcome" Unfortunately at the moment I will have to keep it to contribute ideas. Also it would be nice (so I do not know how) not to have to duplicate code, in order to archive this. Something like generics, maybe. Best Regards Martin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Unicode and UTF8String
Marco van de Voort wrote: In our previous episode, Martin Friebe said: I agree, using RTlString will probably help fpc to optimize your exe for each OS. But, using RTLString means you do not know, if you have UTF8 or not. Correct. Because UTF8 behaves slightly different from other Strings, many operations can not be performed on RTLString foo[1], copy, pos ... simply because you do not know, if the result is a char, a codepoint or a subcodepoint (single utf8 byte) You don't know that about UTF-16 either. Even though that is no problem in True, good point most cases, it is slowly time to abandon too simplistic thinking about strings. The best solution is to minimize editing, and localize them in certain parts of the code, keeping most of the code encoding agnostic. True, too. But we are talking Pascal, not some other language. string[index], copy, pos, length have always been part of Pascal. Of course they are still there, to be used in the few parts of your code, where you specialize on whatever string type you deal with. But otherwise, using RTLString IMHO will abandon this part of pascal syntax. A function of which the result can not be used, as it can change at compile time => such a function can not be used. (or we will have buffer overflows, code injection and more ...) I admit that the Problem started (and that has been discussed more than enough) starts with UTF8string (yes even with utf16 string). But in this case those functions became a new, but predictable meaning. I can do utf8string[1], and I can use the result. Only I have to be aware what it means. I can *not* do rtlString[1], as at the time of code writing I can not be aware what it means. It is only decided, at compilation time. IFDEFs won't help neither, because they can only cope with the set of stringtypes know at the time the code is written. This breaks each time FPC will be extended. > and localize them in > certain parts of the code, keeping most of the code encoding agnostic. Sorry I can't help taking that into another direction, (which also has been discussed before). The above quote sounds like a sentence from a introduction into "object orientation". Sure it is the right thing.. It is right for OO. So it should be right for strings as well. Just again, it simply will be a new language, which a string-object, instead of pascal. And yes, if you lazy, you lose performance due to automatic conversions. It has always been that way (also when mixing short and ansistring) In other words, write pascal code, just do not use some of the (imho) most common elements of pascal syntax? I acknowledge a language is a living thing, and needs to be adjusted to the new things, that come up over time. I only ask, if this is the best way? This is not just a good thing for OS interfacing code, but a good thing in general. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Unicode and UTF8String
Marco van de Voort wrote: In our previous episode, Martin Friebe said: Why would you do this and not MyString := SomeRTLRoutine; ? If I understand that right, this may cause some overhead, that in some(few) cases is not needed. Correct. If I write an application using stringtype "X" (WideString for example), then in the above "MyString" would be WideString. Correct The in/ouput for SomeRTLRoutine are RtlString, they are OS depended. If I compile for a OS using "UTF8" then that means for each and every call, it needs a string conversation. Correct. Of course I understand, *if* some RTLFunction calls the OS, then the string must be converted. But if I simply want to extract the drive letter, or trim the path, and get the file name, without actually accessing the file or OS? Should it be possible to skip converting? Use rtlstring. Do the conversion to widestring after. IOW, you should do it the other way around. Use the OS dependant stringtype for mostly encoding independant operations, and only the few things where you need specific encodings force a certain encoding (using utf8string or widestring) I agree, using RTlString will probably help fpc to optimize your exe for each OS. But, using RTLString means you do not know, if you have UTF8 or not. Because UTF8 behaves slightly different from other Strings, many operations can not be performed on RTLString foo[1], copy, pos ... simply because you do not know, if the result is a char, a codepoint or a subcodepoint (single utf8 byte) RTLString is or will be great, if you simply need to store an OS depended string in order to later give it back to the OS. (eg open file, remember file name, but do not process it (displaying it would be vi OS), and save file back to the same name.) For this you could also use ByteString: if there is such a thing, and if it behaves as "not converting, if assigned to any string" Best Regards Martin --- Disclaimer: Just to keep this discussion where it was: - I do understand why the above is as it is (string index not being utf8 chart access). - I do not believe that this is correct too (and any discussion should be a new thread) ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Unicode and UTF8String
Florian Klaempfl wrote: Felipe Monteiro de Carvalho schrieb: On Mon, Dec 1, 2008 at 10:13 AM, Florian Klaempfl <[EMAIL PROTECTED]> wrote: No, it will be RTLString which type depends on the OS. Ok, so code would be something like this: var OSString: RTLString; MyString: UTF8String; begin OSString := SomeRTLRoutine; MyString := OSString; ? Why would you do this and not MyString := SomeRTLRoutine; ? If I understand that right, this may cause some overhead, that in some(few) cases is not needed. If I write an application using stringtype "X" (WideString for example), then in the above "MyString" would be WideString. The in/ouput for SomeRTLRoutine are RtlString, they are OS depended. If I compile for a OS using "UTF8" then that means for each and every call, it needs a string conversation. Of course I understand, *if* some RTLFunction calls the OS, then the string must be converted. But if I simply want to extract the drive letter, or trim the path, and get the file name, without actually accessing the file or OS? Should it be possible to skip converting? Best Regards Martin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Unicode support in RTL - Roadmap
Felipe Monteiro de Carvalho wrote:
On Fri, Nov 21, 2008 at 2:42 PM, Michael Schnell <[EMAIL PROTECTED]> wrote:
And thus forces all users to "understand the full UTF-8 spec" and to rewrite
their programs, even though the old code perfectly compiles and up to a
certain extent seems to work.
This is what I think is "not at all desirable" :( .
Your comments are absolutely vague and meaningless. Not to mention
thay also don't propose an alternative.
Sorry to be blunt, but so were your comments
I must agree with the "FPC can not to it all automatically" line (as
much as I regret, and admit the beauty there was, if fpc could).
What I mean is:
1) Any Application/Program, that currently compiles and works (using
none utf8, never mind if ascii or ansi) will keep working, if compiled
using *none* utf8 mode.
2) If such a program wants to be compiled to be extended to utf8
support, then there is a need for decisions that can not be made without
knowledge what the program is doing. Or even within the same program in
which context the operation takes place.
Such knowledge is only available to the programmer of this application,
therefore the application must be changed to include this decisions. FPC
simple can not make them. (And even {$SWITCH} would not solve the issue.)
Example is the composed and decomposed "ü":
- If you edit a text (human readable text), or search in a text, you
certainly do want to handle both representations as equals (a Find
dialog must find both)
- If the same text editor saves the file, it must handle them as non
equal. Assume the user has 2 files "wünsche.txt" in the same folder.
The filesystem allows this, because one of them is decomposed and one is
composed. If the user had opened a text from the composed version, it
should be written back to the composed version. If the user had opened
it from the decomposed version it must be written back to the decomposed
version. Otherwise a completely unrelated file would simply be
overwritten, and the contents lost. (the same applies if the application
iterates through the directory content and compares file names. So here
the same compare version that would be used by the "Find dialog" must
behave different)
FPC can simply not know, if a string contains a file name, which must be
kept exactly as it, or a string contains some human readable text, which
would benefit from a "normalisation".
If you are going to put a compiler switch in front of each statement to
indicate the needs, you may as well change the statements. There is no
one statement for the whole application, as both of the above example
occur within a single application.
You could use two different UTF8Strings which behave different on
decomposed chars (I am *not* proposing this as a solution). But then you
can not just recompile your app by saying "string" now means UTF8String
throughout the whole application. You have again to go through all of
the source code and edit the app. So you may as well just go through the
sourcecode, and add the appropriate utf8-clean up calls to those part in
the code, that will need it.
In the end, switching an application to unicode means that within the
same app different parts are going to need different handling of unicode
(where no such difference existed for ascii/ansi). And no compiler can
figure out which part will need which behaviour.
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Unicode and Lazarus
Daniël Mantione wrote: Op Thu, 20 Nov 2008, schreef Felipe Monteiro de Carvalho: So, what kind of support could be implemented in Free Pascal to improve things for Lazarus and it´s users? Maybe a real UTF8String? There will be a real UTF8string, i.e. ansistring with UTF-8 encoding as part of type information, this will help Lazarus users to get rid of the utf8encode/utf8decode. " ansistring with UTF-8 encoding as part of type information" This sounds like a dynamic solution? As in the info that it is UTF8 encoded is part of the data, not of the type? Or do I misunderstand? IMHO UTF8String should be like widestring. The encoding/decoding related parts are decided at compile time; not at runtime. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] assembler error with inlining on power pc / mac? [Fwd: [Lazarus] PPC: SynEditFoldedView giving "parameter syntax error"]
Hi, I do not have myself a power-pc, nor mac. But I have several reports like the below. I know from one instance (where this happened during cross-compiling from a linux box) that the error disappeared, when inlining was disabled (The units explicitly marks a lot of procedures for inlining) First: On power-pc/Mac, does fpc always use an external assembler? Or is there a build in assembler. I know that the person cross-compiling was using an external assembler. / I do not know if the reporter below did or did not use an external assembler (but I see the "Assembling unit" in the output.) I'd appreciate any other hints too. Regards Martin Original Message Subject:[Lazarus] PPC: SynEditFoldedView giving "parameter syntax error" Date: Thu, 6 Nov 2008 12:29:36 +0100 From: EarMaster - Bent Olsen <[EMAIL PROTECTED]> Reply-To: General mailing list <[EMAIL PROTECTED]> To: 'General mailing list' <[EMAIL PROTECTED]> Hi all, I have updated Lazarus to latest revision (17246) on Win32 and Mac i386/PPC, and downloaded latest FPC 2.2.2 (20081106 on Win32, 20081104 on Mac i386/PPC). Lazarus are build nicely and with no errors on Win32 and Mac i386, but on PPC it is throwing: [...] Assembling syneditfoldedview Syneditfoldedview.pp:3191:Parameter syntax error (parameter 2) Syneditfoldedview.pp:3202:Parameter syntax error (parameter 2) [...]/lazarus/components/synedit/syneditfoldedview.pp(81,75) Error: Error while assembling exitcode 1 [...]/lazarus/components/synedit/syneditfoldedview.pp(81,75) Fatal: There were 2 errors compiling module, stopping Except for the compiler path, the environment settings, versions, and installations on Mac i386 and PPC are identical, so I have no clue as to why SynEdit will not compile. These are on Mac OS X version 10.4.11. Any clue as to what I might have missed out? Thanks, and best regards, Normann ___ Lazarus mailing list [EMAIL PROTECTED] http://www.lazarus.freepascal.org/mailman/listinfo/lazarus ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] a few installation issues (trunc on freebsd)
I have a few question, regarding my attempts to install fpc from trunc (I read the build faq, yet I have not succeeded...) On a freebsd (7.0) box, I have downloaded and installed the pre-compiled fpc 2.2.2 ( to use as my initial compiler.) I have created my workind dir /data/fpc_trunc/svn and done my checkout (Revision: 12021) cd /data/fpc_trunc/snv svn co http://svn.freepascal.org/svn/fpc/trunk . then according to the build-faq, I have done cd compiler gmake cycle * First problem: I found if I simply go to the parent (svn) directory, and do a "gmake all PP=/data/fpc_trunc/svn/compiler/ppc386" there, it will do some clean out, and stop because halfway through it will have deleted this compiler. So instead (which is not mentioned in the buildfaq?), I have installed the compiler and rtl at this stage: gmake install INSTALL_PREFIX=/data/fpc_trunc/inst/ cd ../rtl gmake install INSTALL_PREFIX=/data/fpc_trunc/inst/ Note: I am installing to a separate dir, as I want to use this installation in parallel to other installations. * Second Problem: when using fpc (instead of ppc), it will always find the ppc386 in the default installation. The only way I can change this is "fpc -Xp/path/to/my/ppc386". (Which I found no documentation for. I got that from looking through the source) However, if I want to use fpc from lazarus, I may not be able to pass that (convenient) on the command line. Is there any INI file? (fpc.cfg did not help here, at least not the fpc.cfg in the same path as fpc exe). Or is it save to substitute fpc, with ppc386 everywhere it is called? any way I then go and create my fpc.cfg: cd ../../inst ./bin/fpcmkcfg -d basepath=/data/fpc_trunc/inst/lib/fpc/2.2.2 > bin/fpc.cfg * 3rd problem, it uses the same prefix for the units (-Fu) and the bin (-FD), so the above creates correct -Fu config, but I must change the -FD conf. Any solution to this ? -Fu will be correct: -Fu/data/fpc_trunc/inst/lib/fpc/2.2.2/units/$FPCTARGET/rtl but -FD will *NOT* be (which I think it should?, or am I wrong?) -FD/data/fpc_trunc/inst/bin now I go back to the top svn checkout and to gmake all PP=/data/fpc_trunc/inst/lib/fpc/2.2.2/ppc386 * 4th Problem the compilation aborted (Sorry I don't remember where) because it tried to access the x11 package => I did solve that by going int packages/x11, did the make there, and did install it (with the install prefix) * 5th problem packages/graph stops, because it tries to use x86. (This exists in the sources rtl/unix/x86.pp) Somehow all the above has not installed this (not on my FreeBSD), the package is missing int he inst/lib directory And last not least, having checked out trunc (or being under the impression to have done so), why has the "make install" in the rtl installed something with 2.2.2 in the path? (If I run the ppc386 that was installed it will say 2.3.1. Maybe this last bit will solve some of the other questions... Any help is greatly appreciated. Many thanks. Martin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] assign constant text to widestring
Michael Schnell wrote: Hi Experts, [.] When I want to simply assign a constant text "ö2" to a WideString I would think that I just write s := 'ö2'; . But I found that this does not work, but that it creates a WideString of length 3 that contains the three 8-Bit subcodes of the utf8-coded string "ö2", zero-extended to 16 Bits, each in one WideChar element. For me this is very surprising and incompatible to the same code (s := 'ö2'; ) used in a Turbo-Delphi program. Obviously - other than Turbo-Delphi that uses ANSIString here - a constant string gets UTF8String as it's intermediate type. This might be a useful definition, but if that is done this way why does an assignment WideString := UTF8String inot implicitly call UTF8Decode as a type conversion ? In my example it calls fpc_ansistr_to_widestr instead, just as if the UTF8String would be an ANSIString. I am not an expert, but here is what I believe to know: This is the result of 2 (hidden) "features": AFAIK the compiler reads the source as non-utf8 (latin or some 8 bit encoding). This leads to other things too, like identifiers cannot contain utf8. The String within the quotes is a byte sequence to the compiler. And the compiler does not know it to be utf8. From your description I take it the compiler does translate those 3 "8bit chars" into some 16bit chars (correctness of this translation based on the 8bit source encoding is another question) Lazarus uses UTF8 for everything, it will save your string as utf8. If Your string was kept as ansistring, the compiler would treat it as bytes, and pass it through, so any code wanting to see the utf8 would be fine. You can try and tell Lazarus to save you file as latin1. As long as all you strings fit into latin1, this may work; IF and only if the compiler will translate the latin1 into correct Widechars. It will not work for anything not in utf8. AFAIK Lazarus currently doesn't save in ucs2 (or any 16 bit encoding). But even if Lazarus did, since the compiler wants 8bit encoding, your whole source would be broken. Not much help, I know. Maybe some one else does have more ideas / knowledge. Is there some compiler setting to change this ? -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] fcl-web improvement for apache modules, question
Michael Van Canneyt wrote: On Fri, 19 Sep 2008, ABorka wrote Hi Michael, It was created in fpapache.pp in the handlerequest event. M:=MC.create(self); It worked without using the createnew, with the latest FPC source code. This is weird, because the code before that searches if there is an existing instance. If so, it should be reused ? OK, I'll have a look at it. It should probably be created with owner Nil, and then a unique name should be made and assigned, after which self can be put as the owner. Just an idea (not tested), but why creating it with owner nil, and assign self later. Just immediately after the create (or better within the create) assign it a new unique name (using a counter, or the address in memory). Then when the next instance is created it can be created again with the default name, as all others have already new names. (Only have to make sure that it's done thread save, if threads are in use) If the default name is set via the property, setter-method, then it can be modified there too... Regards Martin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Bug in FPC and declaring distinct types
Florian Klaempfl wrote:
Yury Sidorov schrieb:
Yury Sidorov schrieb:
Yes. But it works only partially. For example the following code
is not compilable:
Well, I wouldn't know either what you expect :)
I expect that compiler will choose DoTest with ansistring parameter
in that case.
What rule do you apply to say this?
Compiler treats '1234' as ansistring constant. Therefore ansistring
overload must be choosen here.
No, '1234' is taken as generic string constant.
I was able (with "Free Pascal Compiler version 2.2.2rc1 [2008/06/27] for
i386") to compile the below.
(I know it's an old fpc version, but I am not at my home PC)
As shown with the non-overloaded procedures p1a, p1w, p1s: the string
'xx' can be given to any of them.
On the other hand the compiler allowed me to overload p1 for short, ansi
and wide-sring
I do not know which rule it followes if p1('xx') is called?
type
sstr = String[3];
procedure p1s (s : sstr);
procedure p1a (s : ansistring);
procedure p1w (s : widestring);
procedure p1 (s : sstr); overload;
procedure p1 (s : ansistring); overload;
procedure p1 (s : widestring); overload;
implementation
procedure main;
begin
p1('xx');
p1s('xx');
p1a('xx');
p1w('xx');
end;
Warning: Use of ppc386.cfg is deprecated, please use fpc.cfg instead
Hint: Start of reading config file
c:\lazarus\fpc\2.2.2\bin\i386-win32\fpc.cfg
Hint: End of reading config file c:\lazarus\fpc\2.2.2\bin\i386-win32\fpc.cfg
Warning: You are using the obsolete switch -OG
Free Pascal Compiler version 2.2.2rc1 [2008/06/27] for i386
Copyright (c) 1993-2008 by Florian Klaempfl
Target OS: Win32 for i386
Compiling C:\DOCUME~1\Martin\LOCALS~1\Temp\project1.lpr
Compiling unit1.pas
unit1.pas(23,18) Hint: Parameter "s" not used
unit1.pas(24,18) Hint: Parameter "s" not used
unit1.pas(25,18) Hint: Parameter "s" not used
unit1.pas(27,17) Hint: Parameter "s" not used
unit1.pas(28,17) Hint: Parameter "s" not used
unit1.pas(29,17) Hint: Parameter "s" not used
Linking C:\DOCUME~1\Martin\LOCALS~1\Temp\project1.exe
101 lines compiled, 1.3 sec , 1056816 bytes code, 280748 bytes data
2 warning(s) issued
8 hint(s) issued
Project "project1" successfully built. :)
Like in this case:
const
sss: ansistring = '1234';
...
DoTest(sss);
//--
type
TUTF8String = type ansistring;
procedure DoTest(const s: ansistring); overload;
begin
end;
procedure DoTest(const s: TUTF8String); overload;
begin
end;
begin
DoTest('1234');
end.
//--
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Unicodestring branch, please test and help fixing
listmember wrote: IMHO The discussion splits here between: 1) How can this be done in a specific app 2) what should fpc provide as for 2: This would be on top of yet (afaik) missing basic functions such as Compare using collation x (where collation is given as argument to compare, not as part of any string) I think we're beginning to be on the same page --but, please, can you refrain from using the word 'collation'; every time I see that in this context, I feel a strong need to open the window and shout "collation isn't the most important/used part of a language wrt programming" :) Sorry, but I meant comparing with collation. I did not mean comapring within labguage context. language context is to complex to be basic (see busstop below) 2) actual compare, you need to "normalize" all strings before comparing, then compare the normalized string as bytes. normalizing means for each char to decide how to represent it. German "ae" could be represented as a umlaut for the compare. Or (in German text) you expand all umlaute first. IOW, SameText() and similar stuff must take normalization into account. But, you do know that 'normalization' is a very rough assumption and land you in some very embarassing situations. Here is 2 words from Turkish. 1) 'sıkıcı' which means 'boring' in English (notice the dotless small 'i's) 2) 'sikici' which means 'fucker' in English Depends how you normalize. Normalize should sbstitute all *equal* letters (or combination thereof) into one single form. That allows comparing and matching them. But yes, even this is very limited (busstop), because even if you know the language of the wort (german in my example) you do not know its meaning. Without a full dictionary, you do not know if ss and german-sharp-s are the same or not. So basically what you want to do, can only be done with a full dictionary. Or you have to accept false positives. I also fail to see why a utf8 string is a half baked solution. It will serve most people fine. It can be extended for those who want more. IMHO this is a case for an add-on library. And apparently no one has yet volunteered to write it Now, when you normalize these you get 'SIKICI' for both which --then-- you would assume to be the same. BUT of course there is no way do deal with the ambitious "Busstop" In deed. For this case, you need to know what language "Busstop" was written in. you need a dictionary. knowing it is German is not enough. because all that "it is german" tells you is, that "ss" maybe a sharp-s, but doesn't have to be What I can not do (or what I do not want to do) is to decide which of them other people do want to use. But, isn't this just that: IOW, you're deciding what other people will NOT want to use if you throw the 'language' attribute (for each char) out of the window.. True, I am happy to do that. NOT I am glad we have met :) have we? I remember a mail conversation, but not an actual meeting :) SCNR Why you can always extend this. Store you string in any of the following ways 1) every 2nd char is a language attribute, not a char 2) store the language attributes in a 2nd string, always pass both strings around Of course, these and even more creative hacks could be devised. The question is, is the language an attribute of a unicode character? (I assume "mandatory attribute") Well as much as it is or is not an attribute of a latin1 or iso-whatever char. I do not think it is. I have no proof. But a lot of people seem to think so, if I goggle Unicode (or any other char/latin./iso...) I get nice character tables; and no language info. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Unicodestring branch, please test and help fixing
listmember wrote:
Martin Friebe wrote:
Just to make sure, all of this discussion is based on various collation
No part of this discussion is based on collation.
Ok, so we were talking about different things
Here is a scenario for you:
You have multilanguage text as data. Someone has asked you to search
it and see if a certain peice of string (in a given language) exists
in it.
This search needs to be NOT case-sensitive.
Actually for you example case doesn't matter. as you need to decide if
"ss" = "ß"
How can you do this?
Is it doable if TCharacter (or wahtever you call it) has no 'langauge'
attribite?
For the purpose of case-sensitivity. I still do not know of a character
or rather a pair of upper and lower case char) that maps different in
some languages?
Is there a pair of character "x" and "X" which should in some languages
be matching upper/lower, but in other languages should not?
^^ ignore, found your example at the end of mail
Otherwise how do I understand the case-insensitive part of your
question? Because if "x" is the lowercase of "X" in *all* languages,
then I do not need the language specific info to do the
none-case-sensitive compare.
Sorry if I am still missing some point...
[Note that, here 'TCharacter' isn't necessarily an object; it might as
well be a simple record structure.]
Yes we agreed on this part
Besides instead of storing it per char, you can use unused unicode as
start/stop markers. So it can be implemented on top of a string that
stores unicode-chars (and chars only, no attributes)
Is there, in Unicode, start-stop markes that denote 'language'?
I do not know, that was why I said "unused unicode" and "implemented on
top" (as part of the specific app)
IMHO The discussion splits here between:
1) How can this be done in a specific app
2) what should fpc provide
as for 2: This would be on top of yet (afaik) missing basic functions
such as
Compare using collation x (where collation is given as argument to
compare, not as part of any string)
Why is language intrinsic to the text? An "A" is an "A" in any language.
At best language is intrinsic to sorting/comparing(case on non
case-sense) text
Comparing is a lot more important an operation than collating --or,
rather, collation is achieveable only if you can do proper comparisons.
Take this, for example:
"if SameText(SomeString, SomeOtherString) then do ..."
For this to work properly, in both 'SomeString' and 'SomeOtherString',
you need to know which language *each* character belongs to.
I would rather say:
"There are special cases where you need/want to know which language"
So I do not imply how special or none special those cases are => you do
not always need to know. (continued below on your example)
If you dont have that informtaion, you might as well not have a
SameText() function in FPC.
Please note the 'case-INsensitive' keyword there.
Well I needed an actual example where case sense differs by language
(assuming we talk about language using the same charset (not comparing
Chinese whit English).
Here is a simple example for you:
"if SameText('I am on FoolStrasse', 'I am on FoolStraße') then do ..."
Well that is a good question, do you always want that to return the same?
"Busstop" and "Bußtop" (Yeah the second is not a word, but could occur
in a text)
Also in Names this comparisons does not always apply.
the Name "Heiße" (originally with ß) can be spelled as "Heisse"
But the Name "Heisse" (originally with "ss") is never the same has "Heiße"
But as for asking me: This a specialized comparison, Similar to soundex
(compare sound of 2 words, usually based on english)
Something like this is usually found in extension libraries, but not in
the standard functionally of a (many/most) languages.
In any case I think this also has the minority problem. Most people do
not want to compare pascal strings this way (and if it only is because
of false positives)
That does not mean that I say such functionality is not desirable. It
would be great having a unit that can be used if needed.
Based on the idea that this are optional (or 3rd party) functions, the
normal String would not provide for this. (Besides attaching info to
each char would probably be to costly, even if implemented in the fpc
core string.)
Functions like this could take an additional structure declaring the
start/stop/change point of every language.
In any case, I can write up several different algorithms how to do that.
Please do. SameText(), for one, will need all the help it can get.
The initial comment was based on collation, and basically would have
been about prioritizing in conflict
Re: [fpc-devel] Unicodestring branch, please test and help fixing
Just to make sure, all of this discussion is based on various collation for European languages? Or shall we include Arabic, Chinese and other languages? But they have there own chars, they can be identified without collation, so they do not need the language info, to be distinguished from European text. (They may have collations, the same as a German text could be handled in different collations) listmember wrote: So maybe the design is quite well thought? Adding a flag field is easy enough --if all you're doing is to do some sort of collation. In that sense, everything is well tought out. But.. Life becomes very complicated when you begin to do things like FTS (full text search) on a multilanguage text in a DB engine. Your options, in this case, is just very limited: -- Ignore the langage issue. or -- store each language in a different field (that is if you know how many there will be). Do you think this is a good solution --or, a hack. True, that would be hard to do (in DB or pascal, or most other places). But again this is a very special case. And that is why none of the frameworks (DB, pascal, ...) include it. You have to do your own solution. At no time did I say (nor did afaik anyone else say) that you can not do your own object based text holding objects. The question were: 1) should FPC replace the string, by an object (like java) 2) which additional attributes should be stored by a string (per string / per char) And actually both of those question can be moved out of the context of Unicode implementation. Because, both of them could also bee applied to current (char=byte) based strings. I am going to leave out the object question for now. I said all I can say in earlier mails. And also from your comments it appears more a question of collation being stored with the string, substring, or even each char. As found in the last mail, there is currently no standard for handling cross-collation in any string function (that is string function, which could be collation based). 1) IMHO only few people would need this. For the majority it would be unwanted overhead. 2) Within those few, there would be too many different Expectation as to what the "standard" should be. If FPC choose one such standard at will, it would benefit almost no one. The best FPC could to is provide storage, for something that is not handled or obeyed in any function handling the data. This doesn't sound desirable to me. If anyone who needs it will have to implement the functions, then those may add there own storage for it too. Besides instead of storing it per char, you can use unused unicode as start/stop markers. So it can be implemented on top of a string that stores unicode-chars (and chars only, no attributes) As for Storing info per string or per char. (Info could be anything: collation, color, style, font, source-of-quote, author, creation-date, file, ) everyone would like there own. So again FPC shouldn't do it. Or everyone gets all the overhead of what all the others wanted. Collation is a function of language. Right but language is something you can apply to strings. You are not forced to do so. Strings work very well without language too. Same as you saying "no gui". Strings work without display. Font/Style is a function of rendering. I may want to search a string but only want to look at chars marked as bold. Languages is an extension to string, in the same way than rendering info, or source info is. To you language may matter a great deal. To others other attirbutes will matter. All the others are not an intrinsic part of o a char at all --they vary by context. Why is language intrinsic to the text? An "A" is an "A" in any language. At best language is intrinsic to sorting/comparing(case on non case-sense) text If pascal doesn't suit the need of a specific task, choose a different tool. Instead of inventing a new pascal. Thank you for the advice. But, instead of jailing this discussion to at best a laterally relevant issue of collation, can I ask you to think for a moment: How on earth can you do a case-INsensitive search in *any* given string contains multiple language substrings? Please note the 'case-INsensitive' keyword there. Well I needed an actual example where case sense differs by language (assuming we talk about language using the same charset (not comparing Chinese whit English). In any case, I can write up several different algorithms how to do that. What I can not do (or what I do not want to do) is to decide which of them other people do want to use. search none-case-sensitive 'UP LOW' in ' ups upper lows lower' with the following attributes: 'UP LOW' is a string of 2 languages. The word UP is in a language that defines "U" and "u" as different letters (not only differ by case, but differ the same as "a" and "b" do differ) The word LOW is in a languages where all letters are having low-case equivalents (as in Engl
Re: [fpc-devel] Unicodestring branch, please test and help fixing
listmember wrote:
I also do not know of other apps that could do this. (And it may not be
possible). Look around. Databses for example, AFAIK the most you can do
is define a collation per column.
True. But, that does not mean that those app/databases are well
thought out. Does it?
Point of View. Those DB get sold, so either people take what they can
get and silently accept it (I haven't seen discussions like this on
related DB discussion groups [ or maybe I read the wrong groups :) ])
or the majority of people doesn't need it.
BTW people want there DB to sort text in a way, that help finding
entries in the result. So the ordering process should not rely on
knowledge if a word is English or French. If It did rely on the
language, then the ordering would not help the search, because you have
to know the language of all other words to find the one word you are
looking for.
So maybe the design is quite well thought?
And how would you sort the following example, with mixed collation. Take
the various german collations. ae can be used as a substitution for
a-umlaut.
This is actulaly an arbitary decision --there is no agreed standard on
this, that I am aware-- so, each developer can have their own way.
Well yes of course you can define how to. But then everyone has a
different need, and a different definition. That would mean FPC had to
implement dozens of algorithms.
So it seems better to leave it to each person, as it seems it will be an
individual thing anyway.
As for Storing info per string or per char. (Info could be anything:
collation, color, style, font, source-of-quote, author, creation-date,
file, ) everyone would like there own. So again FPC shouldn't do it.
Or everyone gets all the overhead of what all the others wanted.
Also FPC is a programming language. Not a word processing tool
And FPC is pascal. Pascal (afaik) has reference counted strings. And
objects are not reference counted. Not to mention objects (as string
type) would only benefit if everyone was allowed to create their own
child-classes.
Then instead of asking for strings as object, I would ask for an
additional ref-counted object type (with auto destruction). The string
library could be based on this. I am not asking for suxch a think
because a) it wouldn't be pascal anymore. b) beware of the mem-leaks
If pascal doesn't suit the need of a specific task, choose a different
tool. Instead of inventing a new pascal.
I don't to shell scripts in pascal. And simple web scripts are php or perl.
How would you sort data where one source is of one collation, the other
source of another (or even worse the collation changes halfway through)?
It is impossible by definition.
No. It is not impossible.
But, yes, there is no definition (standard).
It would be upto the developer or the entity that the developer is
working in.
Btw in normal math you can not devide a number by zero... Of course you
can define your own math
I even thing that collation is not part of the string. it does not
change the meaning of the string. It is only used in specific
operations. And then it must be one collation for both strings. So if
each of the string had a collation that would cause an issue.
But, my question is --imho-- a lot more relevant to the thread at hand:
How would you do case-insensitive search in a multilangual text.
same as above applies. If every char (or substring) has a collation of
its own, then you need to define how to compare cross-collation.
because
find('E'[collation1], 'merci'[collation2] + 'mein herr'[collation3])
needs to compare an E (that wants collation1 for the compare) with each
of the 'e' (that want other collations)
maybe collation1 says that E should equal in upper and lower, while the
other collations do not? ore vice versa.
there is no standard.
[this has nothing to do with rendering or GUI.]
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Unicodestring branch, please test and help fixing
listmember wrote: Actually, UTF-8 can contain bidi info, it's indeed a matter of the renderer. And, how do you propose doing a case-insensitive search in a given text that contains multiple languages? I assume you speak of multiply collations in on string? IMHO You can't? But you could use a TStringList. I also do not know of other apps that could do this. (And it may not be possible). Look around. Databses for example, AFAIK the most you can do is define a collation per column. And how would you sort the following example, with mixed collation. Take the various german collations. ae can be used as a substitution for a-umlaut. In some collation it sorts as ae (between ad and af), in others it sorts as "a-umlaut" (immediately behind "a") 1) a, ab, ae 2) a, ae, ab How would you sort data where one source is of one collation, the other source of another (or even worse the collation changes halfway through)? It is impossible by definition. Because taking the 2 Strings above, each of them can come first when sorted depending on the collation, but if more than one collation was involved the result was undefined. I even thing that collation is not part of the string. it does not change the meaning of the string. It is only used in specific operations. And then it must be one collation for both strings. So if each of the string had a collation that would cause an issue. Martin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Unicodestring branch, please test and help fixing
Michael Van Canneyt wrote: On Thu, 11 Sep 2008, Anton Kavalenka wrote: Florian Klaempfl wrote: Graeme Geldenhuys schrieb: Remember, Unicode support is much more that simply storing and displaying text. You have various encodings, RTL or LTR direction etc. I can't see how a simple type can keep track of all such information - but then, I don't know the internals of FPC either. ;-) How would an OOP approach solve this? The problem isn't the tracking of things like encoding or directions but handling all these information. procedure TLabel.Paint(...) begin if *Caption.IsRTL *then DrawCaptionRTL(0,0,*Caption.AsUTF8*, flags) else DrawCaption(0,0,*Caption.AsUTF8*, flags); end; Is not that enough? What is the gain as opposed to procedure TLabel.Paint(...) begin if IsRTL(Caption) then In other words where is the benefit from OOP in this ? 1) keeping track of info: If you can store the info on an object, so you can store it on a record (afaik). And a string (even current string) is nothing else but a (hidden) record. It already contains length info, and char data. 2) OO style vs functional: Caption.IsRtl may be seen as syntactical sugar. But as far as I can see, it can (almost?) always be translated into functional style. Instead of having child-classes you could overload the function for different types of arguments 3) For the real usage of OO using inheritance: I am not sure if that is a good idea, on any kind of *ref-counted* data/object. I can see cases where the full OO power can make sense. But using OO the objects should not be ref-counted. (IMHO) Ref-Counting mainly is used to free memory automatically. People relay on it, and you get memory leaks. Strings as they currently stand can not contain pointer to other strings. You can not get circular references. ref-counting will work. Objects on the other hand can contain any data, including pointers to other objects or self. Even if the buildin string-objects don't contain that kind of pointer, they can be sub-classed and people will end up with circular references. Oh, and yes, I am aware. This risk already exists with dynarrays. But no need to extend it. So in my opinion, it may be nice to have a library of classes handling all kind of string(or shall we call it "text") data. But no magic on them. They can use PChar and there own GetMem internally. Martin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
synchronous class [Re: Multi threading support was Re: [fpc-devel] Russian locale information not compatible withFPClocale variables]
IMHO per class may create some problems? but maybe I am just overlooking something? TBar =class( TFoo ) end synchronized class; Let's say TFoo was declared in another unit, and TFoo was not declared Synchronized. Then TFoo code can be entered by anyone at any time. Even if a thread is in TBar. How would one make sure, that a base class, knows about all classes that inherit from it? Neither can you say the baseclass must also be "synchronous class", because ultimately there always is TObject. and TObject does not want to be Synchronous class, or all Create/destroy would be in critical sections? It my be possible to work around, this, but it would probably create a ig overhead, as every method one very class would have to perform checks at runtime. Similar for instance wide protection. Regards Martin Michael Schnell wrote: Per class. The question is if it's possible to do this per class instance too. To me "Per class" does not seem appropriate at all. "Per instance" in fact seems to make sense. -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] protected members access across classes in the same package, but what about inherited cross package
Just a question, which came to my mind in relation to http://bugs.freepascal.org/view.php?id=11697 I understand that if you declare 2 classes in the same package (same file), then they are allowed to look at each others protected members (variables, properties, func, proc). I wonder what the intend or language definition says about inherited protected members those classes have, if the parent implementation comes from different files? Examples: [file foo.pp] type TThat = class(TObject) protected ThatValue : integer end; TThis = class(TObject) end; implementation ... [/end of file foo.pp] I know it is correct that any code of TThis can access ThatValue. If you have code in any other package/file (belonging to any other class) it should not be able to access ThatValue. [file bar.pp] type TThatFake class(TThat) end; // intntionally empty class TSomething = class(...) [/end of file bar.pp]] - If TThatFake had any code, then any code belonging to TThatFake would be allowed to see it's anchestor protected members, that would be correct. - If TThatFake had any protected mebers implemented in bar.pp they should be visble to other code in bar.pp - But what about inherited members? Should TSomething be able to access ThatValue? I do not know the answer, it can be argued both ways. But it allows to bypass proper encapsulation as seen in the above link. Best Regards Martin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] C style operator doesn't work with properties in 2.3.1
From my understanding ( the fpc team probably can to more exact): * lexical level This would be like a simple "Find and Replace" on the text. This would work with anything, as it doesn't know what it is translating. (It is very similar to a (non-buildin) C-preprocessor) * parser level The parser will likely translate the source into a structure of objects a := a + 1 becomes ObjectAssignmenOp LeftHandTerm = ObjVariable "a" RightHandTerm = ObjExpression ( ObjAddition: Op1 = Variable "a"; Op2= 1 ) If that happens to "+=" then "a+=1" ObjectAssignmenOpIncr LeftHandTerm = ObjVariable RightHandTerm = ObjExpression if the left hand term is a property, then because it is the left hand (which will be assigned to), the lefthand term becomes the property-setter. The ObjectAssignmenOpIncr then has no reference to the property-getter. Graeme Geldenhuys wrote: On Thu, Jul 17, 2008 at 9:07 AM, Micha Nelissen <[EMAIL PROTECTED]> wrote: Jonas Maebe wrote: Indeed. I've checked the code and properties are explicitly not allowed for the C-style operators. The reason is that the x+=y is translated into "x:=x+y" at the parser level rather than at the lexical level. This So I guess the obvious question is: would it be risky to change that so it parses it at the lexical level? :-) My question too Would somebody be so kind as to explain the difference to me? As I mentioned before, I'm not a compiler developer. Regards, - Graeme - ___ fpGUI - a cross-platform Free Pascal GUI toolkit http://opensoft.homeip.net/fpgui/ ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] C style operator doesn't work with properties in 2.3.1
Or Add a Method to the target object. (as long as it is part of your code base, or can be modified by you) and then do FDayList[itm.WeekDayNum].Rows[itm.Timeslot].AvailableSlotsInc(itm.CountSlots); In the class that defines property AvailableSlot; you can add procedure AvailableSlotsInc(n : Integer); Same for include, exclude, and all the others. Best Regards Martin Jonas Maebe wrote: On 17 Jul 2008, at 08:41, Graeme Geldenhuys wrote: Simple one liners like the following: inc(FDayList[itm.WeekDayNum].Rows[itm.Timeslot].AvailableSlots, itm.CountSlots); or FDayList[itm.WeekDayNum].Rows[itm.Timeslot].AvailableSlots += itm.CountSlots; now has to change to this ugly line... FDayList[itm.WeekDayNum].Rows[itm.Timeslot].AvailableSlots := FDayList[itm.WeekDayNum].Rows[itm.Timeslot].AvailableSlots + itm.CountSlots; Or with FDayList[itm.WeekDayNum].Rows[itm.Timeslot] do AvailableSlots:= AvailableSlots+itm.CountSlots; Jonas ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] The usage of Include() doesn't work any more in 2.3.1
Hi,
just to add another opinion. Personally, I would thing that it was nice
to allow (write-able) Properties to any var-param (even out-param).
Write-able include properties with a setter-procedure.
IMHO the way it could be done would be for the compiler to create a
temporary variable. This could be passed in as var/out-param, and then
be assigned to the property, using the proper setter-procedure.
I do of course understand, that this is extra work on the compiler. And
I can not judge if it has enough value for anyone to implement it.
Best Regards
Martin
Jonas Maebe wrote:
On 16 Jul 2008, at 10:01, Graeme Geldenhuys wrote:
On Wed, Jul 16, 2008 at 9:40 AM, Daniël Mantione
<[EMAIL PROTECTED]> wrote:
I think there can be two visions here:
- Include is not a real procedure, so this internal implementation
detail
should be hidden and this can/should be allowed; the
compiler internally should convert it into set:=set+[member].
I think this would be ideal. Using Include() is much cleaner than set
:= set + [member]. And yes I understand that as it was implement, you
could circumvent the getter/setter, but I believe it's something the
compiler must handle correctly.
Adding such exceptions complicates both the compiler sources (special
cases have to be added for particular constructs in particular
circumstances) and the language ("x is not allowed, except in cases a,
b and c"). Over time, such exceptions (even if initially added for
convenience reasons) actually make things harder rather than easier
for most developers, because they keep adding up and you create a
completely inconsistent set of rules.
The reason is that you end up in a situation where people have to be
aware which routines are compiler-internal and thus can be magically
changed to do "the right thing", and which are plain rtl routines
which abide by the normal language rules (other exampleq would be
inc/dec). Such differences in behaviour actually already exist
nowadays, e.g. regarding overloading, causing confusion:
http://bugs.freepascal.org/view.php?id=8399
Jonas___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] move(xx, ansistring[1] versus ansistring, len)
Ah, ok, thanks. that may work. Is it documented somewhere, under which circumstances uniquestring is inserted by hte compiler, and when not? In other words, is there a way to be sure, that it will not be called twice in future versions? (Not that it would be a big issue, if it was called twice, but always eager to learn.) The background for this question is motivated by: http://bugs.freepascal.org/view.php?id=11280 Many of the occurrances look like SetLength(s, count); move(str^, s[1], count); where "s" is a local variable, not used before. So currently it would be safe to skip uniquestring. But I consider this as a trap, because future changes may act on "s" before, without the author being aware of the exact behaviour of move. Since the issue is widespread throughout the code (the bug only shows a few samples), and the current access "s[1]" does work, except for the (falsely) triggered rang-check, I find it a bit over the top to prefix every line with "if length(s)>0" --- I also see that (s: AnsiString) move(str^, s, count); seems to work too. (again without implicit uniquestring) Is there anything against this syntax? It does not convert s to nil, in case of empty string, but since len is 0 nothing will happen. Thanks a lot Martin Jonas Maebe wrote: On 13 May 2008, at 00:17, Martin Friebe wrote: move(somepchar^,pchar(ansi_str_dest)^,len); doesn't check fpc_ansistr_unique. So it works only if ansi_str_dest has only one reference; or both strings will be modified (because no copy-on-write happens). At least, that's what I get to see, if I compile with -al and look at the assembler. You can call uniquestring(ansi_str_dest) before the move() Jonas ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] move(xx, ansistring[1] versus ansistring, len)
move(somepchar^,pchar(ansi_str_dest)^,len); doesn't check fpc_ansistr_unique. So it works only if ansi_str_dest has only one reference; or both strings will be modified (because no copy-on-write happens). At least, that's what I get to see, if I compile with -al and look at the assembler. JoshyFun wrote: Hello Martin, Monday, May 12, 2008, 11:27:08 PM, you wrote: MF> Hi, MF> I am trying to find some info on how I can use move to archive the MF> following: MF> I am looking for an equal replacement of MF> move(somepchar^, ansi_str_dest[1], len) [...] MF> I could prefix it with "if length(dest) > 0 then ", but I would prefer MF> something less noticeable... I had not checked it but I think you can use the typecasting: move(somepchar^,pchar(ansi_str_dest)^,len); as typecasting an ansistring to a pchar should return (repeat "I think") a pointer to ansi_str_dest[1], and if length(ansi_str_dest)=0 then the pointer should be $0001 :-? which should not raise any kind of range check. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] move(xx, ansistring[1] versus ansistring, len)
Hi, I am trying to find some info on how I can use move to archive the following: I am looking for an equal replacement of move(somepchar^, ansi_str_dest[1], len) The problem with ansi_str_dest[1] is that if the above call happens with a len of 0, and an ansi string with 0 length => this leads to a range check exception. SetLength(as,0); copy(src^, as[1], 0); Accessing the first of zero characters => range check. ( It will work without range-checking, since whatever address is returned by [1] is never accessed ) I could prefix it with "if length(dest) > 0 then ", but I would prefer something less noticeable... I could do SetLength(as,0); copy(src^, as, 0); But from looking at the assembler produced during compilation, this does not check for fpc_ansistr_unique. Same for any type casting to PChar. Anyone any idea? Thanks Martin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
interlocked commands [Re: [fpc-devel] LockFree Queue algorithm]
While watching this thread, I started to ask myself 2 questions (which
may be related):
They just came to mind a multi-core systems where mentioned, possible
even systems with several CPUs.
(Admiringly they are more looking like they should be on an
intel-mailing list, I just hope someone may know...)
1)
procedure tFLQueue.setObject(lp : integer;const aobject : tNodeQueue);
begin
tab[lp and fMask]:=aObject;
end;
The index ("lp and fMask") has been derived via "interlockedIncrement",
and the surrounding code makes sure, that only one thread will access
this value at this time.
But lets assume the value was read immediately before, by another
core/cpu. It therefore is in that core/cpu's cache. Will this cache be
invalidated/updated by a *simple* write to memory? Or will the other
core/cpu see the old value from it's cache?
I am no expert on this, but from the page referred below, "lock"ed
cpu-instructions, take special care of this.
I don't know about unlocked instructions?
http://static.scribd.com/docs/59o7jahfstz7r.swf?INITIAL_VIEW=width
chapter 7
Because frequently used memory locations are often
cached in a processor's L1 or L2 caches, atomic operations can often
be carried out
inside a processor's caches without asserting the bus lock. Here the
processor's
cache coherency protocols insure that other processors that are
caching the same
memory locations are managed properly while atomic operations are
performed on
cached memory locations.
"caching the same memory locations are managed properly while atomic
operations are performed"
What does the cache coherency do (if anything) while non atomic
operations are performed?
2)
I found various references that interlockedIncrement and co, work only
on 32 bit bounded data? This may or may not vary on the CPU.
The Intel doc only says, it will affect execution time, but looking at
the MS doc http://msdn2.microsoft.com/en-us/library/ms683614.aspx it
says it must be on a 32bit boundary.
Does that affect FPC? ( as there may be none intel CPUs?)
If so, then the Implementation of the Queue would have to ensure
alignment (as I believe fpc, aligns integer on 16 bit?)
Martin
Florian Klaempfl wrote:
DarekM schrieb:
Martin Friebe pisze:
You will need to test it, but the following may also work
procedure tFlQueue.push(const tm : tNodeQueue);
var
newTemp,
lastTail,
newTail : integer;
begin
newTemp := temp;
while newTemp >= fsize begin
// if it is true, every thread is going to attempt to fix it,
before doing the increment.
// this means no thread will increase the value
// => one thread will to succeed (since the only reason "temp NE
newTemp" is that temp has been decreased)
// newTemp is bigger than fsize, so the result can not become
negative.
interlockedCompareExchange(pointer(temp),pointer(newTemp-fsize),pointer(newTemp));
newTemp := temp;
end;
newTemp:=interlockedIncrement(temp) mod fsize; // make sure we are
always in range
lastTail:=newTemp-1;
if lastTail < 0 then lastTail := fsize -1;
setObject(lastTail,tm); // you can remove the "mod" in setobject
repeat
pointer(newTail):=interlockedCompareExchange(pointer(tail),pointer(newTemp),pointer(lastTail));
until (newTail=lastTail);
end;
It seems ok, but then we have 2 IF more.
An if is unimportant, more important is the number of locked operations,
especially on multi core systems they might eat hundreds of clock cycles.
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] LockFree Queue algorithm
You will need to test it, but the following may also work procedure tFlQueue.push(const tm : tNodeQueue); var newTemp, lastTail, newTail : integer; begin newTemp := temp; while newTemp >= fsize begin // if it is true, every thread is going to attempt to fix it, before doing the increment. // this means no thread will increase the value // => one thread will to succeed (since the only reason "temp NE newTemp" is that temp has been decreased) // newTemp is bigger than fsize, so the result can not become negative. interlockedCompareExchange(pointer(temp),pointer(newTemp-fsize),pointer(newTemp)); newTemp := temp; end; newTemp:=interlockedIncrement(temp) mod fsize; // make sure we are always in range lastTail:=newTemp-1; if lastTail < 0 then lastTail := fsize -1; setObject(lastTail,tm); // you can remove the "mod" in setobject repeat pointer(newTail):=interlockedCompareExchange(pointer(tail),pointer(newTemp),pointer(lastTail)); until (newTail=lastTail); end; The "repeat" at the end does still work. In order to set "tail" to "0" (newTemp=0) => "tail" must first get "9" (as lastTail will be 9). Only flaw is, if fsize is close enough to the upper data tpye boundary, and enough threads do InterLockedInvrement, (after all having simultaneously passed the repeat at the start), then you can still run over the boundary. However this can only happen if you have more threads, than (upperBoundary - fsize). So you can specify, that it is save up to this amount of parallel threads IMHO the unit should have a warning (in the comments) that there it is the users responsibility *not* to store more elements than fsize. There is no check, and you will loose data, if you try to do so. But as I said, no warranty that this will work, I haven't done any testing... Martin DarekM wrote: Martin Friebe pisze: What about a long running (eg daemon) application? If temp/tail hits the upper boundary of Integer? (If I understand it correctly) I don't know if interlockedIncrement gives a boundary error, but if not, it still fails. - With currently integer, it gets a negative value, once crossing 0x7fff, and SetObject will attempt to read/write out-of-bounds memory. - Assuming temp/tail being unsigned: it will go from 0x to 0. "0x mod fsize" may return a value greater 0, "0x00 mod fsize" will be zero. You make an unexpected jump within the list. Thats right. But I can avoid this. 1. make length of tab equal power of two 2. use longword instead of integer 3.instead MOD use result:=tab[lp and fmask]; and fmask:=$F or $FF or $FFF Thanks to find bug. Darek ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] LockFree Queue algorithm
What about a long running (eg daemon) application? If temp/tail hits the upper boundary of Integer? (If I understand it correctly) I don't know if interlockedIncrement gives a boundary error, but if not, it still fails. - With currently integer, it gets a negative value, once crossing 0x7fff, and SetObject will attempt to read/write out-of-bounds memory. - Assuming temp/tail being unsigned: it will go from 0x to 0. "0x mod fsize" may return a value greater 0, "0x00 mod fsize" will be zero. You make an unexpected jump within the list. Martin DarekM wrote: Hi This is my proposition of algorithm and its implementing multithreaded FIFO queue without lock. First use array of pointers to handle messages. I've use it in my program, it works. I think it may by useful. site: http://www.emadar.com/fpc/lockfree.htm source: http://www.emadar.com/fpc/flqueue.pas There is also generic implementation of the same algorithm. Its my first generic approach. Any help appreciated. it;s not so much text, but source is very short, its explain much better. Darek ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Debug Info proposal
sorry my mailer wrapped that a bit awkward. the important part was: |symbol-file [ filename ]| Read symbol table information from file filename. |PATH| is searched when necessary. Use the |file| command to get both symbol table and program to run from the same file. Martin Friebe wrote: I haven't tested this, but from what I read, if strip can preserve the symbols in a separate file then gdb should be able to load them. http://www.delorie.com/gnu/docs/gdb/gdb_125.html |exec-file [ filename ]| Specify that the program to be run (but not the symbol table) is found in filename. GDB searches the environment variable |PATH| if necessary to locate your program. Omitting filename means to discard information on the executable file. |symbol-file [ filename ]| Read symbol table information from file filename. |PATH| is searched when necessary. Use the |file| command to get both symbol table and program to run from the same file. I am not sure what file format gdb expects for symbols. It may well be, that you can take your original (unstripped) executable for the symbols. Please try it out yourself. Good luck Martin Daniël Mantione wrote: Op Sun, 13 Jan 2008, schreef Joost van der Sluis: Op zaterdag 12-01-2008 om 20:11 uur [tijdzone +0100], schreef Bogus?aw Brandys: Jonas Maebe wrote: Could fpc devel team provide application with source code which could translate addresses from bare bone stacktrace generated from stripped executable into full stacktrace with unit/line info using bare bone stacktrace file and executable with debug info included ? Why would we? This is exactly what strip does! It strips out the debug-info to a seperate file. This is wat the -debug package is when you create a .rpm. That -debug package stores the debuginfo. If you install the -debug package, you can use gdb to debug your application. If you don't install it, you can't. To be honest, if you would receive a runtime error + stack backtrace output from a user, converting it back to a source line back trace isn't that easy currently. Sure, you can load the debuginfo executable in gdb, then look up the lines one by one, but it would be a time consuming process. Daniël ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Debug Info proposal
I haven't tested this, but from what I read, if strip can preserve the symbols in a separate file then gdb should be able to load them. http://www.delorie.com/gnu/docs/gdb/gdb_125.html |exec-file [ filename ]| Specify that the program to be run (but not the symbol table) is found in filename. GDB searches the environment variable |PATH| if necessary to locate your program. Omitting filename means to discard information on the executable file. |symbol-file [ filename ]| Read symbol table information from file filename. |PATH| is searched when necessary. Use the |file| command to get both symbol table and program to run from the same file. I am not sure what file format gdb expects for symbols. It may well be, that you can take your original (unstripped) executable for the symbols. Please try it out yourself. Good luck Martin Daniël Mantione wrote: Op Sun, 13 Jan 2008, schreef Joost van der Sluis: Op zaterdag 12-01-2008 om 20:11 uur [tijdzone +0100], schreef Bogus?aw Brandys: Jonas Maebe wrote: Could fpc devel team provide application with source code which could translate addresses from bare bone stacktrace generated from stripped executable into full stacktrace with unit/line info using bare bone stacktrace file and executable with debug info included ? Why would we? This is exactly what strip does! It strips out the debug-info to a seperate file. This is wat the -debug package is when you create a .rpm. That -debug package stores the debuginfo. If you install the -debug package, you can use gdb to debug your application. If you don't install it, you can't. To be honest, if you would receive a runtime error + stack backtrace output from a user, converting it back to a source line back trace isn't that easy currently. Sure, you can load the debuginfo executable in gdb, then look up the lines one by one, but it would be a time consuming process. Daniël ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
