Re: [fpc-devel] Const optimization is a serious bug
Op Thu, 7 Jul 2011, schreef Chad Berchek: The problem comes down to the specs though, or rather the lack thereof. As I have searched the web and read some docs at Embarcadero, things have only become more ambiguous. You are looking in the wrong places. Both Turbo Pascal and Delphi came with a language guide that describe the desired behaviour in deep detail. Daniël___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug (What nexy)
On 08/07/2011 11:05, Jonas Maebe wrote: On 08 Jul 2011, at 05:51, Chad Berchek wrote: I'm more interested now in a solution. The solutions I've seen so far have potential, but in my opinion most of them seem like they are more complex, would be more overhead, and produce worse performance than just getting rid of the const optimization. The main proposals I've seen were a) treat const string parameters the same as value parameters (i.e., get rid of the current const behaviour) b) the same as a), combined with a very conservative heuristic to apply the optimization in case there can be no side effects. I.e., in case there is not a single function call and not a single indirect write (var parameter, pointer, global variable, dynamic array, ...) to any string or anything that might be somehow aliased to a string passed as parameter to the current routine c) keep the current behaviour, but add functionality to the compiler to help debug problems that can occur as a result of problems that can occur as a result of this behaviour Actually I think you can describe that as 3 independent proposals: 1) Add a compiler switch or directive to disable constbehaviour for ref counted types (e.g. simply ignore const for string/dyn-array/interface params; Actually downgrade, still prevent assignment to local var) 2) Add automatic optimization of dropping ref-count and exception frame, if the compiler can prove it to be save (e.g. no calls to any other code...). This can happen completely independent of either the present or meaning of const. 3) Add safety checks (similar to range checks) 1) The only thing I can see, is that future optimization, added to const-param (of all types, e.g also records, etc) can introduce problems, when the value of the const param is indirectly changed, and that may lead to further requests of what const should or should not do. It may be needed to draw a line, and point out that const will always have some dangers. (Apparently with such a ignore-it-fix not any more for ansistrings (copy on write), but still for dyn arrays (also ref counted)). Anyway if the extend off that switch is well defined, it may be worth having. IMHO better as compiler directive, that would allow to put it into the code, and make sure behaviour is stabel (rather than behaviour might change, if a compiler switch is used) 2) Depends, on someone writing and contributing it. Mostly unrelated to the discussion. 3) Might not only help to fix issues, but might also create awareness. However, looking at the generated assembly, I can't believe there isn't a more efficient way to handle the implicit try-finally. It feels to me like that is where the problem is. There is a lot of code and several function calls that go into implementing that implicit 'finally' block. There's got to be some way to decrement the refcount even when there are exceptions but without so much overhead. It would require changes to the exception handling mechanism though. Here my knowledge runs out. The main problem here is that FPC's exception handling is based on setjump/longjump. This technique has a relatively high overhead for try, but low overhead when an exception actually occurs (of course, since exceptions are supposed to happen only exceptionally, that's not a really good selling point). The main reason we use it is because it's easy to implement. For implicit exception (so long as there purpose is only to decrease ref-counts (and free the data if 0 reached), there is another solution. If an exception frame already exists (even if on a higher stackframe), and the compiler can detect that it does exist: then instead of creating an exception frame for the current stackframe, all the variables that need refcount decrement, can be added to a list. This list can be processed by the exception handler, before any other code executes. The draw back is, that memory needs to be allocated for this list. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
Am 08.07.2011 12:05, schrieb Jonas Maebe: However, looking at the generated assembly, I can't believe there isn't a more efficient way to handle the implicit try-finally. It feels to me like that is where the problem is. There is a lot of code and several function calls that go into implementing that implicit 'finally' block. There's got to be some way to decrement the refcount even when there are exceptions but without so much overhead. It would require changes to the exception handling mechanism though. Here my knowledge runs out. The main problem here is that FPC's exception handling is based on setjump/longjump. This technique has a relatively high overhead for try, but low overhead when an exception actually occurs (of course, since exceptions are supposed to happen only exceptionally, that's not a really good selling point). The main reason we use it is because it's easy to implement. A better approach would be to use SEH-based exception handling (which has no overhead at all for try, and a high overhead in case an exception occurs), but that woud require a) support for generating EH frames for all platforms (it's currently only supported for a number of i386 and x86-64 platforms) b) support for parsing EH frames on all platforms and performing exception handling based on that This is definitely something we want, but nobody has found the time yet to implement it. Around a year ago I've experimented with using SEH on Windows (I have not reached a working state back then). It's on my list to revive this again, but before that I want to have Delphi compatible generics and extended RTTI with attributes working :D Regards, Sven ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug (What nexy)
Martin schrieb: c) keep the current behaviour, but add functionality to the compiler to help debug problems that can occur as a result of problems that can occur as a result of this behaviour Concrete: add means to ignore const with managed types. Further debugging aids as far as feasable. Do we want to make FPC an outstanding compiler, WRT bug hunting? IMO *not* a realistic goal. Actually I think you can describe that as 3 independent proposals: 1) Add a compiler switch or directive to disable constbehaviour for ref counted types (e.g. simply ignore const for string/dyn-array/interface params; Actually downgrade, still prevent assignment to local var) +1 2) Add automatic optimization of dropping ref-count and exception frame, if the compiler can prove it to be save (e.g. no calls to any other code...). This can happen completely independent of either the present or meaning of const. ~ [people will point at you if it doesn't work perfectly :-] 3) Add safety checks (similar to range checks) As a debug feature only, on demand (debugging memory manager?) Most people don't have problems with const, according debug features are *not* an urgent requirement. I only agree that a *quick workaround* (global ignore const option) is highly desireable (a *must*) for the rare (but real) cases where it can make runtime errors disappear. Extended debug features are not so important, because code with such problems typically suffers from *general* problems, which suggest a change of the personal coding style. A compiler can prevent an user from shooting himself into one foot, at best, but not prevent him from shooting himself into other limbs :-] 3) Might not only help to fix issues, but might also create awareness. This may apply to newbies, but IMO will be ignored by people which *believe* that they are superior coders, maybe from practice in other languages. A collection of suggested design patters (resource protection, object ownership...) would be nice - in detail for pointing people with coding style problems to. We could add to that collection all demo code, that forces unexpected exceptions, together with a safe version of the same code, that eliminates the demonstrated problem. If an exception frame already exists (even if on a higher stackframe), and the compiler can detect that it does exist: then instead of creating an exception frame for the current stackframe, all the variables that need refcount decrement, can be added to a list. This list can be processed by the exception handler, before any other code executes. The draw back is, that memory needs to be allocated for this list. IMO this is what the compiler already does, or should do - an implicit try-finally block with a list of variables to be finalized on exit of a subroutine (used also in stack unwinding after an exception). One should not confuse try-except (*handling* exceptions) and try-finally (guaranteed *cleanup*). DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
On Friday 08 July 2011 12:05:02 Jonas Maebe wrote: The main problem here is that FPC's exception handling is based on setjump/longjump. This technique has a relatively high overhead for try, but low overhead when an exception actually occurs (of course, since exceptions are supposed to happen only exceptionally, that's not a really good selling point). The main reason we use it is because it's easy to implement. A better approach would be to use SEH-based exception handling (which has no overhead at all for try, and a high overhead in case an exception occurs), but that woud require I strongly prefer the current situation where the occurrence of exceptions does not incur an important extra overhead compared to the nominal situation. In the type of applications I write (Air Traffic Control Radar Screens, Electronic Strips and System Interfaces) it is considered good practice to catch exceptions in every procedure because it is a must that the applications always survive and continue to provide the remaining, not affected services.(graceful degradation principle) I do not want any discussions whether catching exceptions is good or not. If someone thinks otherwise, I do not care at all, do not bother me or this mailing list. However what is imperative is that when many exceptions occur, there should not be a mayor performance penalty. One of the reasons I use languages such Pascal and Ada for this type of applications, is that they are very fast, (though I also have foreseen code to shutdown services with too many problems in a given time frame). So I beg you to leave exceptions as is (please :-)). kind regards, Den Jean ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] Sparc v9
membar and stbar are instructions available as of sparc v9. To get fpc
running on a sparc v9 I have commented out
def_system_macro('FPC_HAS_MEMBAR'); in compiler/options.pas. Is there a
command line option that does the same?
Ludo
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Sparc v9
2011/7/8 Ludo Brands ludo.bra...@free.fr:
membar and stbar are instructions available as of sparc v9. To get fpc
running on a sparc v9 I have commented out
def_system_macro('FPC_HAS_MEMBAR'); in compiler/options.pas. Is there a
command line option that does the same?
maybe -uFPC_HAS_MEMBAR.
Vincent
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
RE : [fpc-devel] Sparc v9
maybe -uFPC_HAS_MEMBAR. Mmm. Didn't expect that a command line undefine would override a define in the code. I'll give it a try. Thanks, Ludo ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Sparc v9
Ludo Brands wrote:
membar and stbar are instructions available as of sparc v9. To get fpc
running on a sparc v9 I have commented out
def_system_macro('FPC_HAS_MEMBAR'); in compiler/options.pas. Is there a
command line option that does the same?
I wonder whether this would be worth considering on a per-OS basis?
Working from memory, when I last looked I think that Linux initialised
the processor to a mode where it didn't reorder memory accesses so (if
my recollection of the documentation is correct) didn't require membars,
in which case an RTL startup check for an unexpected mode should suffice.
--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk
[Opinions above are the author's, not those of his employers or colleagues]
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Sparc v9
Ludo Brands wrote: membar and stbar are instructions available as of sparc v9. To get fpc running on a sparc v9 For completeness and playing Devil's Advocate, I do have to ask whether pre-v9 (i.e. pre-UltraSPARC) systems are relevant any more. I've got one remaining Sun4m system (SPARCstation 20) here which will just about run Linux 2.4, not sure it will handle anything later. I've also got several Sun4d which I've managed to get running SMP 2.4 but not 2.6. Both of those will also run Solaris 8 but I don't think later. I've got a Sun4c (SPARCstation IPC) which I can't get to run a realistic (=2.4) version of Linux, and won't run later than Solaris 7. I must admit that I'm very attached to the Sun4d, because it performs spectacularly and is a Xerox PARC design. But I'd certainly not expect it to run an off-the-shelf FPC. The remaining question is whether there's any mileage in supporting the Gaisler Research LEON processor, which I believe is a v8. -- Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues] ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
On 09/07/2011 00:09, Max Vlasov wrote: The answer is indirect referencing. it's a workaround that probably will solve the problem, but I must admit that I don't know what is the exact performance price. The compiler when it detects const s: ansistring could switch to passing not the actual address of the string, but the address of the variable that holds it. In other words passing PString instead of string. In this case no reference counting or exception frame is probably created and at the same time, if the used string is reallocated occasionally because of a side change, the code will not fail because it will just automatically use the new modified address. function CRCConstString(constref Str: string): integer; does what you describe ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
On Sat, Jul 9, 2011 at 3:14 AM, Martin f...@mfriebe.de wrote: On 09/07/2011 00:09, Max Vlasov wrote: The answer is indirect referencing. it's a workaround that probably will solve the problem, but I must admit that I don't know what is the exact performance price. The compiler when it detects const s: ansistring could switch to passing not the actual address of the string, but the address of the variable that holds it. In other words passing PString instead of string. In this case no reference counting or exception frame is probably created and at the same time, if the used string is reallocated occasionally because of a side change, the code will not fail because it will just automatically use the new modified address. function CRCConstString(constref Str: string): integer; does what you describe Hmm, it's interesting.. Some observations: - constref is implemented in 2.5.1, right? Unfortunately I can not test it right now in 2.4.2, it refuses to recognize it, but I I want to test it. - In LCL it used only with interfaces and TGuid's, no strings or other structures - there are not much information about constrefs in Lazarus, several posts, the largest part of them is yours, Martin :) so you're probably one of the few who really understand it :) If constref is what you said , some quick fix for those who's scary of the current const behavior could be iintroducing a setting in fpc that forces all const to be constrefs. Max ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
On 09/07/2011 00:59, Max Vlasov wrote: On Sat, Jul 9, 2011 at 3:14 AM, Martinf...@mfriebe.de wrote: function CRCConstString(constref Str: string): integer; does what you describ Hmm, it's interesting.. Some observations: - constref is implemented in 2.5.1, right? Unfortunately I can not test it right now in 2.4.2, it refuses to recognize it, but I I want to test it. - In LCL it used only with interfaces and TGuid's, no strings or other structures - there are not much information about constrefs in Lazarus, several posts, the largest part of them is yours, Martin :) so you're probably one of the few who really understand it :) Well the LCL can't use it, if it's trunk only (except if IFDEFed). The LCL should compile with release fpc... See: http://wiki.lazarus.freepascal.org/FPC_New_Features_Trunk#Constref_parameter_modifier - See difference to normal const is that it must be passed by reference. - Note (same as const) it is described as: This modifier means that the compiler can assume that the parameter is constant The assume means that the requirement for constant-ness goes above that what the compiler can enforce (that is: the compiler can enforce only the local var; but constant implies the value by any means of access) With what the compiler does today (afaik) none-constantness would not cause any harm (yet). Unless you count it as harm (and you should) that future FPC may (= is allowed to) compile your code (without any warning or hint) into an exe that behaves different. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
Martin wrote: - See difference to normal const is that it must be passed by reference. I had not read about it before, but I think constref is a huge step in the right direction. It eliminates my fundamental grief with the current implementation. Specifically, the way const is now defined (or not...) is disturbing because it leaves out important details. In C++, you can pass by value or reference, or by pointer which is a value but is used to make references. But what's important is you _always_ know what it's going to do. If you pass something by reference and modify that instance via another reference, they're all going to change, because they are the same instance. And if you pass by value, it's the opposite. The point is, though, you always know what it's going to be. Imagine if we took C++ and redefined the pass by reference syntax as the compiler might pass by value or reference, depending what it feels like. Result? Boom. Tons of carefully written code would suddenly break. It is OK for implementation details to be unspecified. That is how they should be--but only if they are in fact implementation details. If they affect the *meaning* of the program, they have to be defined as part of the language. That's what bothers me. The behavior of const is something that does affect the behavior of the program, but it is undefined. /Usually/ when you pass a record as const it will by by reference, but maybe not. Ditto for other types. All the problems we've seen with const used in various contexts with different data types--AnsiString, ShortString, records, etc.--are the result of one fundamental problem, which is the language design. Const is not clearly defined. All problems mentioned stem from this. If we *knew* that const meant it would be by reference, then that immediately eliminates the confusion in the case of ShortString and records; modifying one instance through several references affects them all, as expected. What the programmer says happens; there can be no bug, except in the programmer's own knowledge. This precision of language is closely tied with the testing approach several people have mentioned. The testing approach is that you run the program with the const optimization on, and, if it works, great, and if not, turn the optimization off. I greatly disagree with this approach. Testing is very important, certainly. But you can only ever test a tiny fraction of real world scenarios. Ultimately you are much better off if you can mentally verify that the program is unconditionally *correct*. That is why I cannot accept the ambiguous const feature, because it is impossible to theoretically prove that the program is correct, which is something I try to do as much as possible. I applaud the notion of constref. I think it should be used everywhere as long as Delphi compatibility is not required. The difference is slight. In many (but not all--and that's the key) cases using const or constref would result in the same assembly, if my understanding is correct. What's really important is that it allows the programmer to reason logically about a piece of code without mysteries popping up. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
On 09/07/2011 02:14, Chad Berchek wrote: Specifically, the way const is now defined (or not...) is disturbing because it leaves out important details. In C++, you can pass by value or reference, or by pointer which is a value but is used to make references. But what's important is you _always_ know what it's going to do. If you pass something by reference and modify that instance via another reference, they're all going to change, because they are the same instance. And if you pass by value, it's the opposite. The point is, though, you always know what it's going to be. Imagine if we took C++ and redefined the pass by reference syntax as the compiler might pass by value or reference, depending what it feels like. Result? Boom. Tons of carefully written code would suddenly break. ... If we *knew* that const meant it would be by reference, then that immediately eliminates the confusion in the case of ShortString and records; modifying one instance through several references affects them all, as expected. What the programmer says happens; there can be no bug, except in the programmer's own knowledge. We do know: http://www.freepascal.org/docs-html/ref/refsu58.html#x135-14500011.4.4 A constant argument is passed by reference if its size is larger than a pointer. It is passed by value if the size is equal or is less then the size of a native pointer. Of course if you want to rely on this, you must ensure the size of your type does ot change (a records definition can be chnaged, and that would have consequences for const param). But then, if you rely on it, but some compiler $IF and $FAIL in the code, to ensure it only compiles if your assumptions are met. Personally I thing this page would be a good place to document the properties of const param with regards to ref-counted types... ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
