Re: [fpc-devel] Const optimization is a serious bug
Max Vlasov schrieb: On Mon, Jul 11, 2011 at 4:45 AM, Chad Berchek wrote: 1. It seems that if a string is a (non-const) local variable it should be safe. I base this on the following reasoning: looks like it's not: procedure TForm1.Button1Click(Sender: TObject); var S: string; procedure SideEffectCall; begin SetLength(S, 0); end; Replace it by S: TObject; and S.Free to see that you can force any undesireable effect with reference types by alias. DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
Chad Berchek schrieb: Some might say that it doesn't matter whether it is passed by value or reference: if you pass it as const, either way you promise not to modify it. See, that is my concern: what is IT? People have used other words, but it ultimately comes down to: if you say the programmer promises not to modify the thing passed as const, what thing *exactly* is that? A variable, reference, or instance? With const, we don't really know. IMO we know that then the data, passed as immediate values on the stack, shall not be modified by the subroutine. The only confusion was about const records (and ShortStrings?), when passed by reference. In this case the content of the referenced record cannot be changed. When alias are of concern, don't use "const". With constref, it means you promise not to modify the memory location (instance) pointed to by that reference. Quite useless with objects, which typically are modified by method calls. Otherwise same as above. With constval it means that you can change the memory location/instance that was passed into the procedure, since the procedure is now using it's own copy anyway; you just can't modify the instance that the procedure now has it's own copy of. IMO that's the normal (non-const) procedure, so far. Any changes to the passed arguments are invisible after exit from the subroutine. One additional problem does arise. Constval implies that the implementation is pass-by-value. However in many (I'd say most) cases it is quite possible that we could be interested only in the semantics of pass-by-value, not the implementation. So for AnsiStrings, neither constref nor constval would be suitable. Constref would mean it must be by reference, but we want by-value semantics. Constval would mean that the string has to be copied to a separate memory location, which we might not really care about, and is slow and wasteful. That's why copy-on-write was invented. As soon as the data is modified, a (local) copy is created. So, I propose: don't have constval literally mean it is passed by value, i.e., pushing the string onto the stack or copying it to a new memory location. Instead, have constval defined as by-value *semantics*. In other words, constval would indicate the meaning of the language, not the implementation. It would not be a calling convention. It would not mean the call would be by value; it would mean the semantics would be by value. This is essentially what I originally thought const would mean with AnsiStrings, though it turned out hazy. The programmer must know the language and the compiler must implement the language. How the compiler does that should not determine how the program behaves. IMO that's such a weak definition, that it gave birth to this never ending thread :-( In the tradition of Wirth, it should be possible to describe the language semantics by a few implementation details. The implications *resulting* from the implementation, like alias, are already discussed in common literature. See regarding const: http://www.freepascal.org/docs-html/ref/refsu58.html The main use for this [const] is reducing the stack size, hence improving performance, and still retaining the semantics of passing by value... (Thanks for the link Alexander!) This is undoubtedly the most conclusive statement I've seen regarding this issue. If this documentation were correct, my initial claim that there is a bug would be correct. Aliasing is not considered a bug, it's a fact. When a pointer is passed to a subroutine, it should be clear to every coder that the referenced memory *can* change during the lifetime of the pointer, even if the pointer (address) is never changed; the same for all references. When a coder allows to pass a data structure byref instead of byval, by using "const", he'll have to accept all the consequences. Since this difference was the only rationale for introducing "const" into the language, it has to be accepted as defined, and the documentation should be updated accordingly. Effectively the "...semantics..." statement should be dropped from the docs, and instead the effects of the "const" modifier on *all* affected datatypes have to be added. DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
Chad Berchek schrieb: I just want to clarify: I am trying to be very careful to distinguish between implementation and semantics. "Implementation" is a very precise definition, which can be used to perfectly describe the semantics of a language. A verbose description of the semantics leaves too much room for misunderstandings, and commonly accepted formalisms are at least as hard to understand as implementation details, while not covering all aspects. DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
Alexander Klenin schrieb: On Mon, Jul 11, 2011 at 04:09, Hans-Peter Diettrich wrote: If you want to introduce different keywords, then please different for beforementioned cases. E.g. constval x: integer; //all value-types constref r: Trecord; //also ShortString const??? o: TObject; //all references, except following const??? s: AnsiString; //all managed types I am not quite sure what are you asking here. Can you elaborate? To which data type do you want apply your "constval" modifier, with which concrete effects? To any datatype, of course. The result should be: 1) Parameter is passed by value How to pass objects by value? Records with embedded AnsiStrings or other managed types? DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
Daniël Mantione schrieb: Within a certain calling convention, the behaviour of const is defined and will not change. I.e. an int64 may be passed by value in one calling convention and by ref in another, but once the calling convention is fixed, future compatibily is maintained; your assembler routines can safely assume the behaviour won't change. The compiler has to implement correctly the ABI of every platform, for compatibility with external functions. Assembler code has to be rewritten for every target machine and ABI. The language can only specify proprietary calling conventions (ABI), that are bound to specific processors, so that assembler code can be written once for every processor, regardless of the platform ABI. The use of "const" parameters and generic types is nonsense with assembler routines, because these are compiler specific implementation details. I'd also ban their use in shared libraries (DLL/so), which must have an immutable ABI. Here's another place for the "constref" modifier. DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
On Mon, Jul 11, 2011 at 4:45 AM, Chad Berchek wrote: > > 1. It seems that if a string is a (non-const) local variable it should be > safe. I base this on the following reasoning: > looks like it's not: procedure TForm1.Button1Click(Sender: TObject); var S: string; procedure SideEffectCall; begin SetLength(S, 0); end; function CalcSomething(const T: string): integer; begin Result:=0; SideEffectCall; MessageDlg(T, mtInformation, [mbOk], 0); // crash here end; begin S:='12345678901234567890'; S:=S + S; CalcSomething(S); end; ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] MySQL 5.1 and Double (trouble)
Can someone check the status of update if the bind parameter for double decimal values? I'm getting unexpected rounding errors after updating - the data in the column looks different than what was specified during the update statement. Value: double; Query.Params.ParamByName(sName).AsFloat:=Value; 1.) Update Value : 40734.825668912039 2.) Actual Value after update : 40734.825668912 3.) Actual Value on read : 40734.825668912003 As far as I know this is a relatively new problem. Could be with Ubuntu but was someone doing MYSQL work lately? ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
Sorry for the additional post; I should have added this to my preceding message. I just want to clarify: I am trying to be very careful to distinguish between implementation and semantics. If my understanding is correct, AnsiString implementation is always pass-by-reference. The problem is the programmer should be forced to stick to certain semantics, not a certain implementation. With const AnsiString, the implementation is consistently pass-by-ref, but the semantics can change as described in my previous message, depending on the instance's refcount when the procedure is invoked and what you may do with it while the procedure is running. It has correctly been pointed out that you therefore can't make an assumption about what exactly a const AnsiString parameter will do; you just have to be prepared in either case, and if it turns out to be by reference, don't modify the instance via another reference. I simply propose constref (existing) and constval (hyptothetical) for those who come across situations where more precise programming would be desired. --- Implementation --- I had a few ideas about implementation of constval, as I described it, for AnsiString. (Or, alternatively, a way to change the behavior of const, though this is no longer what I advocate.) These are all speculative. These are some ideas, basically, not assertions. Also my knowledge of how FPC works is limited as I've said before. 1. It seems that if a string is a (non-const) local variable it should be safe. I base this on the following reasoning: a) In order to trigger the undesired behavior, you have to get an instance with a refcount of 1 which actually has more than one reference to it. The only way to do this is to pass a reference to an instance with a refcount of 1 to a function accepting a const AnsiString parameter. (Aside: The critical value is 1 because that determines whether copy-on-write happens and whether it gets freed the next time the refcount is decremented.) b) Furthermore, you must be able to access the non-const reference at a higher scope (object, class, or global). c) To do this, you could either have a higher-scope variable which you assign to a local variable, or vice versa. d) If you do either of these, the refcount becomes 2 and the problem cannot occur. e) The problem cannot be triggered passing a local variable. 2. Implementing 1 would require that the reference count update can be applied or not applied to a specific function depending how it's called. This could be done in two ways I'm thinking of: a) Move the responsibility for updating the refcount to the caller. There are pros and cons to this idea. It could result in slightly larger code (because the refcount update and try-finally are in more places). However I think it could also speed things up *even more than the current implementation* because the refcount updates for several non-const strings could be combined into a single try-finally in the caller, rather than having one in each function. b) Another possibility is to have two entry points to the function. The update of the refcount and try-finally would remain in the function, not the caller. However the caller could enter the function at either of two entry points. One would do the refcount update and set up the try-finally, whereas the other would skip to the code after that. I don't know if this is even possible. It seemed like something that might be doable with some tweaking when I was looking at the assembly code. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
On 7/10/2011 2:40 PM, Daniël Mantione wrote: However, for a new calling convention (on an existing or a new platform) it's completely up to the compiler designer what will be passed as const and what will be passed as reference. Agreed. That's why I say you can't really make assumptions: if you assume a certain platform, you can make a limited set of assumptions (very limited as I will show in a moment). However one of FPC's great features is its cross-platform power, so I don't want to make assumptions about the platform. Without assumptions about platform you can make even less assumptions about const. Within a certain calling convention, the behaviour of const is defined and will not change. Even if the calling convention does not change, the semantics can, as currently implemented. Even within a single platform const is ill-defined. Consider the case of AnsiString. There are three cases as it is currently implemented: 1) Ref count > 1 when passed as const parameter results in strict pass-by-value semantics 2) Ref count = 1 and you modify the original instance in-place results in the value of the parameter changing 3) Ref count = 1 and you modify the original reference such that the instance's ref count decreases to zero and it may crash or just give weird behavior Although the programmer knows about reference counting, it is not expected for the programmer to know the details of the implementation, which can change. Therefore you cannot be 100% certain what the refcount is at any given time. And therefore you cannot be certain how const will be treated. We know the implementation will be pass-by-ref with no refcount change. The semantics could be pass-by-value, pass-by-ref, or just plain crashing, depending on details we cannot predict. You Pascal code should be able to handle both value *and* reference. After all, with const you are leaving the decision to the compiler. This is what it means in the end. Then your Pascal code should not depend on specific behaviour. Yes, I agree fully. That is the only choice with const. Constref, as it exists, with the addition of constval, provide a more precise form of expression on those occasions where one might feel the ambiguity of const is insufficient. And before anyone says something, yes, I also understand that if you pass something as a const, you have to be prepared in case it is passed by reference, and you have to realize that if it is passed by reference, you have to promise not to modify the instance via another reference. That's OK. I am a relative newcomer to Pascal, having used it about 1.5 years now. It was disturbing that something so important was not documented anywhere, and, in fact, there was documentation to the contrary. Aside from the lacking and sometimes incorrect documentation (I refer not only to FPC but to Delphi), I was surprised when the semantics of const AnsiStrings, which usually turn out to be pass-by-value, actually might not be in rare cases that are hard to predict. Aside from good documentation (which Delphi is in need of too in this particular case), I am proposing constval in conjunction with the existing constref to provide greater expressive power and precision for the programmer. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
On 11 Jul 2011, at 00:03, Daniël Mantione wrote: > Calling conventions should in principle be eternally frozen. This includes > const, because for exampe a DLL compiled in version x can be imported by a > program compiled by version y. That's indeed a good point. > The fact that it has happened (for whatever reason) does not mean it is good > practise. True. Jonas___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
Op Sun, 10 Jul 2011, schreef Jonas Maebe: On 10 Jul 2011, at 21:40, Daniël Mantione wrote: Op Sun, 10 Jul 2011, schreef Chad Berchek: I think we really cannot assume anything about what exactly will happen with regard to the calling convention, which is what I was worried about earlier. Within a certain calling convention, the behaviour of const is defined and will not change. Actually, even that is not true: http://wiki.freepascal.org/User_Changes_Trunk#Const_parameter_passing_on_non-Win64_platforms const parameters are indeed not suited to assembler routines for that reason. Calling conventions should in principle be eternally frozen. This includes const, because for exampe a DLL compiled in version x can be imported by a program compiled by version y. The fact that it has happened (for whatever reason) does not mean it is good practise. Daniël___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
On 10 Jul 2011, at 21:40, Daniël Mantione wrote: > Op Sun, 10 Jul 2011, schreef Chad Berchek: > >> I think we really cannot assume anything about what exactly will happen with >> regard to the calling convention, which is what I was worried about earlier. > > Within a certain calling convention, the behaviour of const is defined and > will not change. Actually, even that is not true: http://wiki.freepascal.org/User_Changes_Trunk#Const_parameter_passing_on_non-Win64_platforms const parameters are indeed not suited to assembler routines for that reason. Jonas___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
Op Sun, 10 Jul 2011, schreef Chad Berchek: I think we really cannot assume anything about what exactly will happen with regard to the calling convention, which is what I was worried about earlier. Within a certain calling convention, the behaviour of const is defined and will not change. I.e. an int64 may be passed by value in one calling convention and by ref in another, but once the calling convention is fixed, future compatibily is maintained; your assembler routines can safely assume the behaviour won't change. However, for a new calling convention (on an existing or a new platform) it's completely up to the compiler designer what will be passed as const and what will be passed as reference. You Pascal code should be able to handle both value *and* reference. After all, with const you are leaving the decision to the compiler. This is what it means in the end. Then your Pascal code should not depend on specific behaviour. Daniël___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
Some thoughts on the meaning of const, constref, and "constval", and how they can usefully be applied: My initial understanding of const was hazy. I have come to appreciate that it is defined, but only in a very loose way. Instead of undefined I should say ill-defined. I'm not entirely sure what some of the attacks have been about, but I will acknowledge that I was not 100% sure about const when I started. What I have said about it being unclear is certainly true though. Let me explain how, and what I think a better idea would be. I wrote: If we *knew* that const meant it would be by reference, then that immediately eliminates the confusion in the case of ShortString and records; modifying one instance through several references affects them all, as expected. What the programmer says happens; there can be no "bug", except in the programmer's own knowledge. Martin wrote: We do know: http://www.freepascal.org/docs-html/ref/refsu58.html#x135-14500011.4.4 A constant argument is passed by reference if its size is larger than a pointer. It is passed by value if the size is equal or is less then the size of a native pointer. Jonas wrote: A constant argument is passed by reference if its size is larger than a pointer. It is passed by value if the size is equal or is less then the size of a native pointer. That part of the manual is wrong. What const does is by design completely implementation-dependent I should not say const is undefined. Actually I mean const is ambiguous; sort of defined as undefined, in that: 1) with the current definition, it changes by platform, 2) the current definition is wrong anyway, and 3) it is implementation dependent, which I interpret to mean it could change in the future. If the meaning of const is "entirely implementation-dependent", and if I interpret that correctly, I think we really cannot assume anything about what exactly will happen with regard to the calling convention, which is what I was worried about earlier. Now, constref was intended for compatibility with other languages, but I think it is a valuable addition to Pascal in its own right and that it would be wise to use. The hypothetical constval modifier is a natural corollary to this. In the current implementation, if you were to pass a record to a procedure for example, you have two choices: 1) pass it by value but have it be non-const, 2) pass it as const, which might be by reference or value. With constval and constref you would have the other two highly useful choices: 1) pass by reference and it's const, 2) pass by value and it's const. Without making assumptions about const, which I don't think we can, these two options do not currently exist, so I think these new keywords are a useful addition. Some might say that it doesn't matter whether it is passed by value or reference: if you pass it as const, either way you promise not to modify it. See, that is my concern: what is IT? People have used other words, but it ultimately comes down to: if you say the programmer promises not to modify the thing passed as const, what thing *exactly* is that? A variable, reference, or instance? With const, we don't really know. With constref, it means you promise not to modify the memory location (instance) pointed to by that reference. With constval it means that you can change the memory location/instance that was passed into the procedure, since the procedure is now using it's own copy anyway; you just can't modify the instance that the procedure now has it's own copy of. One additional problem does arise. Constval implies that the implementation is pass-by-value. However in many (I'd say most) cases it is quite possible that we could be interested only in the semantics of pass-by-value, not the implementation. So for AnsiStrings, neither constref nor constval would be suitable. Constref would mean it must be by reference, but we want by-value semantics. Constval would mean that the string has to be copied to a separate memory location, which we might not really care about, and is slow and wasteful. So, I propose: don't have constval literally mean it is passed by value, i.e., pushing the string onto the stack or copying it to a new memory location. Instead, have constval defined as by-value *semantics*. In other words, constval would indicate the meaning of the language, not the implementation. It would not be a calling convention. It would not mean the call would be by value; it would mean the semantics would be by value. This is essentially what I originally thought const would mean with AnsiStrings, though it turned out hazy. The programmer must know the language and the compiler must implement the language. How the compiler does that should not determine how the program behaves. If constval is defined in semantics, not implementation, this means the compiler can still take whatever optimizations are possible as long as they do not break those semantics. So you would not have to actually copy strin
Re: [fpc-devel] Const optimization is a serious bug
On Mon, Jul 11, 2011 at 04:09, Hans-Peter Diettrich wrote: >>> If you want to introduce different keywords, then please different for >>> beforementioned cases. E.g. >>> constval x: integer; //all value-types >>> constref r: Trecord; //also ShortString >>> const??? o: TObject; //all references, except following >>> const??? s: AnsiString; //all managed types >> >> I am not quite sure what are you asking here. Can you elaborate? > > To which data type do you want apply your "constval" modifier, with which > concrete effects? To any datatype, of course. The result should be: 1) Parameter is passed by value 2) Parameter modifications are forbidden by compiler -- Alexander S. Klenin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
Alexander Klenin schrieb: 3) Current documentation (http://www.freepascal.org/docs-html/ref/refsu58.html) declares that "const" modifier is "retaining the semantics of passing by value", so there is definitely a bug here -- you can declare it to be bug in specification, but still. Sorry, I cannot find anything like this statement in the link. It is a direct quote -- you can use "find on page" feature of your browser (usually activated by pressing Ctrl+F) to locate it. The whole sentence applies to the byval/byref modification, nothing is said about the cases where references are passed even without const. In this case I agree that aliasing can change the semantics. On the more constructive note, I have yet another proposal: 1) Leave current "const" implementation broken as-is. 2) Introduce new "constval" modifier which is similar to "const", but guarantees correct "by-value" semantics. If you want to introduce different keywords, then please different for beforementioned cases. E.g. constval x: integer; //all value-types constref r: Trecord; //also ShortString const??? o: TObject; //all references, except following const??? s: AnsiString; //all managed types I am not quite sure what are you asking here. Can you elaborate? To which data type do you want apply your "constval" modifier, with which concrete effects? When the meaning of "const"/"constref" is documented as kind of a *calling convention*, that allows the compiler to optimize the code, no more discussion is required. "constref" is a calling convention, but "const" is not, since it is "optimization hint". From the user VP both affect code generation only. Both can be broken by by aliasing. DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
On 10/07/2011 11:33, Marco van de Voort wrote: To be honest, I'm a bit surprised by the progression of this discussion. I sympathise a bit with the proponents of a simple "disable this optimization" switch, but to me the rest all sounds like people desperately defending their turf, and losing sight of the big picture. I agree the discussion, if the feature should be as it is, went off, far out off control. And I myself take a considerable share of the blame in making it do so. The discussion was meant to be about, what can or should be done, in addition to the feature. Accepting the feature itself as it is. I think the finegrained detection option that Martin proposes has a bad work vs occuarance tradeoff, but a global killswitch for this might be enough. I do agree. At least from the point of the developer(s) that have to implement it. And I do accept that. Never the mind, that does not mean that not from the users point of view, it would be nice to have. not that it cannot be proposed. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
On 10/07/2011 05:34, Alexander Klenin wrote: As for the whole "optimization hint" angle, I'd like to note that: 1) This is contrary to previous posts in this same thread, where FPC developers insisted that "const" semantics is defined as "no refcounting", which is quite different from a hint. Jonas himself pointed out that this documentation is at least misleading. Anyway: I have at no point said or implied (or at lest have I not mean to do so) that the documentation was perfect. In fact, I do thing the documentation has severe shortcomings. Anyway, so far I have talked about the feature. The feature is correctly implemented (if the defintition is known). 2) If "const" is indeed an optimization hint, that places it in the same category as, say, "inline". What would you say if adding "inline" keyword to a procedure converted working program into randomly crashing one? I'd say this is optimization bug, much like the title of this thread. I guess an inline can make a program crash. I the inlined procedure contains code trying to access it's own stack frame, and being inlined actually access the stackframe of the would-be-caller... I am not going to try to prove this, but I guess with enough determination it can be done. Anyway, as we well know by now "const" is more than that, hence it can not be compared with inline. ...and never mind any issues with the documentation of it. If the docs are wrong, then correct them, but do not blame the feature as bad, because the docs are bad... 3) Current documentation (http://www.freepascal.org/docs-html/ref/refsu58.html) declares that "const" modifier is "retaining the semantics of passing by value", so there is definitely a bug here -- you can declare it to be bug in specification, but still. No argument with that. As I pointed out, the correctness (or incorrectness) were never my point On the more constructive note, I have yet another proposal: 1) Leave current "const" implementation broken as-is. You made a typo: 1) Leave current "const" implementation WORKING as-is. ;-p SCNR 2) Introduce new "constval" modifier which is similar to "const", but guarantees correct "by-value" semantics. no problem with that. 3) Slowly migrate existing code to either "constref" or "constval", use "const" for legacy/compatibility/extreme optimization cases. Maybe... BUT: I have actually used "const" in some places with exactly the intend to have it doing what it does. I have used const with the very intention of the ref-count not being increased, and by this with the effect of the implicit exception frame not being needed. Given that, I would not migrate that code to anything else. BTW: when I first did that, I made the very mistake of doing it wrong myself too. I learned, I corrected my code, I started using it as it is meant to.. Best Regards Martin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
On Sun, Jul 10, 2011 at 12:01 PM, Alexander Klenin wrote: > On Sun, Jul 10, 2011 at 18:53, Hans-Peter Diettrich > wrote: >> When the meaning of "const"/"constref" is documented as kind of a *calling >> convention*, that allows the compiler to optimize the code, no more >> discussion is required. > > "constref" is a calling convention, but "const" is not, since it is > "optimization hint". > As you said above, this is the reason why "constref" was invented in > the first place. > Although I liked constref at first, I'd rather think that the current behavior is more of a side effect of the current implementation. I see that it was introduce to support well-known c/c++ concept of "give me a pointer to the thing and I promise I won't change the thing" used in many apis. But since ansistring is not used in any universal api, the compiler currently is free to change the meaning and for example pass not the address of the entry that hold the pointer, but pointer itself. In other words the compiler if free to choose because lazarus/fpc is the only place on earth where the pair constref-ansistring exists (currently). So the constref immunity is just a side effect And returning to the original discussion currently I'd like to look from another point (or maybe just summation of previous thoughts of others). I think that anything passed by reference (pure reference) pose the problem OP described, even if you use current "immune" constref. For example, when you want to investigate every char of the string you will probably use for i:=1 to Length(Str). As I recall taking length in advance and using it for every step is even the part Wirth's pascal as the requirement. If so, during the loop a callback or just some call shorten the string, the loop will fail almost for sure. Not because unexpectedly the string is invalid, but because the loop expects the string to be longer that it is now. And the problem like this also can be hard to detect. So personally I'd not separate the reference-based problems like this from the problems posed by const. We can blame the compiler for some optimization results (rarely), but we can not blame the programmer who cached the length of the string in a local variable thinking that the string will be same till the end of the function. And imho understanding that referencing can be dangerous in this regard should be a part of knowledge for every programmer. Some good news )... If we look at this, we will see that the problem exists for every data passed by reference in any language. But in contrary to some languages we have a luxury of passing by value not only register-sized data, but also arge data. And to make things even better we have a luxury of having a tricky ansistring with its copy-on-write concept. So for those who appreciate this luxury and afraid of reference-based problems, using passing ansistrings by value is the answer. Remember, it's a luxury ;) Max ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
In our previous episode, Alexander Klenin said: > 2) If "const" is indeed an optimization hint, that places it in the > same category as, > say, "inline". What would you say if adding "inline" keyword to a procedure > converted working program into randomly crashing one? I'd say this > is optimization bug, This sentence is the main problem. If you define "working" as any code that happens to run must remain working, you are moving away from building a compiler, and starting to build an emulator. A compiler produces a defined output for a defined input. But the input is defined as adhering to the language rules. An emulator produces a defined output for a set of inputs (existing programs). The input is defined as is. Of course in a project the size of FPC not every corner case is documented to the last detail, so the rules are made up as we go. And of course the frequency of the problem has a say in it too. To be honest, I'm a bit surprised by the progression of this discussion. I sympathise a bit with the proponents of a simple "disable this optimization" switch, but to me the rest all sounds like people desperately defending their turf, and losing sight of the big picture. Specially since I already discussed this issue a bit with Martin before the email discussion began. The main argumentation for this switch for me is that in many cases programmers don't have complete control of the codebase (yet). Parts are inherited from previous employees, taken of the net or simply bought. In such cases remarks like "you shouldn't this" are totally useless since you inherited a mountain of code, which will take years to comb through. If you then have strange crashes, a simple recompile with such switch might give you some information if such problem is happening, or not, somewhere deep in your framework. Just like you compile once with -gtt with -gh, run it through valgrind etc etc. I think the finegrained detection option that Martin proposes has a bad work vs occuarance tradeoff, but a global killswitch for this might be enough. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
On Sun, Jul 10, 2011 at 18:53, Hans-Peter Diettrich wrote: > Alexander Klenin schrieb: >> 1) This is contrary to previous posts in this same thread, where >> FPC developers insisted that "const" semantics is defined as >> "no refcounting", which is quite different from a hint. > Refcounting is one optimization aspect, passing large data byref another one. Yes, but the argument was made that "no refcounting" is the required feature of const, not merely an "optimization aspect". >> What would you say if adding "inline" keyword to a procedure >> converted working program into randomly crashing one? I'd say this >> is optimization bug, much like the title of this thread. > > IMO the situation is very different for "inline". Both optimizations have > very little in common (from the compiler VP). Having no common implementation code does not prevent those features from being similar from user's POV. BTW they are not as independent as you might think -- did you know that adding "inline" currently changes semantics of "const"? >> 3) Current documentation >> (http://www.freepascal.org/docs-html/ref/refsu58.html) >> declares that "const" modifier is "retaining the semantics of >> passing by value", >> so there is definitely a bug here -- you can declare it to be bug in >> specification, but still. > > Sorry, I cannot find anything like this statement in the link. It is a direct quote -- you can use "find on page" feature of your browser (usually activated by pressing Ctrl+F) to locate it. >> On the more constructive note, I have yet another proposal: >> 1) Leave current "const" implementation broken as-is. >> 2) Introduce new "constval" modifier which is similar to "const", >> but guarantees correct "by-value" semantics. > > If you want to introduce different keywords, then please different for > beforementioned cases. E.g. > constval x: integer; //all value-types > constref r: Trecord; //also ShortString > const??? o: TObject; //all references, except following > const??? s: AnsiString; //all managed types I am not quite sure what are you asking here. Can you elaborate? > BTW the "constref" keyword has been introduced for *external* subroutines > (MacOS ABI), which require that a value is always passed by reference, > independent from SizeOf(param) and SizeOf(pointer). Its use should be > restricted accordingly. Why? It is well-defined and useful construct, certainly better then "const". >> 3) Slowly migrate existing code to either "constref" or "constval", >> use "const" for legacy/compatibility/extreme optimization cases. > > "const" is already used far too often in the FPC/Lazarus libraries. Totally agree. > The compiler instead should treat it as an error, when an unmanaged (object) > reference is used with "const". Good idea, BTW! Probably error is out of question for compatibility reasons, but a warning would certainly be good. FPC developers: maybe I'll create feature request for this? It is quite independent of the rest of thread. > When the meaning of "const"/"constref" is documented as kind of a *calling > convention*, that allows the compiler to optimize the code, no more > discussion is required. "constref" is a calling convention, but "const" is not, since it is "optimization hint". As you said above, this is the reason why "constref" was invented in the first place. -- Alexander S. Klenin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
Alexander Klenin schrieb: Martin, I do not know why you and Hans suddently have an urge to insult Chad, but he really did not deserve this. Perhaps a threshold was reached, when the same wrong argumentation was refrained too often. As for the whole "optimization hint" angle, I'd like to note that: 1) This is contrary to previous posts in this same thread, where FPC developers insisted that "const" semantics is defined as "no refcounting", which is quite different from a hint. Refcounting is one optimization aspect, passing large data byref another one. 2) If "const" is indeed an optimization hint, that places it in the same category as, say, "inline". What would you say if adding "inline" keyword to a procedure converted working program into randomly crashing one? I'd say this is optimization bug, much like the title of this thread. IMO the situation is very different for "inline". Both optimizations have very little in common (from the compiler VP). 3) Current documentation (http://www.freepascal.org/docs-html/ref/refsu58.html) declares that "const" modifier is "retaining the semantics of passing by value", so there is definitely a bug here -- you can declare it to be bug in specification, but still. Sorry, I cannot find anything like this statement in the link. The addressed byval/byref move doesn't change the semantics, provided that the item is not modified in user code - this can be verified for records or other simple types by the compiler. The situation is very different for parameters, which already *are* references. In these cases the passed object can be modified, regardless of "const", the compiler only checks that the reference *itself* is not changed. The situation is different again for *managed* types, which also are always passed byref. Here the removal of refcounting applies. On the more constructive note, I have yet another proposal: 1) Leave current "const" implementation broken as-is. 2) Introduce new "constval" modifier which is similar to "const", but guarantees correct "by-value" semantics. If you want to introduce different keywords, then please different for beforementioned cases. E.g. constval x: integer; //all value-types constref r: Trecord; //also ShortString const??? o: TObject; //all references, except following const??? s: AnsiString; //all managed types BTW the "constref" keyword has been introduced for *external* subroutines (MacOS ABI), which require that a value is always passed by reference, independent from SizeOf(param) and SizeOf(pointer). Its use should be restricted accordingly. 3) Slowly migrate existing code to either "constref" or "constval", use "const" for legacy/compatibility/extreme optimization cases. "const" is already used far too often in the FPC/Lazarus libraries. The compiler instead should treat it as an error, when an unmanaged (object) reference is used with "const". It's kidding when somebody requires that the compiler prevents him from inadvertently modifying an given value parameter. He who doesn't have enough coding discipline to care for such restrictions himself, will make more logical errors that no compiler can detect. When the meaning of "const"/"constref" is documented as kind of a *calling convention*, that allows the compiler to optimize the code, no more discussion is required. DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel