Re: [fpc-devel] Const optimization is a serious bug
Alexander Klenin schrieb: Martin, I do not know why you and Hans suddently have an urge to insult Chad, but he really did not deserve this. Perhaps a threshold was reached, when the same wrong argumentation was refrained too often. As for the whole optimization hint angle, I'd like to note that: 1) This is contrary to previous posts in this same thread, where FPC developers insisted that const semantics is defined as no refcounting, which is quite different from a hint. Refcounting is one optimization aspect, passing large data byref another one. 2) If const is indeed an optimization hint, that places it in the same category as, say, inline. What would you say if adding inline keyword to a procedure converted working program into randomly crashing one? I'd say this is optimization bug, much like the title of this thread. IMO the situation is very different for inline. Both optimizations have very little in common (from the compiler VP). 3) Current documentation (http://www.freepascal.org/docs-html/ref/refsu58.html) declares that const modifier is retaining the semantics of passing by value, so there is definitely a bug here -- you can declare it to be bug in specification, but still. Sorry, I cannot find anything like this statement in the link. The addressed byval/byref move doesn't change the semantics, provided that the item is not modified in user code - this can be verified for records or other simple types by the compiler. The situation is very different for parameters, which already *are* references. In these cases the passed object can be modified, regardless of const, the compiler only checks that the reference *itself* is not changed. The situation is different again for *managed* types, which also are always passed byref. Here the removal of refcounting applies. On the more constructive note, I have yet another proposal: 1) Leave current const implementation broken as-is. 2) Introduce new constval modifier which is similar to const, but guarantees correct by-value semantics. If you want to introduce different keywords, then please different for beforementioned cases. E.g. constval x: integer; //all value-types constref r: Trecord; //also ShortString const??? o: TObject; //all references, except following const??? s: AnsiString; //all managed types BTW the constref keyword has been introduced for *external* subroutines (MacOS ABI), which require that a value is always passed by reference, independent from SizeOf(param) and SizeOf(pointer). Its use should be restricted accordingly. 3) Slowly migrate existing code to either constref or constval, use const for legacy/compatibility/extreme optimization cases. IMO const is already used far too often in the FPC/Lazarus libraries. The compiler instead should treat it as an error, when an unmanaged (object) reference is used with const. It's kidding when somebody requires that the compiler prevents him from inadvertently modifying an given value parameter. He who doesn't have enough coding discipline to care for such restrictions himself, will make more logical errors that no compiler can detect. When the meaning of const/constref is documented as kind of a *calling convention*, that allows the compiler to optimize the code, no more discussion is required. /IMO DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
On Sun, Jul 10, 2011 at 12:01 PM, Alexander Klenin kle...@gmail.com wrote: On Sun, Jul 10, 2011 at 18:53, Hans-Peter Diettrich drdiettri...@aol.com wrote: When the meaning of const/constref is documented as kind of a *calling convention*, that allows the compiler to optimize the code, no more discussion is required. constref is a calling convention, but const is not, since it is optimization hint. As you said above, this is the reason why constref was invented in the first place. Although I liked constref at first, I'd rather think that the current behavior is more of a side effect of the current implementation. I see that it was introduce to support well-known c/c++ concept of give me a pointer to the thing and I promise I won't change the thing used in many apis. But since ansistring is not used in any universal api, the compiler currently is free to change the meaning and for example pass not the address of the entry that hold the pointer, but pointer itself. In other words the compiler if free to choose because lazarus/fpc is the only place on earth where the pair constref-ansistring exists (currently). So the constref immunity is just a side effect And returning to the original discussion currently I'd like to look from another point (or maybe just summation of previous thoughts of others). I think that anything passed by reference (pure reference) pose the problem OP described, even if you use current immune constref. For example, when you want to investigate every char of the string you will probably use for i:=1 to Length(Str). As I recall taking length in advance and using it for every step is even the part Wirth's pascal as the requirement. If so, during the loop a callback or just some call shorten the string, the loop will fail almost for sure. Not because unexpectedly the string is invalid, but because the loop expects the string to be longer that it is now. And the problem like this also can be hard to detect. So personally I'd not separate the reference-based problems like this from the problems posed by const. We can blame the compiler for some optimization results (rarely), but we can not blame the programmer who cached the length of the string in a local variable thinking that the string will be same till the end of the function. And imho understanding that referencing can be dangerous in this regard should be a part of knowledge for every programmer. Some good news )... If we look at this, we will see that the problem exists for every data passed by reference in any language. But in contrary to some languages we have a luxury of passing by value not only register-sized data, but also arge data. And to make things even better we have a luxury of having a tricky ansistring with its copy-on-write concept. So for those who appreciate this luxury and afraid of reference-based problems, using passing ansistrings by value is the answer. Remember, it's a luxury ;) Max ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
On 10/07/2011 05:34, Alexander Klenin wrote: As for the whole optimization hint angle, I'd like to note that: 1) This is contrary to previous posts in this same thread, where FPC developers insisted that const semantics is defined as no refcounting, which is quite different from a hint. Jonas himself pointed out that this documentation is at least misleading. Anyway: I have at no point said or implied (or at lest have I not mean to do so) that the documentation was perfect. In fact, I do thing the documentation has severe shortcomings. Anyway, so far I have talked about the feature. The feature is correctly implemented (if the defintition is known). 2) If const is indeed an optimization hint, that places it in the same category as, say, inline. What would you say if adding inline keyword to a procedure converted working program into randomly crashing one? I'd say this is optimization bug, much like the title of this thread. I guess an inline can make a program crash. I the inlined procedure contains code trying to access it's own stack frame, and being inlined actually access the stackframe of the would-be-caller... I am not going to try to prove this, but I guess with enough determination it can be done. Anyway, as we well know by now const is more than that, hence it can not be compared with inline. ...and never mind any issues with the documentation of it. If the docs are wrong, then correct them, but do not blame the feature as bad, because the docs are bad... 3) Current documentation (http://www.freepascal.org/docs-html/ref/refsu58.html) declares that const modifier is retaining the semantics of passing by value, so there is definitely a bug here -- you can declare it to be bug in specification, but still. No argument with that. As I pointed out, the correctness (or incorrectness) were never my point On the more constructive note, I have yet another proposal: 1) Leave current const implementation broken as-is. sarcasm You made a typo: 1) Leave current const implementation WORKING as-is. ;-p SCNR /sarcasm 2) Introduce new constval modifier which is similar to const, but guarantees correct by-value semantics. no problem with that. 3) Slowly migrate existing code to either constref or constval, use const for legacy/compatibility/extreme optimization cases. Maybe... BUT: I have actually used const in some places with exactly the intend to have it doing what it does. I have used const with the very intention of the ref-count not being increased, and by this with the effect of the implicit exception frame not being needed. Given that, I would not migrate that code to anything else. BTW: when I first did that, I made the very mistake of doing it wrong myself too. I learned, I corrected my code, I started using it as it is meant to.. Best Regards Martin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
On 10/07/2011 11:33, Marco van de Voort wrote: To be honest, I'm a bit surprised by the progression of this discussion. I sympathise a bit with the proponents of a simple disable this optimization switch, but to me the rest all sounds like people desperately defending their turf, and losing sight of the big picture. I agree the discussion, if the feature should be as it is, went off, far out off control. And I myself take a considerable share of the blame in making it do so. The discussion was meant to be about, what can or should be done, in addition to the feature. Accepting the feature itself as it is. I think the finegrained detection option that Martin proposes has a bad work vs occuarance tradeoff, but a global killswitch for this might be enough. I do agree. At least from the point of the developer(s) that have to implement it. And I do accept that. Never the mind, that does not mean that not from the users point of view, it would be nice to have. not that it cannot be proposed. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
Alexander Klenin schrieb: 3) Current documentation (http://www.freepascal.org/docs-html/ref/refsu58.html) declares that const modifier is retaining the semantics of passing by value, so there is definitely a bug here -- you can declare it to be bug in specification, but still. Sorry, I cannot find anything like this statement in the link. It is a direct quote -- you can use find on page feature of your browser (usually activated by pressing Ctrl+F) to locate it. The whole sentence applies to the byval/byref modification, nothing is said about the cases where references are passed even without const. In this case I agree that aliasing can change the semantics. On the more constructive note, I have yet another proposal: 1) Leave current const implementation broken as-is. 2) Introduce new constval modifier which is similar to const, but guarantees correct by-value semantics. If you want to introduce different keywords, then please different for beforementioned cases. E.g. constval x: integer; //all value-types constref r: Trecord; //also ShortString const??? o: TObject; //all references, except following const??? s: AnsiString; //all managed types I am not quite sure what are you asking here. Can you elaborate? To which data type do you want apply your constval modifier, with which concrete effects? When the meaning of const/constref is documented as kind of a *calling convention*, that allows the compiler to optimize the code, no more discussion is required. constref is a calling convention, but const is not, since it is optimization hint. From the user VP both affect code generation only. Both can be broken by by aliasing. DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
On Mon, Jul 11, 2011 at 04:09, Hans-Peter Diettrich drdiettri...@aol.com wrote: If you want to introduce different keywords, then please different for beforementioned cases. E.g. constval x: integer; //all value-types constref r: Trecord; //also ShortString const??? o: TObject; //all references, except following const??? s: AnsiString; //all managed types I am not quite sure what are you asking here. Can you elaborate? To which data type do you want apply your constval modifier, with which concrete effects? To any datatype, of course. The result should be: 1) Parameter is passed by value 2) Parameter modifications are forbidden by compiler -- Alexander S. Klenin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
Some thoughts on the meaning of const, constref, and constval, and how they can usefully be applied: My initial understanding of const was hazy. I have come to appreciate that it is defined, but only in a very loose way. Instead of undefined I should say ill-defined. I'm not entirely sure what some of the attacks have been about, but I will acknowledge that I was not 100% sure about const when I started. What I have said about it being unclear is certainly true though. Let me explain how, and what I think a better idea would be. I wrote: If we *knew* that const meant it would be by reference, then that immediately eliminates the confusion in the case of ShortString and records; modifying one instance through several references affects them all, as expected. What the programmer says happens; there can be no bug, except in the programmer's own knowledge. Martin wrote: We do know: http://www.freepascal.org/docs-html/ref/refsu58.html#x135-14500011.4.4 A constant argument is passed by reference if its size is larger than a pointer. It is passed by value if the size is equal or is less then the size of a native pointer. Jonas wrote: A constant argument is passed by reference if its size is larger than a pointer. It is passed by value if the size is equal or is less then the size of a native pointer. That part of the manual is wrong. What const does is by design completely implementation-dependent I should not say const is undefined. Actually I mean const is ambiguous; sort of defined as undefined, in that: 1) with the current definition, it changes by platform, 2) the current definition is wrong anyway, and 3) it is implementation dependent, which I interpret to mean it could change in the future. If the meaning of const is entirely implementation-dependent, and if I interpret that correctly, I think we really cannot assume anything about what exactly will happen with regard to the calling convention, which is what I was worried about earlier. Now, constref was intended for compatibility with other languages, but I think it is a valuable addition to Pascal in its own right and that it would be wise to use. The hypothetical constval modifier is a natural corollary to this. In the current implementation, if you were to pass a record to a procedure for example, you have two choices: 1) pass it by value but have it be non-const, 2) pass it as const, which might be by reference or value. With constval and constref you would have the other two highly useful choices: 1) pass by reference and it's const, 2) pass by value and it's const. Without making assumptions about const, which I don't think we can, these two options do not currently exist, so I think these new keywords are a useful addition. Some might say that it doesn't matter whether it is passed by value or reference: if you pass it as const, either way you promise not to modify it. See, that is my concern: what is IT? People have used other words, but it ultimately comes down to: if you say the programmer promises not to modify the thing passed as const, what thing *exactly* is that? A variable, reference, or instance? With const, we don't really know. With constref, it means you promise not to modify the memory location (instance) pointed to by that reference. With constval it means that you can change the memory location/instance that was passed into the procedure, since the procedure is now using it's own copy anyway; you just can't modify the instance that the procedure now has it's own copy of. One additional problem does arise. Constval implies that the implementation is pass-by-value. However in many (I'd say most) cases it is quite possible that we could be interested only in the semantics of pass-by-value, not the implementation. So for AnsiStrings, neither constref nor constval would be suitable. Constref would mean it must be by reference, but we want by-value semantics. Constval would mean that the string has to be copied to a separate memory location, which we might not really care about, and is slow and wasteful. So, I propose: don't have constval literally mean it is passed by value, i.e., pushing the string onto the stack or copying it to a new memory location. Instead, have constval defined as by-value *semantics*. In other words, constval would indicate the meaning of the language, not the implementation. It would not be a calling convention. It would not mean the call would be by value; it would mean the semantics would be by value. This is essentially what I originally thought const would mean with AnsiStrings, though it turned out hazy. The programmer must know the language and the compiler must implement the language. How the compiler does that should not determine how the program behaves. If constval is defined in semantics, not implementation, this means the compiler can still take whatever optimizations are possible as long as they do not break those semantics. So you would not have to actually copy strings,
Re: [fpc-devel] Const optimization is a serious bug
On 11 Jul 2011, at 00:03, Daniël Mantione wrote: Calling conventions should in principle be eternally frozen. This includes const, because for exampe a DLL compiled in version x can be imported by a program compiled by version y. That's indeed a good point. The fact that it has happened (for whatever reason) does not mean it is good practise. True. Jonas___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
On 7/10/2011 2:40 PM, Daniël Mantione wrote: However, for a new calling convention (on an existing or a new platform) it's completely up to the compiler designer what will be passed as const and what will be passed as reference. Agreed. That's why I say you can't really make assumptions: if you assume a certain platform, you can make a limited set of assumptions (very limited as I will show in a moment). However one of FPC's great features is its cross-platform power, so I don't want to make assumptions about the platform. Without assumptions about platform you can make even less assumptions about const. Within a certain calling convention, the behaviour of const is defined and will not change. Even if the calling convention does not change, the semantics can, as currently implemented. Even within a single platform const is ill-defined. Consider the case of AnsiString. There are three cases as it is currently implemented: 1) Ref count 1 when passed as const parameter results in strict pass-by-value semantics 2) Ref count = 1 and you modify the original instance in-place results in the value of the parameter changing 3) Ref count = 1 and you modify the original reference such that the instance's ref count decreases to zero and it may crash or just give weird behavior Although the programmer knows about reference counting, it is not expected for the programmer to know the details of the implementation, which can change. Therefore you cannot be 100% certain what the refcount is at any given time. And therefore you cannot be certain how const will be treated. We know the implementation will be pass-by-ref with no refcount change. The semantics could be pass-by-value, pass-by-ref, or just plain crashing, depending on details we cannot predict. You Pascal code should be able to handle both value *and* reference. After all, with const you are leaving the decision to the compiler. This is what it means in the end. Then your Pascal code should not depend on specific behaviour. Yes, I agree fully. That is the only choice with const. Constref, as it exists, with the addition of constval, provide a more precise form of expression on those occasions where one might feel the ambiguity of const is insufficient. And before anyone says something, yes, I also understand that if you pass something as a const, you have to be prepared in case it is passed by reference, and you have to realize that if it is passed by reference, you have to promise not to modify the instance via another reference. That's OK. I am a relative newcomer to Pascal, having used it about 1.5 years now. It was disturbing that something so important was not documented anywhere, and, in fact, there was documentation to the contrary. Aside from the lacking and sometimes incorrect documentation (I refer not only to FPC but to Delphi), I was surprised when the semantics of const AnsiStrings, which usually turn out to be pass-by-value, actually might not be in rare cases that are hard to predict. Aside from good documentation (which Delphi is in need of too in this particular case), I am proposing constval in conjunction with the existing constref to provide greater expressive power and precision for the programmer. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
Sorry for the additional post; I should have added this to my preceding message. I just want to clarify: I am trying to be very careful to distinguish between implementation and semantics. If my understanding is correct, AnsiString implementation is always pass-by-reference. The problem is the programmer should be forced to stick to certain semantics, not a certain implementation. With const AnsiString, the implementation is consistently pass-by-ref, but the semantics can change as described in my previous message, depending on the instance's refcount when the procedure is invoked and what you may do with it while the procedure is running. It has correctly been pointed out that you therefore can't make an assumption about what exactly a const AnsiString parameter will do; you just have to be prepared in either case, and if it turns out to be by reference, don't modify the instance via another reference. I simply propose constref (existing) and constval (hyptothetical) for those who come across situations where more precise programming would be desired. --- Implementation --- I had a few ideas about implementation of constval, as I described it, for AnsiString. (Or, alternatively, a way to change the behavior of const, though this is no longer what I advocate.) These are all speculative. These are some ideas, basically, not assertions. Also my knowledge of how FPC works is limited as I've said before. 1. It seems that if a string is a (non-const) local variable it should be safe. I base this on the following reasoning: a) In order to trigger the undesired behavior, you have to get an instance with a refcount of 1 which actually has more than one reference to it. The only way to do this is to pass a reference to an instance with a refcount of 1 to a function accepting a const AnsiString parameter. (Aside: The critical value is 1 because that determines whether copy-on-write happens and whether it gets freed the next time the refcount is decremented.) b) Furthermore, you must be able to access the non-const reference at a higher scope (object, class, or global). c) To do this, you could either have a higher-scope variable which you assign to a local variable, or vice versa. d) If you do either of these, the refcount becomes 2 and the problem cannot occur. e) The problem cannot be triggered passing a local variable. 2. Implementing 1 would require that the reference count update can be applied or not applied to a specific function depending how it's called. This could be done in two ways I'm thinking of: a) Move the responsibility for updating the refcount to the caller. There are pros and cons to this idea. It could result in slightly larger code (because the refcount update and try-finally are in more places). However I think it could also speed things up *even more than the current implementation* because the refcount updates for several non-const strings could be combined into a single try-finally in the caller, rather than having one in each function. b) Another possibility is to have two entry points to the function. The update of the refcount and try-finally would remain in the function, not the caller. However the caller could enter the function at either of two entry points. One would do the refcount update and set up the try-finally, whereas the other would skip to the code after that. I don't know if this is even possible. It seemed like something that might be doable with some tweaking when I was looking at the assembly code. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] MySQL 5.1 and Double (trouble)
Can someone check the status of update if the bind parameter for double decimal values? I'm getting unexpected rounding errors after updating - the data in the column looks different than what was specified during the update statement. Value: double; Query.Params.ParamByName(sName).AsFloat:=Value; 1.) Update Value : 40734.825668912039 2.) Actual Value after update : 40734.825668912 3.) Actual Value on read : 40734.825668912003 As far as I know this is a relatively new problem. Could be with Ubuntu but was someone doing MYSQL work lately? ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
On Mon, Jul 11, 2011 at 4:45 AM, Chad Berchek ad...@vobarian.com wrote: 1. It seems that if a string is a (non-const) local variable it should be safe. I base this on the following reasoning: looks like it's not: procedure TForm1.Button1Click(Sender: TObject); var S: string; procedure SideEffectCall; begin SetLength(S, 0); end; function CalcSomething(const T: string): integer; begin Result:=0; SideEffectCall; MessageDlg(T, mtInformation, [mbOk], 0); // crash here end; begin S:='12345678901234567890'; S:=S + S; CalcSomething(S); end; ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
Alexander Klenin schrieb: On Mon, Jul 11, 2011 at 04:09, Hans-Peter Diettrich drdiettri...@aol.com wrote: If you want to introduce different keywords, then please different for beforementioned cases. E.g. constval x: integer; //all value-types constref r: Trecord; //also ShortString const??? o: TObject; //all references, except following const??? s: AnsiString; //all managed types I am not quite sure what are you asking here. Can you elaborate? To which data type do you want apply your constval modifier, with which concrete effects? To any datatype, of course. The result should be: 1) Parameter is passed by value How to pass objects by value? Records with embedded AnsiStrings or other managed types? DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Const optimization is a serious bug
Chad Berchek schrieb: I just want to clarify: I am trying to be very careful to distinguish between implementation and semantics. Implementation is a very precise definition, which can be used to perfectly describe the semantics of a language. A verbose description of the semantics leaves too much room for misunderstandings, and commonly accepted formalisms are at least as hard to understand as implementation details, while not covering all aspects. DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
