Re: [fpc-devel] Suggestion: reference counted objects
Am 21.09.2014 06:11 schrieb Boian Mitov mi...@mitov.com: Unless you implement something like the semi-deterministic GC that I proposed few years ago, weak pointers are the only solution for avoiding circular reference deadlock in ARC as far as I know. They are simply necessary evil, and a small one base on my current experience with them. There are cycle detection algorithms that could be used upon a decrease of a reference count. Biggest problem is that they would be quite expensive (regarding performance) in case of complex object instance graphs like the LCL/VCL. Regards, Sven ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
Am 21.09.2014 um 07:22 schrieb Hans-Peter Diettrich: Boian Mitov schrieb: That is easy. it gets incremented when it gets assigned. The running threads have no way of accessing it if there is no reference (assignment) already in place. The problem arises when an object is destroyed, or even elected for destruction in _Release, while another thread starts using the same instance. This is not possible for a correctly written program: if two threads having references to the same instance, the ref. count is 2. So no destruction is possible. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
[fpc-devel] Question about Syntax: I there a reason for this design?
I was looking at the following example in DXE7
http://delphiaball.co.uk/
procedure TFormThreading.Button1Click(Sender: TObject); var aTask: ITask; begin
// not a thread safe snippet aTask := TTask.Create (procedure () begin sleep
(3000); // 3 seconds ShowMessage ('Hello'); end); aTask.Start; end;
I dislike this syntax and tried to write it this way because I think it's
better readable code.
But it's not accepted.
procedure TFormThreading.Button1Click(Sender: TObject); var aTask: ITask;
procedure MyTask; begin sleep (3000); // 3 secondsShowMessage ('Hello');end
begin // not a thread safe snippet aTask := TTask.Create(MyTask); aTask.Start;
end;
I just wonder if there is a valuable reason to do it why my example is not
allowed or would it be easy to make this available in free-pascal?
Thank you.
Sam
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
Il 21/09/2014 05:51, Hans-Peter Diettrich ha scritto: Giuliano Colla schrieb: I might, for example, tell you that my company has been successfully implementing since more than 30 years a class of applications for the control of industrial processes, with hundreds of threads running simultaneously in a multi-CPU environment, [...] IMO realtime applications require an realtime OS, providing all required means of process synchronization and communication. Ordinary systems and developers should be happy with primitive threads, doing their work in the background and exiting when done. Our class of applications takes advantage of a distributed intelligence model. IOW a number of CPU's working in parallel, each one running his own real-time multitasking environment. They're organized in a hierarchical tree and almost all of them take advantage of dedicated real-time OS's, up to the root of the tree, which governs the full process, and also provides human interface, data collection, connection with factory network and data-bases, remote connection for assistance, debugging and upgrading. The top level, until recent times, was provided by a Linux box, with a real-time patched kernel (Adeos-Ipipe). The result was a real-time nanokernel, running the application real-time tasks, and having Linux as its idle task. On Linux side, the visual portion was provided by Kylix. But, with the increase of CPU speed, and the the significant improvements of Linux kernel latency, it's now become possible to get rid of the foreign real-time patch, and to take advantage directly of Linux kernel thread scheduling, without a significant performance loss. It's good to know that FPC allows to implement and manage more complex parallel processing, if I understand you and Boian correctly? FPC comes into play with the latest evolution I described above, because all the high level functions (such as user interface, data collection, etc.) are being handled using Lazarus for the visual functions, and a number of FPC threads for the rest. While Kylix did provide a more suitable (and less buggy :-( ) widgetset, synchronization of the main thread with FPC/Lazarus is significantly better. Giuliano ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Question about Syntax: I there a reason for this design?
On 21/09/14 08:10, Samuel Herzog wrote:
But it's not accepted.
*procedure* TFormThreading.Button1Click(Sender: TObject);
*var*
aTask: ITask;
**procedure* *MyTask*;
*begin*
*sleep (3000); // 3 seconds*
*ShowMessage ('Hello');*
* end*
begin*
// not a thread safe snippet
aTask := TTask.Create(*MyTask*);
aTask.Start;
*end*;
I just wonder if there is a valuable reason to do it why my example is
not allowed or would it be easy to make this available in free-pascal?
In a perfect world: yes, it would be easy to make this available in FPC.
In the real world where people write a ton of code in assembler or use
pointer hacks to expose low level implementation details no one should
rely on: it would probably make a few very vocal people unhappy because
to be able to pass nested routines to reference to procedure/function
parameters, the way they access variables in the parent frame (and the
way they receive the parent frame) needs to be changed. And hence all
these low level hacks would break.
I think that this is, possibly together with the fact that anonymous
methods are a buzzword that you want to be able to add to your feature
list, probably the reason Embarcadero didn't simply use nested routines
instead of adding anonymous methods.
We could of course introduce yet another calling convention modification
for nested routines to work around this (we already have two) so that
both the old and new behaviour can coexist. The downside is that it
makes the compiler more complex and hence harder to modify in the future
and/or easier to break (every such modification needs to be implemented
for every calling convention on every platform).
As you observed, anonymous methods and nested routines are essentially
identical other than the syntax, and I agree the latter generally also
result in better readable code. Therefore I'm still inclined to
implement it somehow (at least for C-blocks, as already partially
implemented in http://svn.freepascal.org/svn/fpc/branches/blocks/),
because it seems a shame not to use an existing, perfectly suited
language construct and instead force everyone to use a new one.
Jonas
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Question about Syntax: I there a reason for this design?
On 21.09.2014 12:13, Jonas Maebe wrote:
On 21/09/14 08:10, Samuel Herzog wrote:
But it's not accepted.
*procedure* TFormThreading.Button1Click(Sender: TObject);
*var*
aTask: ITask;
**procedure* *MyTask*;
*begin*
*sleep (3000); // 3 seconds*
*ShowMessage ('Hello');*
* end*
begin*
// not a thread safe snippet
aTask := TTask.Create(*MyTask*);
aTask.Start;
*end*;
I just wonder if there is a valuable reason to do it why my example is
not allowed or would it be easy to make this available in free-pascal?
In a perfect world: yes, it would be easy to make this available in FPC.
In the real world where people write a ton of code in assembler or use
pointer hacks to expose low level implementation details no one should
rely on: it would probably make a few very vocal people unhappy because
to be able to pass nested routines to reference to procedure/function
parameters, the way they access variables in the parent frame (and the
way they receive the parent frame) needs to be changed. And hence all
these low level hacks would break.
I think that this is, possibly together with the fact that anonymous
methods are a buzzword that you want to be able to add to your feature
list, probably the reason Embarcadero didn't simply use nested routines
instead of adding anonymous methods.
We could of course introduce yet another calling convention modification
for nested routines to work around this (we already have two) so that
both the old and new behaviour can coexist. The downside is that it
makes the compiler more complex and hence harder to modify in the future
and/or easier to break (every such modification needs to be implemented
for every calling convention on every platform).
As you observed, anonymous methods and nested routines are essentially
identical other than the syntax, and I agree the latter generally also
result in better readable code. Therefore I'm still inclined to
implement it somehow (at least for C-blocks, as already partially
implemented in http://svn.freepascal.org/svn/fpc/branches/blocks/),
because it seems a shame not to use an existing, perfectly suited
language construct and instead force everyone to use a new one.
Agreed.
Though I would also love to have real lambda expressions which would
only use an expression, not a statement and neither a block of
statements (though I'm still fighting with the syntax in my mind):
=== code begin ===
procedure FilterGreater(aSomeList: specialize TListLongInt; aValue:
LongInt);
begin
aSomeList.Filter(lambda x: LongInt as x aValue);
end;
=== code end ===
Regards,
Sven
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
On 21.09.2014 05:34, Hans-Peter Diettrich wrote: Sven Barth schrieb: Am 20.09.2014 20:34 schrieb Giuliano Colla A general mechanism to be reliable should take into account all possibilities. If it does, it will block threads even when unnecessary. If it doesn't, it will be unsafe. That would work the same way as it does in interfaces, arrays and strings: using Interlocked*-functions. As I understand Interlocked Inc/Dec funtionality, it only protects the update of the reference counter against interrupts, but not the tests required before/after this update. As a precaution a RefCount should at least be incremented as soon as there exists a *chance*, that the reference is used/copied in some piece of currently active code (thread...). No, why should it? It works pretty well with interfaces, strings and arrays (let's put aside modifying the latter two, because COW is not needed here). Regards, Sven ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
In our previous episode, Hans-Peter Diettrich said: IMO Weak references should be reserved for users who accept possible consequential problems, but should never be used in standard libraries. At least I'd suggest to make weak references subject to an compiler switch, so that every user has a chance to disable them in case of trouble. IMHO weak references trade one manual memory system in for a different manual memory system. The hard part of manual memory systems, figuring out how a complex dynamic structure deallocates (that is usually tackled by having a bit of design and thought go into it), remains. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
On 20.09.2014 13:42, Sven Barth wrote: On 20.09.2014 13:11, Peter Popov wrote: Please do not reference count TObject. This is a uniquely bad and unnecessary idea. I will switch to ANSI C if you guys do it Please enlighten me why you think it is bad. Give reasons and don't be like a farmer doesn't eat what he doesn't know (rough translation of a German proverb). One just needs to provoke the others correctly and then the discussion starts. Great! :) I've thought about the topic a bit more (I've already thought about it quite much in the past months) and came up with the following RFC: - No reference counting for classes by default - Reference counting can be introduced to a part of the class hierarchy by declaring it as refcounted like this (Note: syntax is not final, but is based on other syntaxes we have (namely sealed and abstract)): === code begin === type TARCObject = class refcounted(TObject) end; === code end === - a reference counted class (and its child classes) would include a reference count field that the compiler knows how to access (for automatic reference counting) and which can be accessed through RTTI (for manual reference counting); it is *not* exposed as a regular field as this might lead to identifier conflicts - only *variables* (or parameters) that have a reference counted class as its type will be subject to ARC; in extension this means that assignments from/to a variable of a not reference counted base class (e.g. TObject) will *not* change the reference count. Take this code for example: === code begin === function CreateObject: TObject; begin Result := TARCObject.Create; end; === code end === This will most likely result in CreateObject returning an instance to class that is already freed (because the only reference inside CreateObject is a (hypothetical) temporary of type TARCObject which goes out of scope once CreateObject returns) - to remedy this TObject is extended with non-virtual methods that allow manual reference counting and would rely on the RTTI data I mentioned (let's call the methods AddRef, Release, IsReferenceCounted and RefCount for now, which can also be used to hook up the reference counting of IUnknown interfaces); the code from above would then look like this to make it safe: === code begin === function CreateObject: TObject; begin Result := TARCObject.Create; Result.AddRef; end; === code end === - TObject.Free would be extended to take reference counting into account as well. If the object is reference counted (IsReferenceCounted returns true) it will call Release and otherwise it will continue to Destroy. - there would be a TARCObject declared in System which is a direct descendant of TObject, but with reference counting enabled; same maybe also for TInterfacedObject - all classes can now have operator overloads as well though it should be warned in the documentation that non-reference counted objects might result in memory leaks there - this now only leaves the problems of cycles; take this code: === code begin === type TSomeClass = class(TARCObject) Children: specialize TListTSomeClass; Owner: TSomeClass; constructor Create(aOwner: TSomeClass); end; constructor TSomeClass.Create(aOwner: TSomeClass); begin Children := specialize TListTSomeClass.Create; Owner := aOwner; if Assigned(Owner) then Owner.Children.Add(Self); end; procedure Test; var t1, t2: TSomeClass; begin t1 := TSomeClass.Create(Nil); t2 := TSomeClass.Create(t1); // do something end; === code end === Now once Test is left it would leave the instances which were assigned to t1 and t2 hanging, because they have references to each other. There are (as far as I see) three ways to solve this: * provide a way to break the circle (in this example e.g. setting Owner to Nil before leaving Test; this is what Delphi provides with the DisposeOf virtual method) * introduce weak references which would disable reference counting, e.g.: === code begin === type TSomeClass = class(TARCObject) // ... Owner: TSomeClass weak; // ... end; === code end === Now the TSomeClass.Create(t1) line in Test wouldn't increase the reference count of t1 further and thus both class instances would be destroyed after Test is left. * provide a possibilty to execute a cycle detection algorithm during the Release part of the reference counting; this has the benefit of avoiding the need for weak, but there would be the problem that the algorithm can be potentially expensive especially with large object instance hierarchies (think LCL here) and this would also need to be executed for *each* decrement of the reference count, thus for both the automated one (which till now could have been rather efficient) and the manual one. I think this approach would allow the best of both worlds and if someone really wants to have TObject reference counted (and face the potential consequences), then he/she
Re: [fpc-devel] Suggestion: reference counted objects
In our previous episode, Sven Barth said: (as sb purely interested from an academic viewpoint, I have no real need for this) - a reference counted class (and its child classes) would include a reference count field that the compiler knows how to access (for automatic reference counting) and which can be accessed through RTTI (for manual reference counting); it is *not* exposed as a regular field as this might lead to identifier conflicts Does this mean multiple roots? That would make creating helpers for the refcounting different (since the field would be at a different offset everwhere) - only *variables* (or parameters) that have a reference counted class as its type will be subject to ARC; in extension this means that assignments from/to a variable of a not reference counted base class (e.g. TObject) will *not* change the reference count. Take this code for example: One of the performance sinks with refcounting is passing parameters through a call chain. (e.g. a class calling an inner class in a field to do the actual work, inherited() etc). How do you think to tackle that? With const? Maybe that can This will most likely result in CreateObject returning an instance to class that is already freed (because the only reference inside CreateObject is a (hypothetical) temporary of type TARCObject which goes out of scope once CreateObject returns) It seems to me that implementation details of the return value handling (hidden/pseudo temp var etc) now trickle into the ARC design. Wouldn't it be more logical to make the result temp var a bit special so that it is not released when returned (requiring a func(); // ignores return value to pop the result and decrease the refcount? ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
On 21.09.2014 13:20, Marco van de Voort wrote:
In our previous episode, Sven Barth said:
(as sb purely interested from an academic viewpoint, I have no real need for
this)
Understood.
- a reference counted class (and its child classes) would include a
reference count field that the compiler knows how to access (for
automatic reference counting) and which can be accessed through RTTI
(for manual reference counting); it is *not* exposed as a regular field
as this might lead to identifier conflicts
Does this mean multiple roots? That would make creating helpers for the
refcounting different (since the field would be at a different offset
everwhere)
No, not multiple roots. If a class is declared as refcounted (and it's
parent class is not) then a hidden field is inserted into its list of
fields, just as a normal one would, only that it isn't accesible using
the .-operator (because it won't have a valid Pascal identifier). The
compiler of course knows the offset at compile time and at runtime the
field can be accessed using the class' RTTI which would have a
RefCountOffset field added (if that offset is 0 then the object does
not support reference counting).
- only *variables* (or parameters) that have a reference counted class
as its type will be subject to ARC; in extension this means that
assignments from/to a variable of a not reference counted base class
(e.g. TObject) will *not* change the reference count. Take this code for
example:
One of the performance sinks with refcounting is passing parameters through
a call chain. (e.g. a class calling an inner class in a field to do the
actual work, inherited() etc).
How do you think to tackle that? With const?
Maybe that can
In the same way it's a problem with strings, interfaces and arrays. They
all react accordingly to var/const/constref by not changing the
reference count. Reference counted classes would follow the same rules
(consistency here). And of course as a user of reference counted classes
one needs to be aware of the potential performance impact.
Did anyone till now really complain about the performance impact of
reference counting on interfaces? Especially since they are used *so*
much in today's OOP designs?
This will most likely result in CreateObject returning an instance to
class that is already freed (because the only reference inside
CreateObject is a (hypothetical) temporary of type TARCObject which
goes out of scope once CreateObject returns)
It seems to me that implementation details of the return value handling
(hidden/pseudo temp var etc) now trickle into the ARC design.
Wouldn't it be more logical to make the result temp var a bit special so
that it is not released when returned (requiring a
func(); // ignores return value
to pop the result and decrease the refcount?
The function result is just a special case. This can also be achived
with normal variables or parameters so it is important to have a
consistent handling here. The only way to consistently handle this
without making *all* object variables subject to reference counting
again is to not have assignments of reference counted object instances
from/to non-reference counted variables influence the reference count in
any way.
Also this is compatible to how interfaces work, though the analogous
interface example is highly constructed:
=== code begin ===
program tintfrefcount;
{$mode objfpc}
type
TTestClass = class(TInterfacedObject, IInterface)
destructor Destroy; override;
end;
destructor TTestClass.Destroy;
begin
Writeln('Destroy');
inherited;
end;
function Test: TObject;
begin
Result := (TTestClass.Create as IInterface) as TTestClass;
end;
var
o: TObject;
begin
o := Test;
Writeln('After Test');
o.Free;
end.
=== code end ===
Will result in:
=== output begin ===
Destroy
After Test
=== output end ===
Regards,
Sven
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
On 21.09.2014 14:15, Chriss Kalogeropoulos wrote: Hi Sven, This design means that old classes need to be subclassed in order to enable this feature, correct? Would it be possible instead of declaring this on the class definition, to declare it on the actual variable? Something like var aArcObj: TFooObj; refcounted; Or similar. This would allow the same classes to be used both ways. No. You would need to have a place where to store the reference count and one of the objections some people have is the size increase of TObject if it would get a RefCount field (which would be the only solution with your approach). Also you'd need to take care that all variables where you pass along your object instance are marked as refcounted. This just complicates things too much (both for programmers using this as well as for the compiler). Also how about a smart pointer record implementation with generics? Is it possible ? What's missing is the possiblity to hook into the initialization (unlike interfaces, strings and arrays class variables fields are not initialized with Nil) and finalization (you need to adjust the reference count) of records. And also it's currently not possible to overload the assignment operator if both types have the same record type. Regards, Sven ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
In our previous episode, Sven Barth said: No, not multiple roots. If a class is declared as refcounted (and it's parent class is not) then a hidden field is inserted into its list of fields, just as a normal one would, only that it isn't accesible using the .-operator (because it won't have a valid Pascal identifier). The compiler of course knows the offset at compile time and at runtime the field can be accessed using the class' RTTI which would have a RefCountOffset field added (if that offset is 0 then the object does not support reference counting). So in summary: the compiler must pass the fieldoffset to the runtime helper, instead of having one refcounted root with known offset. Did anyone till now really complain about the performance impact of reference counting on interfaces? Especially since they are used *so* much in today's OOP designs? I never really benchmarked it. I never use interfaces in my own abstract data structure designs, mostly because the existing classes don't, and I don't like the refcount. (since i usually also need to access the classes as normal instances) I did benchmark interfaces in combination with variants as part of a decal evaluation. That was, euh, painful. function Test: TObject; begin Result := (TTestClass.Create as IInterface) as TTestClass; end; Hmm. This looks like a (Delphi) faq item to me, how to convert interface back to a class. Usually the answer is IComponentreference (or what it is called), if this is legal, I wonder why it didn't come up before. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
On 21.09.2014 15:09, Marco van de Voort wrote: In our previous episode, Sven Barth said: No, not multiple roots. If a class is declared as refcounted (and it's parent class is not) then a hidden field is inserted into its list of fields, just as a normal one would, only that it isn't accesible using the .-operator (because it won't have a valid Pascal identifier). The compiler of course knows the offset at compile time and at runtime the field can be accessed using the class' RTTI which would have a RefCountOffset field added (if that offset is 0 then the object does not support reference counting). So in summary: the compiler must pass the fieldoffset to the runtime helper, instead of having one refcounted root with known offset. Yes. That avoids having a single rooted refcounted object and thus allowing the usage of refcounted further down the inheritance tree. Considering that the helper would only consist of a InterlockedIncrement(AddressOfRefCount) or InterlocedDecrement(AddressOfRecCount) one could optimize this by declaring the helper compilerproc as inline, thus avoiding one call. Did anyone till now really complain about the performance impact of reference counting on interfaces? Especially since they are used *so* much in today's OOP designs? I never really benchmarked it. I never use interfaces in my own abstract data structure designs, mostly because the existing classes don't, and I don't like the refcount. (since i usually also need to access the classes as normal instances) I did benchmark interfaces in combination with variants as part of a decal evaluation. That was, euh, painful. I can imagine that benchmarking variants was painful. ;) But considering that interfaces are so en vogue today in Delphi communities the performance hit might not be that killer criteria... function Test: TObject; begin Result := (TTestClass.Create as IInterface) as TTestClass; end; Hmm. This looks like a (Delphi) faq item to me, how to convert interface back to a class. Usually the answer is IComponentreference (or what it is called), if this is legal, I wonder why it didn't come up before. The intf-to-class-as-operator was added with Delphi 2007 AFAIK and FPC supports it as well in 2.6.0. If you want specifics, then look at the helper functions fpc_intf_is_class (for the is operator) and fpc_intf_cast_class (for the as operator) in $fpcdir/rtl/inc/objpas.inc. Regards, Sven ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
On Sun, Sep 21, 2014 at 9:15 AM, Chriss Kalogeropoulos iz.iznog...@gmail.com wrote: Hi Sven, This design means that old classes need to be subclassed in order to enable this feature, correct? Would it be possible instead of declaring this on the class definition, to declare it on the actual variable? Something like var aArcObj: TFooObj; refcounted; Or similar. This would allow the same classes to be used both ways. Also how about a smart pointer record implementation with generics? Is it possible ? Thanks, Chriss Or enabling via compiler switch, like: http://docwiki.embarcadero.com/RADStudio/XE4/en/Automatic_Reference_Counting_in_Delphi_Mobile_Compilers -- Silvio Clécio My public projects - github.com/silvioprog ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
Hans, There is no such problem. We run many hundreds of threads sharing and passing interface data for 11 years already, and in the last 2 years doing it with Anonymous methods as well. I can assure you it works 100%. There is no problem. And since we are component vendor, it also works for all of our customers as well. So far there has not been a single bug report on anything like this. With best regards, Boian Mitov --- Mitov Software www.mitov.com --- -Original Message- From: Hans-Peter Diettrich Sent: Saturday, September 20, 2014 10:22 PM To: FPC developers' list Subject: Re: [fpc-devel] Suggestion: reference counted objects The problem arises when an object is destroyed, or even elected for destruction in _Release, while another thread starts using the same instance. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
The biggest problem is when to initiate the detection and still maintain some form of determinism. That was the algorithm I proposed a while back, but it was not without a drawbacks. It would have affected performance. There is not really a silver bullet for this as far as I know. With best regards, Boian Mitov --- Mitov Software www.mitov.com --- From: Sven Barth Sent: Sunday, September 21, 2014 12:33 AM To: FPC developers' list Subject: Re: [fpc-devel] Suggestion: reference counted objects Am 21.09.2014 06:11 schrieb Boian Mitov mi...@mitov.com: Unless you implement something like the semi-deterministic GC that I proposed few years ago, weak pointers are the only solution for avoiding circular reference deadlock in ARC as far as I know. They are simply necessary evil, and a small one base on my current experience with them. There are cycle detection algorithms that could be used upon a decrease of a reference count. Biggest problem is that they would be quite expensive (regarding performance) in case of complex object instance graphs like the LCL/VCL. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Question about Syntax: I there a reason for this design?
Hi Sam,
You can watch my CodeRage session last year explaining the huge advantages of
the first syntax.
That being said, I consider it a design bug that local functions can’t be
passed as anonymous methods in Delphi. There is no reason for that not to be
done, except that they did not bother.
I hope if FPC implements anonymous methods, that they will also allow local
functions to be passed, and outdo Delphi :-) .
With best regards,
Boian Mitov
---
Mitov Software
www.mitov.com
---
From: Samuel Herzog
Sent: Saturday, September 20, 2014 11:10 PM
To: FPC developers' list
Subject: [fpc-devel] Question about Syntax: I there a reason for this design?
I was looking at the following example in DXE7
http://delphiaball.co.uk/
procedure TFormThreading.Button1Click(Sender: TObject);
var
aTask: ITask;
begin
// not a thread safe snippet
aTask := TTask.Create (procedure ()
begin
sleep (3000); // 3 seconds
ShowMessage ('Hello');
end);
aTask.Start;
end;
I dislike this syntax and tried to write it this way because I think it's
better readable code.
But it's not accepted.
procedure TFormThreading.Button1Click(Sender: TObject);
var
aTask: ITask;
procedure MyTask;
begin
sleep (3000); // 3 seconds
ShowMessage ('Hello');
endbegin
// not a thread safe snippet
aTask := TTask.Create(MyTask);
aTask.Start;
end;I just wonder if there is a valuable reason to do it why my example is not
allowed or would it be easy to make this available in free-pascal?
Thank you.
Sam
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Question about Syntax: I there a reason for this design?
I used to think the same way, until I actually started to use anonymous methods, and I realized what a full I used to be. The simplest difference is that you can return anonymous method as a result of your function and it will preserve all the captured context. Local methods can't do that. Furthermore local methods will not work when passed as parameter to another thread, as they lose context when the owner function exits. You can watch my session on CodeRage last year explaining most of that, and I will be doing some sessions this year on multithreading, and parallel processing where even more will be shown both at ITDevCon and hopefully CodeRage. With best regards, Boian Mitov --- Mitov Software www.mitov.com --- -Original Message- From: Jonas Maebe Sent: Sunday, September 21, 2014 3:13 AM To: FPC developers' list Subject: Re: [fpc-devel] Question about Syntax: I there a reason for this design? On 21/09/14 08:10, Samuel Herzog wrote: As you observed, anonymous methods and nested routines are essentially identical other than the syntax, and I agree the latter generally also result in better readable code. Therefore I'm still inclined to implement it somehow (at least for C-blocks, as already partially implemented in http://svn.freepascal.org/svn/fpc/branches/blocks/), because it seems a shame not to use an existing, perfectly suited language construct and instead force everyone to use a new one. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
Absolutely! Interfaces have been used in threading for decades already in may languages. I have also never heard of anyone having issues with them. With best regards, Boian Mitov --- Mitov Software www.mitov.com --- -Original Message- From: Sven Barth Sent: Sunday, September 21, 2014 3:41 AM To: fpc-devel@lists.freepascal.org Subject: Re: [fpc-devel] Suggestion: reference counted objects On 21.09.2014 05:34, Hans-Peter Diettrich wrote: No, why should it? It works pretty well with interfaces, strings and arrays (let's put aside modifying the latter two, because COW is not needed here). ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
Every memory management system has a manual part, even the M$/Java GC has element of it (the dreadful Disposable pattern to name one). There is a huge difference however. ARC requires in general less coding for the same result. Less code in general is cheaper to develop, debug, read and maintain. It is a simple business formula actually (and as businessman I need to calculate this). There is also the old saying What is the only line of code where you can't have a bug? with the obvious answer - The one you don't write. ;-) . So on the balance sheet ARC is a cheaper and in general easier to maintain form of manual memory management, and my experience with it (Mainly interfaces - almost everything we do is interfaced nowadays), confirms that 100%. With best regards, Boian Mitov --- Mitov Software www.mitov.com --- -Original Message- From: Marco van de Voort Sent: Sunday, September 21, 2014 3:58 AM To: FPC developers' list Subject: Re: [fpc-devel] Suggestion: reference counted objects In our previous episode, Hans-Peter Diettrich said: IMO Weak references should be reserved for users who accept possible consequential problems, but should never be used in standard libraries. At least I'd suggest to make weak references subject to an compiler switch, so that every user has a chance to disable them in case of trouble. IMHO weak references trade one manual memory system in for a different manual memory system. The hard part of manual memory systems, figuring out how a complex dynamic structure deallocates (that is usually tackled by having a bit of design and thought go into it), remains. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
In general I would prefer to see this implemented with attributes rather than with new keywords. For god shake, we already have more keywords in this language than it is healthy. We should be extremely cautions when adding new ones. That is one of the beauty of the attributes. They can eliminate a large number of keywords, and expand the language dynamically (trough the library not trough the compiler). If you implement it with attributes, there is almost no need for any big changes in the compiler at least in the parsing side. Adding new keyword will probably propagate all the way to the tokenizer. With best regards, Boian Mitov --- Mitov Software www.mitov.com --- -Original Message- From: Sven Barth Sent: Sunday, September 21, 2014 4:12 AM To: fpc-devel@lists.freepascal.org Subject: Re: [fpc-devel] Suggestion: reference counted objects === code begin === type TARCObject = class refcounted(TObject) end; === code end === === code begin === type TSomeClass = class(TARCObject) // ... Owner: TSomeClass weak; // ... end; === code end === ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Question about Syntax: I there a reason for this design?
[Jonas: anonymous methods and local functions could use the same syntax] On 21/09/14 18:43, Boian Mitov wrote: I used to think the same way, until I actually started to use anonymous methods, and I realized what a full I used to be. The simplest difference is that you can return anonymous method as a result of your function and it will preserve all the captured context. Local methods can't do that. I'm talking about syntax, not behaviour. When assigning a local function to an anonymous method (reference to ...), the context will have to be captured in exactly the same way as when specifying an inline nested function (aka anonymous method) that requires context. A nested function and a reference to... are different types, so a type conversion would have to be inserted by the compiler anyway (a bit similar to how assigning a method to a procedure of object works, which also captures context, be it in a much simpler way). You could still first keep the context on the stack for quick access by the parent function and later add indirection if necessary (in case it can escape the life of the parent function), as handled by e.g. C blocks. Furthermore local methods will not work when passed as parameter to another thread, as they lose context when the owner function exits. It's the same thing. Jonas ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Question about Syntax: I there a reason for this design?
Hi Jonas, I actually agree that it is a nice feature, and I want to have it both in Delphi and FPC :-) . I prefer the anonymous method syntax as it means less typing, and typing is expensive, however there are cases when I really would love to capture local function. I had few of them already. The problem is even bigger. You can't even call a local function from anonymous method doe to the same capturing issue :-( . This really should have been addressed in Delphi. With best regards, Boian Mitov --- Mitov Software www.mitov.com --- -Original Message- From: Jonas Maebe Sent: Sunday, September 21, 2014 10:23 AM To: FPC developers' list Subject: Re: [fpc-devel] Question about Syntax: I there a reason for this design? [Jonas: anonymous methods and local functions could use the same syntax] I'm talking about syntax, not behaviour. When assigning a local function to an anonymous method (reference to ...), the context will have to be captured in exactly the same way as when specifying an inline nested function (aka anonymous method) that requires context. A nested function and a reference to... are different types, so a type conversion would have to be inserted by the compiler anyway (a bit similar to how assigning a method to a procedure of object works, which also captures context, be it in a much simpler way). You could still first keep the context on the stack for quick access by the parent function and later add indirection if necessary (in case it can escape the life of the parent function), as handled by e.g. C blocks. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
I just want to remind you people (whoever is working on it) that ARC is already implemented in FPC for interface types and I am personally satisfited with the way it works now for interface variables, and only wish that I could enable this feature for class instance variables. Can't someone just, you know, Ctrl+C Ctrl+V it?21.09.2014, 21:18, "Boian Mitov" mi...@mitov.com: In general I would prefer to see this implemented with attributes rather than with new keywords. For god shake, we already have more keywords in this language than it is healthy. We should be extremely cautions when adding new ones. That is one of the beauty of the attributes. They can eliminate a large number of keywords, and expand the language dynamically (trough the library not trough the compiler). If you implement it with attributes, there is almost no need for any big changes in the compiler at least in the parsing side. Adding new keyword will probably propagate all the way to the tokenizer. With best regards, Boian Mitov --- Mitov Software www.mitov.com --- -Original Message- From: Sven Barth Sent: Sunday, September 21, 2014 4:12 AM To: fpc-devel@lists.freepascal.org Subject: Re: [fpc-devel] Suggestion: reference counted objects === code begin === typeTARCObject = class refcounted(TObject)end; === code end === === code begin === typeTSomeClass = class(TARCObject) // ... Owner: TSomeClass weak; // ...end; === code end === ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
1. I mostly care for performance, so I will start here: They add overheads which in certain situations can be very bad. These can come in many shapes, including additional memory for the reference counter, creating cache incoherence when doing local object assignments (which would cause some function calls on various patches of memory), the actual inc/dec which would probably be in the form of virtual functions (which may cause pointer chasing), etc. 2. There is no need to force such behavior for all objects: if you want reference counting, work with interfaces, or make a special derived object or both. 3. They break the semantics of existing code. 4. You will mess up code which is tuned to work in certain ways with the heap. For example, I routinely allocate huge chunks of instances the same class to avoid the performance penalty of mallocs, to avoid heap fragmentation and to decrease cache misses in loops. These objects are managed by a custom heap manager via NewInstance/FreeInstance/Destroy. Why on earth should the compiler tell me how to manage such situations, when clearly I have something very specific in mind which does not fit the general scheme of things? 5. Reference counting does not eliminate memory leaks, the trivial situation being a loop assignements a=b, b=c, c=a. 6. You may want control on the deallocation order of classes. 7. Last but not least, you make the language inconsistent. It allows direct pointer mallocs which could be turned into classes, arrays, etc. Forcing different behavior for classes will create an inconsistent structure of the language. If you want reference counting in a consistent way you have to ban getmem/freemem. Either all memory operations are type-managed (no thanks), or you give an option for reference counting which is contained in the grammar of the language (e.g. inheriting a class). Forcing a bad mix is, as I said earlier, uniquely bad idea. In summary, if it is already there, why force it on all, change the language and create a mess? Engouh? On Sat, 20 Sep 2014 14:42:53 +0300, Sven Barth pascaldra...@googlemail.com wrote: On 20.09.2014 13:11, Peter Popov wrote: Please do not reference count TObject. This is a uniquely bad and unnecessary idea. I will switch to ANSI C if you guys do it Please enlighten me why you think it is bad. Give reasons and don't be like a farmer doesn't eat what he doesn't know (rough translation of a German proverb). And just to give you a reason *for* reference counted objects: they would finally allow to use operator overloads on classes, because then temporarily created objects are created correctly (think about an expression like a * (b + c) with classes). Please also note that I said that I don't consider them to be the holy grail especially considering that I know of some of the technial problems one needs to face when implementing them in an already existing language. Regardts, Sven ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
I just want to remind you people (whoever is working on it) that ARC is already implemented in FPC for interface types and I am personally satisfited with the way /it works now/ for interface variables, and only wish that I could enable this feature for class instance variables. Can't someone just, you know, Ctrl+C Ctrl+V it? I assume you are talking about ARC for variables/fields of some class that implements interface? Yes me, too! Coming from Java I could not even comprehend at the beginning that if variable type is interface - it is reference counted, but if variable type is a class that implements interface - it is not ?! Regards, Gena On 9/21/2014 2:02 PM, hinsta...@yandex.ru wrote: I just want to remind you people (whoever is working on it) that ARC is already implemented in FPC for interface types and I am personally satisfited with the way /it works now/ for interface variables, and only wish that I could enable this feature for class instance variables. Can't someone just, you know, Ctrl+C Ctrl+V it? 21.09.2014, 21:18, Boian Mitov mi...@mitov.com mailto:mi...@mitov.com: In general I would prefer to see this implemented with attributes rather than with new keywords. For god shake, we already have more keywords in this language than it is healthy. We should be extremely cautions when adding new ones. That is one of the beauty of the attributes. They can eliminate a large number of keywords, and expand the language dynamically (trough the library not trough the compiler). If you implement it with attributes, there is almost no need for any big changes in the compiler at least in the parsing side. Adding new keyword will probably propagate all the way to the tokenizer. With best regards, Boian Mitov --- Mitov Software www.mitov.com http://www.mitov.com/ --- -Original Message- From: Sven Barth Sent: Sunday, September 21, 2014 4:12 AM To: fpc-devel@lists.freepascal.org mailto:fpc-devel@lists.freepascal.org Subject: Re: [fpc-devel] Suggestion: reference counted objects === code begin === type TARCObject = class refcounted(TObject) end; === code end === === code begin === type TSomeClass = class(TARCObject) // ... Owner: TSomeClass weak; // ... end; === code end === ___ fpc-devel maillist - fpc-devel@lists.freepascal.org mailto:fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
So why are you not programming in Assembler? The arguments you provide are the same that ware provided against higher level languages, adding OOP, adding anonymous methods, just to name few. I spent 2 years streamlining all of our code - 4 mln. lines of code to use the latest language features, and in the following year we had a 10x boost in development performance, so it immediately paid off, and now we have 800K code instead of 4 mln. With the old code we ware running out of steam just to maintain it. The new code just largely takes care of itself. Rewriting code is actually a good thing, and I do rewrite cycles in the company every 3-4 years, sometimes for some pars even more often. Every time the code improves as does the performance, and quality. If ARC forces you to review some code, that actually is a good thing. You will need to retest it after the changes, but if you have good unit or functional testing should not be big issue, and if you don’t will be a good chance to add it, or to at least make sure the code really behaves. With best regards, Boian Mitov --- Mitov Software www.mitov.com --- -Original Message- From: Peter Popov Sent: Sunday, September 21, 2014 11:13 AM To: fpc-devel@lists.freepascal.org ; Sven Barth Subject: Re: [fpc-devel] Suggestion: reference counted objects In summary, if it is already there, why force it on all, change the language and create a mess? Engouh? ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
Ahh... the schizophrenic nature of Object Pascal :-D, I never understood why interface and class should be different at first place, considering that interface is practically nothing more than pure abstract class with ref. counting. Why the separated syntax etc. is beyond me... Surely helps with the language keyword pollution With best regards, Boian Mitov --- Mitov Software www.mitov.com --- From: Gennady Agranov Sent: Sunday, September 21, 2014 11:15 AM To: fpc-devel@lists.freepascal.org Subject: Re: [fpc-devel] Suggestion: reference counted objects Coming from Java I could not even comprehend at the beginning that if variable type is interface - it is reference counted, but if variable type is a class that implements interface - it is not ?! Regards, Gena ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
I strongly disagree... I think there is a great value in a reference counted object, especially if we can enable/disable that with virtual method or even better attribute ;-) . Then you will have it any way you want :-) . Hi Boian I do not mind if this is done on a derived object. As Marco said earlier, one can do anything form his base root object. Just don't do it to the root class Personally, I operate from a base class descending from TPersistent. Down the hierarchy I do have reference counted base class (done explicitly). If I want it implicitly, I would pass interface around. I do explicitly reference count some objects. However, in other cases (I posted earlier) this is a huge problem. Someone also mentioned the issue of race conditions in multithreaded environments. I can see this happening to me. I would not mind declaring a class implicitly reference counted (an all descendants, presubably). However, not TObject. Peter With best regards, Boian Mitov --- Mitov Software www.mitov.com --- -Original Message- From: Peter Popov Sent: Saturday, September 20, 2014 4:11 AM To: fpc-devel@lists.freepascal.org ; Sven Barth Subject: Re: [fpc-devel] Suggestion: reference counted objects Please do not reference count TObject. This is a uniquely bad and unnecessary idea. I will switch to ANSI C if you guys do it Peter ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
Interfaces, among other things, are a way to bring in some features from multiple inheritance. I think the term is one-sided multiple inheritance. If multiple inheritance is good or bad is a separate topic. With time, one begins to doubt the utility of classes in the first place and starts to appreciate ANSI C :-) On Sun, 21 Sep 2014 21:28:24 +0300, Boian Mitov mi...@mitov.com wrote: Ahh... the schizophrenic nature of Object Pascal :-D, I never understood why interface and class should be different at first place, considering that interface is practically nothing more than pure abstract class with ref. counting. Why the separated syntax etc. is beyond me... Surely helps with the language keyword pollution With best regards, Boian Mitov --- Mitov Software www.mitov.com --- From: Gennady Agranov Sent: Sunday, September 21, 2014 11:15 AM To: fpc-devel@lists.freepascal.org Subject: Re: [fpc-devel] Suggestion: reference counted objects Coming from Java I could not even comprehend at the beginning that if variable type is interface - it is reference counted, but if variable type is a class that implements interface - it is not ?! Regards, Gena ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
Sven, based on your comments a bizarre thought came to me. Currently, class instances are pointers. What you propose will somehow create a special pointer type (referenced class). Should this be done, I have a request: create a 4 byte compressed class reference which expands into an 8 byte pointer type prior to usage: 1. Most architectures align data. 2. 64 bit comps align pointers on 8 byte boundaries, meaning that the 3 leftmost bits are 0. 3. You could store a class pointer as a 4 byte integer and, prior to usage, expand it to 8 bytes a leftshift 3. In this way you will have 64GB of data available to your program and your pointers will occupy twice less. If, for example one uses large amount of pointers, this could be ... aesthetically pleasing ... I've thought about the topic a bit more (I've already thought about it quite much in the past months) and came up with the following RFC: - No reference counting for classes by default - Reference counting can be introduced to a part of the class hierarchy by declaring it as refcounted like this (Note: syntax is not final, but is based on other syntaxes we have (namely sealed and abstract)): === code begin === type TARCObject = class refcounted(TObject) end; === code end === - a reference counted class (and its child classes) would include a reference count field that the compiler knows how to access (for automatic reference counting) and which can be accessed through RTTI (for manual reference counting); it is *not* exposed as a regular field as this might lead to identifier conflicts - only *variables* (or parameters) that have a reference counted class as its type will be subject to ARC; in extension this means that assignments from/to a variable of a not reference counted base class (e.g. TObject) will *not* change the reference count. Take this code for example: === code begin === function CreateObject: TObject; begin Result := TARCObject.Create; end; === code end === This will most likely result in CreateObject returning an instance to class that is already freed (because the only reference inside CreateObject is a (hypothetical) temporary of type TARCObject which goes out of scope once CreateObject returns) - to remedy this TObject is extended with non-virtual methods that allow manual reference counting and would rely on the RTTI data I mentioned (let's call the methods AddRef, Release, IsReferenceCounted and RefCount for now, which can also be used to hook up the reference counting of IUnknown interfaces); the code from above would then look like this to make it safe: === code begin === function CreateObject: TObject; begin Result := TARCObject.Create; Result.AddRef; end; === code end === - TObject.Free would be extended to take reference counting into account as well. If the object is reference counted (IsReferenceCounted returns true) it will call Release and otherwise it will continue to Destroy. - there would be a TARCObject declared in System which is a direct descendant of TObject, but with reference counting enabled; same maybe also for TInterfacedObject - all classes can now have operator overloads as well though it should be warned in the documentation that non-reference counted objects might result in memory leaks there - this now only leaves the problems of cycles; take this code: === code begin === type TSomeClass = class(TARCObject) Children: specialize TListTSomeClass; Owner: TSomeClass; constructor Create(aOwner: TSomeClass); end; constructor TSomeClass.Create(aOwner: TSomeClass); begin Children := specialize TListTSomeClass.Create; Owner := aOwner; if Assigned(Owner) then Owner.Children.Add(Self); end; procedure Test; var t1, t2: TSomeClass; begin t1 := TSomeClass.Create(Nil); t2 := TSomeClass.Create(t1); // do something end; === code end === Now once Test is left it would leave the instances which were assigned to t1 and t2 hanging, because they have references to each other. There are (as far as I see) three ways to solve this: * provide a way to break the circle (in this example e.g. setting Owner to Nil before leaving Test; this is what Delphi provides with the DisposeOf virtual method) * introduce weak references which would disable reference counting, e.g.: === code begin === type TSomeClass = class(TARCObject) // ... Owner: TSomeClass weak; // ... end; === code end === Now the TSomeClass.Create(t1) line in Test wouldn't increase the reference count of t1 further and thus both class instances would be destroyed after Test is left. * provide a possibilty to execute a cycle detection algorithm during the Release part of the reference counting; this has the benefit of avoiding the need for weak, but there would be the problem that the algorithm can be potentially expensive especially with large object instance hierarchies (think LCL here) and
Re: [fpc-devel] Suggestion: reference counted objects
In our previous episode, Boian Mitov said: Every memory management system has a manual part, even the M$/Java GC has element of it (the dreadful Disposable pattern to name one). There is a huge difference however. ARC requires in general less coding for the same result. Less code in general is cheaper to develop, debug, read and maintain. It is a simple business formula actually (and as businessman I need to calculate this). I work with Delphi professionally too, and I don't really notice allocation as time costs. It's negiable. Occasionally (in the six months magnitude) I run the app through fastmm, but that time is general cleanup, so even not entirely the cost of memory management. I can't find any basis for your allegations in my work. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
This is not true. Ref pointer is exactly the same size as non ref pointer. the counter is in the instance not in the pointer to the instance ;-) . You can study the interface implementations or the String implementations they are done the same way. With best regards, Boian Mitov --- Mitov Software www.mitov.com --- -Original Message- From: Peter Popov Sent: Sunday, September 21, 2014 12:29 PM To: fpc-devel@lists.freepascal.org ; Sven Barth Subject: Re: [fpc-devel] Suggestion: reference counted objects Sven, based on your comments a bizarre thought came to me. Currently, class instances are pointers. What you propose will somehow create a special pointer type (referenced class). Should this be done, I have a request: create a 4 byte compressed class reference which expands into an 8 byte pointer type prior to usage: 1. Most architectures align data. 2. 64 bit comps align pointers on 8 byte boundaries, meaning that the 3 leftmost bits are 0. 3. You could store a class pointer as a 4 byte integer and, prior to usage, expand it to 8 bytes a leftshift 3. In this way you will have 64GB of data available to your program and your pointers will occupy twice less. If, for example one uses large amount of pointers, this could be ... aesthetically pleasing ... ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
Marco van de Voort schrieb: In our previous episode, Hans-Peter Diettrich said: IMO Weak references should be reserved for users who accept possible consequential problems, but should never be used in standard libraries. At least I'd suggest to make weak references subject to an compiler switch, so that every user has a chance to disable them in case of trouble. IMHO weak references trade one manual memory system in for a different manual memory system. Weak references steer/guide *automatic* memory management. The hard part of manual memory systems, figuring out how a complex dynamic structure deallocates (that is usually tackled by having a bit of design and thought go into it), remains. And this is what the user of such a structure (standard libraries...) does not always know. He may be unable to determine the reason for some runtime error in his code, when an object was destroyed automagically where it should still be alive. The user can debug (and fix) ordinary (owner/owned) patterns, implemented in high level code, but not table-driven (RTTI) or otherwise hidden (intrinsic) management procedures. While the user can change the owner of an object at runtime, he cannot change a weak reference into a strong one, without recompilation of at least the unit containing that declaration, and without figuring out the consequences of such an change. DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
Sven Barth schrieb: On 20.09.2014 13:42, Sven Barth wrote: On 20.09.2014 13:11, Peter Popov wrote: - to remedy this TObject is extended with non-virtual methods that allow manual reference counting and would rely on the RTTI data I mentioned (let's call the methods AddRef, Release, IsReferenceCounted and RefCount for now, which can also be used to hook up the reference counting of IUnknown interfaces); I'd add a _RefCount field to TObject, regardless of whether it's really used later; this will fix the field offset - just like the VMT reference is fixed in TObject, but not in Object types. This will eliminate problems with class helpers. This approach also would allow to switch any object from managed to unmanaged on the fly, by setting the counter to -1, because the special value -1 already indicates an unmanaged/const memory object (like with string literals). In my first draft I considered virtual _AddRef/_Release methods, but calling a virtual method is more expensive than calling or inlining a static method. the code from above would then look like this to make it safe: === code begin === function CreateObject: TObject; begin Result := TARCObject.Create; Result.AddRef; end; === code end === Here the compiler would always insert _AddRef, just like with interfaces, eventually optimized (inlined?) like: if Result._RefCounter -1 then Result._AddRef; //or InterlockedIncrement(Result._RefCounter); - TObject.Free would be extended to take reference counting into account as well. If the object is reference counted (IsReferenceCounted returns true) it will call Release and otherwise it will continue to Destroy. - there would be a TARCObject declared in System which is a direct descendant of TObject, but with reference counting enabled; same maybe also for TInterfacedObject The convention, of -1 meaning unmanaged, favors managed objects by default, when InitInstance zeroes all fields of the instance just created. But when the VMT reference must be excluded or inserted afterwards afterwards, then _RefCount can be initialized at the same time (to -1 for the unmanaged default). Later on a TARCObject base class constructor/initializer will reset _RefCount to zero again. - all classes can now have operator overloads as well though it should be warned in the documentation that non-reference counted objects might result in memory leaks there ...unless operators also test _RefCount - this now only leaves the problems of cycles; take this code: === code begin === type TSomeClass = class(TARCObject) Children: specialize TListTSomeClass; Owner: TSomeClass; constructor Create(aOwner: TSomeClass); end; constructor TSomeClass.Create(aOwner: TSomeClass); begin Children := specialize TListTSomeClass.Create; Owner := aOwner; if Assigned(Owner) then Owner.Children.Add(Self); end; Here I'd prefer Owner.AddChild(Self); so that the Owner can implement any decent/appropriate child management under the hood. procedure Test; var t1, t2: TSomeClass; begin t1 := TSomeClass.Create(Nil); t2 := TSomeClass.Create(t1); // do something end; === code end === Now once Test is left it would leave the instances which were assigned to t1 and t2 hanging, because they have references to each other. This depends on the implementation of TOwner.Children[] and TChild.Owner. Is a stored TChild.Owner reference really required in a managaged environment? IMO a (strong) unidirectional reference from Owner to Child will do it all. Then no child will be destroyed, as long as its owner holds a reference to it. That's the intended purpose of both owner/child and automatic memory management. When it's desireable to definitely destroy an owned object at will, then its owner must be known, of course. In this case two different management approaches conflict with each other. In this case I'd accept a weak Owner reference, because the referenced Owner will stay alive longer than it's listed children. More problematic are circular references without a decicated owner/child relationship. There are (as far as I see) three ways to solve this: * provide a way to break the circle (in this example e.g. setting Owner to Nil before leaving Test; this is what Delphi provides with the DisposeOf virtual method) * introduce weak references which would disable reference counting, e.g.: === code begin === type TSomeClass = class(TARCObject) // ... Owner: TSomeClass weak; // ... end; === code end === Now the TSomeClass.Create(t1) line in Test wouldn't increase the reference count of t1 further and thus both class instances would be destroyed after Test is left. This IMO is the preferable way to go, in a definite owner/child relationship. The lifetime of an owner can not depend on the existence of owned children, so that the owner will survive until it has destroyed/released all his children himself. A child-to-owner
Re: [fpc-devel] Suggestion: reference counted objects
Florian Klämpfl schrieb: Am 21.09.2014 um 07:22 schrieb Hans-Peter Diettrich: Boian Mitov schrieb: That is easy. it gets incremented when it gets assigned. The running threads have no way of accessing it if there is no reference (assignment) already in place. The problem arises when an object is destroyed, or even elected for destruction in _Release, while another thread starts using the same instance. This is not possible for a correctly written program: if two threads having references to the same instance, the ref. count is 2. So no destruction is possible. What happens if such a reference (to A) is part of another object (B), known to the thread? I suspect a chance for a race condition here, when one thread clears B.A while another thread tries to acquire a reference from B.A. But this may be an excursion into threadsafe coding, where any modification to a shared resource requires a lock in a multi-threaded environment. Then above situation should never occur... Is it really sufficient to protect refcounter changes by Interlocked Inc/Dec, to prevent race conditions while obtaining object references? DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
In general you can have good and bad designs with and without ARC/weak pointers/ or sweep GC. From all of the above, and I have done them all for years, the ARC is the easiest to manage, and the one with the least side effects. What you was does not differ from the issues of lost pointers in non ARC. In essence you normally work with only Weak pointers, so if you let say like them so much now, I don’t see a reason to hate them in ARC ;-) . With best regards, Boian Mitov --- Mitov Software www.mitov.com --- -Original Message- From: Hans-Peter Diettrich Sent: Sunday, September 21, 2014 10:14 AM To: FPC developers' list Subject: Re: [fpc-devel] Suggestion: reference counted objects And this is what the user of such a structure (standard libraries...) does not always know. He may be unable to determine the reason for some runtime error in his code, when an object was destroyed automagically where it should still be alive. The user can debug (and fix) ordinary (owner/owned) patterns, implemented in high level code, but not table-driven (RTTI) or otherwise hidden (intrinsic) management procedures. While the user can change the owner of an object at runtime, he cannot change a weak reference into a strong one, without recompilation of at least the unit containing that declaration, and without figuring out the consequences of such an change. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
Yes! With best regards, Boian Mitov --- Mitov Software www.mitov.com --- Is it really sufficient to protect refcounter changes by Interlocked Inc/Dec, to prevent race conditions while obtaining object references? ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
That is not what I suggested. The idea is to compact an 8 byte pointer to a 4 byte one. Basically, save memory. This would limit the accessible memory tough. Probably for a different thread. HOWEVER: here is another one idea. Current memory buses are 48 bit wide, so basically a pointer can be stored in 6 bytes. It is stored in 8, because of memory alignment issues. So, for classes which are reference counted, store the reference count @ the highest two bytes of the class instance (which in practice is a pointer to the VMT). This would let you ref-count up to 2^16. You need to mask it out from the rest of the pointer when accessing VMT. No need for additional changes to the RTTI, just mask the highest two bytes when using the class. Peter On Sun, 21 Sep 2014 23:03:43 +0300, Boian Mitov mi...@mitov.com wrote: This is not true. Ref pointer is exactly the same size as non ref pointer. the counter is in the instance not in the pointer to the instance ;-) . You can study the interface implementations or the String implementations they are done the same way. With best regards, Boian Mitov --- Mitov Software www.mitov.com --- -Original Message- From: Peter Popov Sent: Sunday, September 21, 2014 12:29 PM To: fpc-devel@lists.freepascal.org ; Sven Barth Subject: Re: [fpc-devel] Suggestion: reference counted objects Sven, based on your comments a bizarre thought came to me. Currently, class instances are pointers. What you propose will somehow create a special pointer type (referenced class). Should this be done, I have a request: create a 4 byte compressed class reference which expands into an 8 byte pointer type prior to usage: 1. Most architectures align data. 2. 64 bit comps align pointers on 8 byte boundaries, meaning that the 3 leftmost bits are 0. 3. You could store a class pointer as a 4 byte integer and, prior to usage, expand it to 8 bytes a leftshift 3. In this way you will have 64GB of data available to your program and your pointers will occupy twice less. If, for example one uses large amount of pointers, this could be ... aesthetically pleasing ... ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
Am 20.09.2014 22:54, schrieb Gennady Agranov: If you want to do ARC for objects - just remove this limitation and allow compiler to generate reference counting code not just for for interfaces but for classes that implement interface - AFAIK any interface in FPC/Delphi extends IUnknown and has reference counting methods. But not in CORBA mode. In this case no reference counting is done. I hope this behaviour will not change. Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
22.09.2014 0:28, Peter Popov: [...] So, for classes which are reference counted, store the reference count @ the highest two bytes of the class instance (which in practice is a pointer to the VMT). This would let you ref-count up to 2^16. You need to mask it out from the rest of the pointer when accessing VMT. No need for additional changes to the RTTI, just mask the highest two bytes when using the class. This would look aesthetically horid at least. Somehow reminds of segment:offset ugliness of 8086 ;-) Besides, thinking about a magnificent brand new pointer type, and as long as certain performance penalties can be tolerated, one might want to also consider providing some path for heap defragmentation, for the sake of completeness (Not that I know how to properly do it, but still my $0.02). Regards, Nikolai Peter On Sun, 21 Sep 2014 23:03:43 +0300, Boian Mitov mi...@mitov.com wrote: This is not true. Ref pointer is exactly the same size as non ref pointer. the counter is in the instance not in the pointer to the instance ;-) . You can study the interface implementations or the String implementations they are done the same way. With best regards, Boian Mitov --- Mitov Software www.mitov.com --- -Original Message- From: Peter Popov Sent: Sunday, September 21, 2014 12:29 PM To: fpc-devel@lists.freepascal.org ; Sven Barth Subject: Re: [fpc-devel] Suggestion: reference counted objects Sven, based on your comments a bizarre thought came to me. Currently, class instances are pointers. What you propose will somehow create a special pointer type (referenced class). Should this be done, I have a request: create a 4 byte compressed class reference which expands into an 8 byte pointer type prior to usage: 1. Most architectures align data. 2. 64 bit comps align pointers on 8 byte boundaries, meaning that the 3 leftmost bits are 0. 3. You could store a class pointer as a 4 byte integer and, prior to usage, expand it to 8 bytes a leftshift 3. In this way you will have 64GB of data available to your program and your pointers will occupy twice less. If, for example one uses large amount of pointers, this could be ... aesthetically pleasing ... ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
We probably should if possible keep pointers just pointers (Ref counted pointer is same a non ref counted as we discussed so same size). Otherwise we may end up adding bits for the Moon Phases or something in it. Even if we add ARC we should do it as simple as possible and without increasing the size of the pointers (After all Delphi already has done it, and their ref counted pointers are still 4 bytes for 32 bit processes). With best regards, Boian Mitov --- Mitov Software www.mitov.com --- -Original Message- From: Nikolai Zhubr Sent: Sunday, September 21, 2014 2:59 PM To: fpc-devel@lists.freepascal.org Subject: Re: [fpc-devel] Suggestion: reference counted objects 22.09.2014 0:28, Peter Popov: Besides, thinking about a magnificent brand new pointer type, and as long as certain performance penalties can be tolerated, one might want to also consider providing some path for heap defragmentation, for the sake of completeness (Not that I know how to properly do it, but still my $0.02). ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
On 21.09.2014 19:17, Boian Mitov wrote: In general I would prefer to see this implemented with attributes rather than with new keywords. For god shake, we already have more keywords in this language than it is healthy. We should be extremely cautions when adding new ones. That is one of the beauty of the attributes. They can eliminate a large number of keywords, and expand the language dynamically (trough the library not trough the compiler). If you implement it with attributes, there is almost no need for any big changes in the compiler at least in the parsing side. Adding new keyword will probably propagate all the way to the tokenizer. I openly *hate* the attribute syntax. It's just not Pascal and simply copied from their .Net compiler, because hey, other .Net compilers do that just like that... -.- Also considering that the trunk compiler currently does not have support for attributes (let's put aside that branch for now) it is definitely easier to add a new keyword, because the compiler is built in a way to allow the easy addition of new keywords. Additionally using attributes would mean that one would need to add detection for special attributes. And that's more annoying stuff then a keyword is. Additionally this can be added more safely to other modes (e.g. macpas, which supports classes as well) than the attributes can. Regards, Sven ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Suggestion: reference counted objects
On 21.09.2014 20:28, Boian Mitov wrote: Ahh... the schizophrenic nature of Object Pascal :-D, I never understood why interface and class should be different at first place, considering that interface is practically nothing more than pure abstract class with ref. counting. Why the separated syntax etc. is beyond me... Surely helps with the language keyword pollution It's different, because interfaces were originally introduced to support interfacing with COM and the automatic reference counted was just added as a big pile of sugar to simplyfy development in context with COM. Because in C++ you have to call AddRef and Release yourself... Regards, Sven ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
