Re: [fpc-devel] dynamic package support
- Original Message - From: <[EMAIL PROTECTED]> To: "FPC developers' list" Sent: Saturday, February 26, 2005 12:05 PM Subject: Re: [fpc-devel] dynamic package support On Sat, 26 Feb 2005, Florian Klaempfl wrote: > Hans-JÃrg Vasold wrote: > > > for a long time support for loading packages at runtime was "planned > > for > > later versions". > > > > But now i can not find that this is planned anymore. Will FreePascal > > support dynamic packages in future or not. > > http://www.freepascal.org/wiki/index.php/Detailed_2.1.0_branch_todo > > However, it had quite low priority the last years because dyn. loaded > packages > for a dynamic moving project like fpc with daily snapshots are a real > problem. > As soon as one unit interface changes, all dependend packages must be > recompiled anyways. And usually there are only 1-2 fpc projects running > at the > same time on a system so main memory savings are little. Unless you have a _good_ build tool. Porting want ( http://sourceforge.net/projects/want/ ) could help. On the other hand, for people who use FPC to develop end-user apps, it makes sense: they will normally not change their installed FPC every 2 months. That's why i ask. We have a large inhouse application (ERP, paper document processing, document managemen server), written mostly in Delphi. The ERP Part is build on the top of tiopf ( http://www.techinsite.com.au/ ). We have started porting the tiopf framework and our application but this makes no sense unless dynamic package loading is supported, so we have stopped our activities. cu Hans-Joerg ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Modernising Pascal
DrDiettrich wrote: Jamie McCracken wrote: GC is very inefficient with memory and current implementations tend to cost a lot performance wise too. I don't see how GC is inefficient with memory? Reference counting and (mark/sweep) garbage collection have a different runtime behaviour: Reference counting occurs with many references to objects, whereas GC only runs when required, or when idle time is left. The problem with GC is the lifetime of (dead) objects is significantly extended under GC. This means it hogs more memory for much longer. Memory allocation is also potentially slowed as a result of more objects remaining allocated on the heap so less chance of holes appearing which can be reused (unless you use a compacting GC). GC runs in a seperate thread so will incur additional overhead from that too and you cant always predict when it runs and it can snare you at the most inappropriate times (EG if memory is in short supply the thread must become extremeley aggressive to avoid swap so you will lose performance big time in those cases and with compacting variety even more so if it does end up in swap) GC gets a lot slower with the more objects you create. It's not the creation that costs GC time, instead it's the destruction of the objects. Its potentially both. All GCs have a tradeoff between memory usage and performance. A compacting one may have faster allocation but uses more memory as a result. There is no perfect GC that both uses memory efficiently and gives good performance (when compared to manual). Mixing unmanaged with managed code cause severe performance issues (marshalling penalties calling ummanaged c functions) Interoperability with C gets increasingly difficult. Writing bindings becomes much more complicated and time consuming. IMO it's never a good idea to mix managed and unmanaged code, to suffer from the worst from two worlds at the same time :-( GC is worse here because the pointers are not static in GC so referencing an object means looking up the actual address in a lookup list (so more overhead whenever you call a method or pass an object reference). GC does not use the stack efficiently and has to use a lot of heap so its extremely inefficient. Ever see GC compact swap?<< I don't know what GC implementation you're referring to. When memory gets too much fragmented, every memory manager has to do a compaction, that's inevitable. But GC offers a chance to update the references to moved objects, so that no address tables are required. Other memory managers don't know about the locations of pointers to the objects, so that moving objects is not possible at all! Consequently GC allows for better use of available memory, where other methods have to stop with "out of memory". IMO you should learn a bit more about memory management, your argumentation doesn't look well founded to me. There are over a dozen different GC strategies so some of my points apply to some but not others (same with a lot of the replies here) thats why this thread has become long and controversial - you simply cant make a lot of generalities without specifying which type of GC you are reffering to. jamie. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] Improving Ref Counting
Hi, Rather than continuing the GC stuff which seems fruitless I thought it might be better to improve what we have with ref counting (whilst taking a leaf out of the GC book as well). I note that ref counting is very fast without all the implicit try..finally stuff so I was wondering if it was possible to implement it without them (whilst still being able to free them even if an exception occurs). My proposal is to do the following (its a hybrid ref count/GC): 1) Remove all implicit try..finally in ref counting. 2) Have a single linked list that contains references to all managed objects (ansi strings, variants etc) 3) Each managed object also has a single linked list that contains addresses of all pointers that reference it. 4) inc ref count means adding the pointer's address to that list 5) dec ref count means removing the pointer's address from that list. When that list is empty the object is freed. (a check for eliminating circular refs should also be done if applicable) 6) Whenever an exception is thrown, wait until its either handled or fully propagated and then perform some garbage collection. (traverse the single linked list of all managed objects and for each object check whether anything that references it is still valid and delete if appropriate). Im not sure if there are any corner cases where the above would not work but it does have the advantage of being very fast and very efficient memory wise. It also avoids the problem of extended lifetimes that you get with GCs. The only performance hit you will get is when an exception occurs but considering they only occur in exceptional circumstances and that exceptions are slow anyhow I dont think thats any loss. If no exceptions are thrown then no GC needs to be performed ever. Can we get away with doing this? jamie. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Improving Ref Counting
> 6) Whenever an exception is thrown, wait until its either handled or > fully propagated and then perform some garbage collection. (traverse the > single linked list of all managed objects and for each object check > whether anything that references it is still valid and delete if > appropriate). If so raising an exception will freeze your application, checking some thousand hundreds of lists. And how do you reconnaize a valid reference from an invalid one? Bye Uberto ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Improving Ref Counting
> circular refs should also be done if applicable) > 6) Whenever an exception is thrown, wait until its either handled or > fully propagated and then perform some garbage collection. (traverse the > single linked list of all managed objects and for each object check > whether anything that references it is still valid and delete if > appropriate). I also thought immediately what Uberto already said: how do you recognize a valid/invalid reference without accessing memory that is invalid in the mean time. Also note that the list of references must be a dynamically size structure, incurring getmem and size-change overhead on a simple assignment. I doubt performance would improve, specially since the edge of this problem (the worst 90%) can be taken off by simply disabling try...finally generation in places where they are unlikely to happen. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Improving Ref Counting
Uberto Barbini wrote: 6) Whenever an exception is thrown, wait until its either handled or fully propagated and then perform some garbage collection. (traverse the single linked list of all managed objects and for each object check whether anything that references it is still valid and delete if appropriate). If so raising an exception will freeze your application, checking some thousand hundreds of lists. There would be a small delay yes but Pascal is very fast so even checking many thousands of lists should only take a few milliseconds on a reasonable machine. Dont forget all GCs do this whenever garbage allocation is performed. And how do you reconnaize a valid reference from an invalid one? The same way a GC does? jamie. Bye Uberto ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Improving Ref Counting
Marco van de Voort wrote: circular refs should also be done if applicable) 6) Whenever an exception is thrown, wait until its either handled or fully propagated and then perform some garbage collection. (traverse the single linked list of all managed objects and for each object check whether anything that references it is still valid and delete if appropriate). I also thought immediately what Uberto already said: how do you recognize a valid/invalid reference without accessing memory that is invalid in the mean time. How does a GC do this? It would have the same problem? Also note that the list of references must be a dynamically size structure, incurring getmem and size-change overhead on a simple assignment. yes a "single linked list" is a very efficient dynamically sized structure and much cheaper than try..finally or using Tlist. I doubt performance would improve, specially since the edge of this problem (the worst 90%) can be taken off by simply disabling try...finally generation in places where they are unlikely to happen. Is there a real failsafe way of doing that? jamie. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Improving Ref Counting
> > a valid/invalid reference without accessing memory that is invalid in the > > mean > > time. > > How does a GC do this? It would have the same problem? A GC manages all memory, local variable allocation inclusive. IOW, the way a GC does it, is not possible in a mixed environment. > > Also note that the list of references must be a dynamically size structure, > > incurring getmem and size-change overhead on a simple assignment. > > yes a "single linked list" is a very efficient dynamically sized > structure and much cheaper than try..finally or using Tlist. It's very expensive. getmem is quite expensive, and you need it for every reference this way. > > I doubt performance would improve, specially since the edge of this problem > > (the worst 90%) can be taken off by simply disabling try...finally > > generation > > in places where they are unlikely to happen. > > Is there a real failsafe way of doing that? Nothing is failsafe. However e.g. in RTL string routines exceptions shouldn't occur unless memory is exhausted, in case it doesn't matter much anyway. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Improving Ref Counting
Marco van de Voort wrote: a valid/invalid reference without accessing memory that is invalid in the mean time. How does a GC do this? It would have the same problem? A GC manages all memory, local variable allocation inclusive. IOW, the way a GC does it, is not possible in a mixed environment. Are you saying it would be a managed pointer then which is allocated on the heap? (I had assumed the pointer would be on the stack and the object in the heap) Also note that the list of references must be a dynamically size structure, incurring getmem and size-change overhead on a simple assignment. yes a "single linked list" is a very efficient dynamically sized structure and much cheaper than try..finally or using Tlist. It's very expensive. getmem is quite expensive, and you need it for every reference this way. Okay then use Tlist with preallocation of say half a dozen references - that should be efficient for 99% of cases for an individual object's references. I doubt performance would improve, specially since the edge of this problem (the worst 90%) can be taken off by simply disabling try...finally generation in places where they are unlikely to happen. Is there a real failsafe way of doing that? Nothing is failsafe. However e.g. in RTL string routines exceptions shouldn't occur unless memory is exhausted, in case it doesn't matter much anyway. What about all other non string exceptions that can occur between creation and destruction of the ansistring? Multithreaded environments too? jamie. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Improving Ref Counting
> > I also thought immediately what Uberto already said: how do you recognize > > a valid/invalid reference without accessing memory that is invalid in the > > mean time. > > How does a GC do this? It would have the same problem? A GC dont' try to recognize a valid/invalid reference, it is invoked to free unused memory, which it assume point to valid memory. Anyway I still don't understand the goal of this discussion. Do you want to add some managed object in fpc? OK, but I suggest to start studing how GC works in Java, Python and dotnet. As for me I'd rather ask for not managed Interfaces in Delphi (fpc ones are ok). Bye Uberto ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Improving Ref Counting
Uberto Barbini wrote: I also thought immediately what Uberto already said: how do you recognize a valid/invalid reference without accessing memory that is invalid in the mean time. How does a GC do this? It would have the same problem? A GC dont' try to recognize a valid/invalid reference, it is invoked to free unused memory, which it assume point to valid memory. A GC needs to trace an object's references to see if anything still points to it. How else can it decide whether an object is no longer in use? Anyway I still don't understand the goal of this discussion. Do you want to add some managed object in fpc? OK, but I suggest to start studing how GC works in Java, Python and dotnet. I dont want a full blown GC just a way to speed up ref counting so that it can be used elsewhere. As for me I'd rather ask for not managed Interfaces in Delphi (fpc ones are ok). You already have them in Iunknown, ansistrings and variants. Its all a question of making them faster cause they are dog slow atm. jamie. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Improving Ref Counting
> A GC needs to trace an object's references to see if anything still > points to it. How else can it decide whether an object is no longer in use? Yes, this is right, but it hasn't to decide if reference are valid or invalid. Moreover also the simpliest GC techniques (mark'n'swift) are quite slow and are usually running in a low priority thread in background. > I dont want a full blown GC just a way to speed up ref counting so that > it can be used elsewhere. Good luck, may you success where everyone else failed! ;) > > As for me I'd rather ask for not managed Interfaces in Delphi (fpc ones > > are ok). > > You already have them in Iunknown, ansistrings and variants. Its all a > question of making them faster cause they are dog slow atm. I wish them "NOT" managed, you cannot free a interface in Delphi. Anyway, why you're saying that refcount is slow? Do you have any benchmark? If I recall you mentioned try..finally as the bottleneck, but AFAIK modern cpu should do it almost with zero overhead, differently from try..except. Bye Uberto Bye Uberto ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Improving Ref Counting
05-02-27 12.35, skrev Jamie McCracken följande: > Hi, > > Rather than continuing the GC stuff which seems fruitless I thought it > might be better to improve what we have with ref counting (whilst taking > a leaf out of the GC book as well). A more simplictic alternative could be to have objects (declared to be managed) managed in the same way as ansistrings. This would be easy to implement since the ansistring facilty could be reused. Also a block of memory could be declared to be managed in this way. Drawback for objects is that they could not refere to other objects in a circular manner, so this has to be handled manually. This feature could be especially useful for functions which returns objects or memory chuncks of arbitrary size like bitmaps. Olle ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Improving Ref Counting
> A more simplictic alternative could be to have objects (declared to be > managed) managed in the same way as ansistrings. This is exactly what delphi do with interfaces, the result is an orrible mess, and passing them as parameters a nightmare. Refcounted objects are possible, python used them till version 2.x, but I don't think they should stay in pascal. Bye Uberto ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Improving Ref Counting
05-02-27 13.47, skrev Uberto Barbini följande: >> A GC needs to trace an object's references to see if anything still >> points to it. How else can it decide whether an object is no longer in use? > > Yes, this is right, but it hasn't to decide if reference are valid or invalid. > Moreover also the simpliest GC techniques (mark'n'swift) are quite slow and > are usually running in a low priority thread in background. Afaik the simplest GC's need to have exclusive access to the heap and stack, so it cant be run in parallell with ordinary processing. Olle ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Improving Ref Counting
Marco van de Voort wrote: circular refs should also be done if applicable) 6) Whenever an exception is thrown, wait until its either handled or fully propagated and then perform some garbage collection. (traverse the single linked list of all managed objects and for each object check whether anything that references it is still valid and delete if appropriate). I also thought immediately what Uberto already said: how do you recognize a valid/invalid reference without accessing memory that is invalid in the mean time. Another possibility is for the exception handling to Null the pointers on the stack if its not handled in the current subroutine. jamie. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Improving Ref Counting
> Afaik the simplest GC's need to have exclusive access to the heap and > stack, so it cant be run in parallell with ordinary processing. What you mean with parallell? I imagine that it's protected into a critical section. Anyway I'm not a GC expert but there's tons of stuff on GC theories and all python and Java sources to study! ;) Bye Uberto ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Improving Ref Counting
> Marco van de Voort wrote: > >>>a valid/invalid reference without accessing memory that is invalid in the > >>>mean > >>>time. > >> > >>How does a GC do this? It would have the same problem? > > > > > > A GC manages all memory, local variable allocation inclusive. IOW, the > > way a GC does it, is not possible in a mixed environment. > > Are you saying it would be a managed pointer then which is allocated on > the heap? (I had assumed the pointer would be on the stack and the > object in the heap) That is a solution yes. Or on the stack, but on termination the reference is killed, and you are back at the try..finally again > > It's very expensive. getmem is quite expensive, and you need it for every > > reference this way. > > Okay then use Tlist with preallocation of say half a dozen references - > that should be efficient for 99% of cases for an individual object's > references. Still an dyn allocation extra. Ansistring performance is partially bound to it, and you double it. > > Nothing is failsafe. However e.g. in RTL string routines exceptions > > shouldn't occur unless memory is exhausted, in case it doesn't matter > > much anyway. > > What about all other non string exceptions that can occur between > creation and destruction of the ansistring? They lead to mem leaks. > Multithreaded environments too? Yes. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Improving Ref Counting
[ Charset ISO-8859-1 unsupported, converting... ] > Marco van de Voort wrote: > >>circular refs should also be done if applicable) > >>6) Whenever an exception is thrown, wait until its either handled or > >>fully propagated and then perform some garbage collection. (traverse the > >>single linked list of all managed objects and for each object check > >>whether anything that references it is still valid and delete if > >>appropriate). > > > > > > I also thought immediately what Uberto already said: how do you recognize > > a valid/invalid reference without accessing memory that is invalid in the > > mean > > time. > > > > Another possibility is for the exception handling to Null the pointers > on the stack if its not handled in the current subroutine. And that will have to be guarded in a try..finally, so we are back ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Improving Ref Counting
At 12:35 27-2-2005, you wrote:
Hi,
Rather than continuing the GC stuff which seems fruitless I thought it
might be better to improve what we have with ref counting (whilst taking a
leaf out of the GC book as well).
I note that ref counting is very fast without all the implicit
try..finally stuff so I was wondering if it was possible to implement it
without them (whilst still being able to free them even if an exception
occurs).
Why are you looking at GC/Refcounting when the problem is the try..finally?
It is better to rewrite the try..finally code using the C++ ABI for
exception handling.
My proposal is to do the following (its a hybrid ref count/GC):
1) Remove all implicit try..finally in ref counting.
Already supported: {$implicitexceptions off}
Peter
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Improving Ref Counting
On Sunday 27 February 2005 15:29, Peter Vreman wrote: > Why are you looking at GC/Refcounting when the problem is the try..finally? > It is better to rewrite the try..finally code using the C++ ABI for > exception handling. +1 and it'd be benefical to all applications. Bye Uberto ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] InstantObjects
I almost finished the porting of the core part of IO to fpc. I added bugs for the (few) function I had to rewrite or modify passing from Delphi to fpc. Anyway I'm still in trouble with streams. IO use descendants of TRead and TWrite like files. It goes back when an error rises and it needs reading/writing block of binary data. Someone could point me the best way to add similar behavior over fpc TRead and TWrite? It could be added to the "driver"? Another problem that could arise is that the same data could be saved and restore differently between fpc and delphi. In this case it'd be not possible to share config files. Annoying but not dramatic. Bye Uberto ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] InstantObjects
On Sun, 27 Feb 2005, Uberto Barbini wrote: > I almost finished the porting of the core part of IO to fpc. I am *very* interested in seeing this working :) > > I added bugs for the (few) function I had to rewrite or modify passing from > Delphi to fpc. > > Anyway I'm still in trouble with streams. > IO use descendants of TRead and TWrite like files. What is TRead and TWrite ? > It goes back when an error rises and it needs reading/writing block of binary > data. > Someone could point me the best way to add similar behavior over fpc TRead > and > TWrite? > It could be added to the "driver"? Could you explain this ? > > Another problem that could arise is that the same data could be saved and > restore differently between fpc and delphi. In this case it'd be not possible > to share config files. Annoying but not dramatic. You should write using text files only if you need this. There is no binary compatibility when writing data from memory to file. Maybe if you use packed records and packed arrays, but by default alignment is different. I am also not sure about floating point values... Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] InstantObjects
> > I almost finished the porting of the core part of IO to fpc. > > I am *very* interested in seeing this working :) Nice to know! ;) > > I added bugs for the (few) function I had to rewrite or modify passing > > from Delphi to fpc. > > > > Anyway I'm still in trouble with streams. > > IO use descendants of TRead and TWrite like files. > > What is TRead and TWrite ? mmh, I meant TReader and TWriter, the stream classes descendants of TFiler. > > It could be added to the "driver"? > > Could you explain this ? I could explain better later, with some code. Anyway I noted that fpc use a smarter strategy than delphi: instead of an internal TStream fpc use a "Driver", an internal object delegated to all read/write operations. I'm not sure how could it made compatible with IO needs. > You should write using text files only if you need this. Yes, I agree. Maybe there were some minor issues with text too anyway (like cr/lf or floating point formats). Bye Uberto ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Improving Ref Counting
05-02-27 14.23, skrev Uberto Barbini följande: >> Afaik the simplest GC's need to have exclusive access to the heap and >> stack, so it cant be run in parallell with ordinary processing. > > What you mean with parallell? I imagine that it's protected into a critical > section. Thats a possibility, but then you do not win anything by running it in a thread. It could as well be run when a memory allocation is done, and then as a subroutine. True parallell GC's exists, but they are then not simple. Olle ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Improving Ref Counting
> Thats a possibility, but then you do not win anything by running it in a > thread. It could as well be run when a memory allocation is done, and then > as a subroutine. No, because the background thread get more time slices during idle moments and none at all during intense computations. GC have to release the control as fast as possible and be able to resume the job where it left it. > True parallell GC's exists, but they are then not simple. Do you mean in SMP systems? Bye Uberto ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Improving Ref Counting
> > Thats a possibility, but then you do not win anything by running it in a > > thread. It could as well be run when a memory allocation is done, and then > > as a subroutine. > > No, because the background thread get more time slices during idle moments > and > none at all during intense computations. > GC have to release the control as fast as possible and be able to resume the > job where it left it. It can also compact generations that are not in the current working set. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] Lazarus doesn't compile with the latest cvs
Hi all, I tried to compile lazarus(cvs) with 1.9.9 and got this error: lclproc.pas(832,8) Error: Illegal type conversion: "Text" to "" on this line: if TextRec(Output).Mode=fmClosed then Mode and fmClosed are integers. Can anyone help me with this? (1.9.8 works fine) -- Met vriendelijke groeten, Joost van der Sluis CNOC Informatiesystemen en Netwerken http://www.cnoc.nl ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Improving Ref Counting
Uberto Barbini wrote: On Sunday 27 February 2005 15:29, Peter Vreman wrote: Why are you looking at GC/Refcounting when the problem is the try..finally? It is better to rewrite the try..finally code using the C++ ABI for exception handling. +1 and it'd be benefical to all applications. Using the C++ ABI the overhead is almost zero. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Improving Ref Counting
> >>Why are you looking at GC/Refcounting when the problem is the > >> try..finally? It is better to rewrite the try..finally code using the > >> C++ ABI for exception handling. > > > > +1 > > and it'd be benefical to all applications. > > Using the C++ ABI the overhead is almost zero. Out of curiosity, can you be more specific on why using C++ Application Binary Interface will reduce overhead? Bye Uberto ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Lazarus doesn't compile with the latest cvs
Joost van der Sluis wrote: Hi all, I tried to compile lazarus(cvs) with 1.9.9 and got this error: lclproc.pas(832,8) Error: Illegal type conversion: "Text" to "" on this line: if TextRec(Output).Mode=fmClosed then Mode and fmClosed are integers. Can anyone help me with this? (1.9.8 works fine) Recompile everything, the size of textrec changed so units and compiler must match. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Improving Ref Counting
05-02-27 19.16, skrev Uberto Barbini följande: >> Thats a possibility, but then you do not win anything by running it in a >> thread. It could as well be run when a memory allocation is done, and then >> as a subroutine. > > No, because the background thread get more time slices during idle moments and > none at all during intense computations. > GC have to release the control as fast as possible and be able to resume the > job where it left it. What I ment was that simple GC's is all-or-nothing, so if it is interupted, all work it has done so far is spoiled. But OK, it is better to attempt to do a collect during idle time. >> True parallell GC's exists, but they are then not simple. > > Do you mean in SMP systems? No I mean those which work incrementally. Afaik these also need to lock everytning when they do their increment, but that the increment is small in time. So OK it is not true parallell. Olle ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Improving Ref Counting
05-02-27 19.28, skrev Marco van de Voort följande: >>> Thats a possibility, but then you do not win anything by running it in a >>> thread. It could as well be run when a memory allocation is done, and then >>> as a subroutine. >> >> No, because the background thread get more time slices during idle moments >> and >> none at all during intense computations. >> GC have to release the control as fast as possible and be able to resume the >> job where it left it. > > It can also compact generations that are not in the current working set. If it is not in the working set it is not alive, no ? Then it doesn't need compaction ? Olle ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] dynamic package support
[EMAIL PROTECTED] wrote: > At work, we have a plugin system which cannot work unless packages are used. Agreed. > I also think that Lazarus would benefit from it. Currently Lazarus must be rebuilt for the installation of custom controls, which in Delphi can be loaded dynamically. If this behaviour is related to the lack of dynamic package support, I also vote for an implementation with high priority. I do not ask for a Delphi compatible solution, in detail I don't appreciate that Delphi packages are bound to a specific Delphi version. I understand that changes to FPC or to the RTL may require a rebuild of all affected packages, but perhaps this effect can be reduced, somehow. With regards to Lazarus it would be sufficient when the packages are rebuilt whenever required, but Lazarus itself should be able to dynamically load packages. DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Modernising Pascal
Jamie McCracken wrote: > The problem with GC is the lifetime of (dead) objects is significantly > extended under GC. This means it hogs more memory for much longer. > Memory allocation is also potentially slowed as a result of more objects > remaining allocated on the heap so less chance of holes appearing which > can be reused (unless you use a compacting GC). Not using memory is a waste of resources. When memory becomes a precious resource, e.g. when swapping may occur, GC may be called to recycle the dead objects. > GC runs in a seperate thread so will incur additional overhead from that > too and you cant always predict when it runs and it can snare you at the > most inappropriate times (EG if memory is in short supply the thread > must become extremeley aggressive to avoid swap so you will lose > performance big time in those cases and with compacting variety even > more so if it does end up in swap) IMO it doesn't make much sense to run GC in its own thread. This would result in never ending GC, when the other threads continue to create objects at the same time. If your observations on the runtime behaviour of GC are based on such an implementation, I agree that this is a poor and inappropriate GC implementation. > > It's not the creation that costs GC time, instead it's the destruction > > of the objects. > > Its potentially both. All GCs have a tradeoff between memory usage and > performance. A compacting one may have faster allocation but uses more > memory as a result. There is no perfect GC that both uses memory > efficiently and gives good performance (when compared to manual). Where do you see any difference between memory allocation in GC and in any other memory manager? I can't see any difference. Every memory manager must know where chunks of free memory exist, and which of these chunks to use for a new object. > There are over a dozen different GC strategies so some of my points > apply to some but not others (same with a lot of the replies here) thats > why this thread has become long and controversial - you simply cant make > a lot of generalities without specifying which type of GC you are > reffering to. Some, IMO most, of the strategies can be dropped immediately, after determination of their complexity. O(n^2) is inacceptable when O(n) is achievable by other methods. I have experience only with mark/sweep GC, which has almost none of the problems which you described. DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Modernising Pascal
peter green wrote: > one thing i have heared (i don't use garbage collected languages much so > i've never seen this myself) is that the GC gets some CPU time and starts > causing stuff to be swapped in. This slows other threads down which gives > the GC more CPU and causes more stuff to be swapped in. GC is started either when the system is (almost) idle, or when no more memory is available for some request. In the first case nothing is blocked by GC, and in the second case everything is blocked by out-of-memory, until the GC has collected free memory. > another effect that i have seen is when CPU usage is high a garbage > collected app can suck up all the systems memory. It can get all the memory > it needs for itself because it can force run its garbage collector when it > runs out but no other app on the system can. When GC runs in idle time, no other apps exist that could be blocked. When GC must swap something into memory, then the system is equipped with too little RAM for the current situation; in such a state everything runs slowly, due to permanent swapping. I know that people often discuss how to reduce swapping overhead, and typically they all miss the simple solution that they should extend their RAM to prevent swapping to occur at all. > some objects may use resources other than memory which are more valuable and > need to be freed asap. Destructors (or finalizers) can be called to release no longer required resources, even when GC is used. When it's known or suspected that an object could be destroyed, the GC can be invoked immediately. GC for itself doesn't eliminate all resource woes, of course. In special situations it may make sense to support GC with additional code, just like with any other memory management method (see below). > refcounting doesn't suffer from any of theese issues Reference counting has more critical problems, like zombie objects. GC will always find and remove all dead objects, whereas reference counting never can eliminate bidirectionally linked objects, as exist in a GUI (parent <-> child controls). > also note that use of const parameters can eliminate a huge amount of > refcounting overhead. I'm not sure about the real impact of "const". And even if the use of const parameters results in a significant runtime reduction, this only indicates that much more runtime could be saved by not using reference counting at all! DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Improving Ref Counting
Peter Vreman wrote: > Why are you looking at GC/Refcounting when the problem is the try..finally? > It is better to rewrite the try..finally code using the C++ ABI for > exception handling. Where do you see improvements in the C++ ABI? Or even differences? Windows implements this ABI, and every language should use it in the intended way. Perhaps FPC implements a different method on other systems? DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Improving Ref Counting
Uberto Barbini wrote: > > You already have them in Iunknown, ansistrings and variants. Its all a > > question of making them faster cause they are dog slow atm. > > I wish them "NOT" managed, you cannot free a interface in Delphi. You can finalize it, so that it releases all private resources. That's common practice in a GC environment. But then you are responsible when the interfaced object is referenced from one of the still remaining references, and it fails to act properly due to the missing resources. > Anyway, why you're saying that refcount is slow? Do you have any benchmark? > > If I recall you mentioned try..finally as the bottleneck, but AFAIK modern cpu > should do it almost with zero overhead, differently from try..except. Please separate the overhead in normal operation from the overhead in exception handling. We had a lengthy thread about exception handling in a Delphi group, with interesting results. I don't know how exactly FPC implements Try blocks, but there shouldn't be much room for implementations very different from the Delphi (Windows) implementation. In normal operation a Try block requires to push and pop some pointers, comparable to the argument handling of a subroutine call. The invocation of the Finally code may cost a few extra cycles, because it has to be executable both in normal and abnormal program flow. The Exception code executes only when an exception occurs, so it does not add any time penalty in normal operation. Consequently one should eliminate chances for exceptions, not try/except/finally blocks ;-) DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Improving Ref Counting
Jamie McCracken wrote:
> Rather than continuing the GC stuff which seems fruitless I thought it
> might be better to improve what we have with ref counting (whilst taking
> a leaf out of the GC book as well).
A reasonable attempt.
> 2) Have a single linked list that contains references to all managed
> objects (ansi strings, variants etc)
A pointer to the next object could be part of every managed object.
> 3) Each managed object also has a single linked list that contains
> addresses of all pointers that reference it.
Now you replace an simple reference counter by a list of pointers?
What a waste of memory and runtime :-(
Consider what will happen with every assignment to a (managed) string
variable.
> 5) dec ref count means removing the pointer's address from that list.
> When that list is empty the object is freed. (a check for eliminating
> circular refs should also be done if applicable)
A check for circular references is just what GC never has to do. That's
why GC can execute in linear time. It also cannot be determined when a
check for circular references is required, as soon as more than one
object has to be managed. This means that this check must be run very
often, with low probability that it will find discardable object
clusters.
> 6) Whenever an exception is thrown, wait until its either handled or
> fully propagated and then perform some garbage collection. (traverse the
> single linked list of all managed objects and for each object check
> whether anything that references it is still valid and delete if
> appropriate).
This requires that it's known which objects are in/valid - how that?
Your ideas are not bad at all, so let me refine your model:
{Notes:
"object" in the following text means *managed* object.
A "client" object contains a reference to a "server" object.
}
1) Every object is accompanied by a list of references to other objects.
This list is required for the maintenance of the reference lists. The
list can become part of the RTTI, as a list of offsets to reference
fields in the objects of the same type.
2) All objects include an pointer to the next object. (Your 2).
3) Every object has a list of client objects (it is the list header).
A reference (list record) consists of an pointer to the referenced
object, and of an pointer to the next reference to the same (server)
object. (Your 3)
4) Subroutines include a list of references to managed objects. This
list is used to determine which references become invalid, after normal
or abnormal exit from a subroutine. (Your 6)
This list can be optimized by allocating all such variables in a
contiguous stack area, to eliminate linked-list management overhead.
Subroutine parameters must be managed similarly, but they can't be
reordered. The lists are static, part of the subroutine signature.
Now let's replace (3) by this:
5) Non-local variables are collected in another list of references.
and, oh wonder, we have all what's required to implement a GC!
In your model the reference lists (3) must be updated with every
assignment to a reference variable, and after exit from a subroutine.
Adding a reference (inc ref) doesn't cost much, but removing a reference
(dec ref) from the linked list can become expensive.
In a mark/sweep GC an additional mark bit in every object is required.
There exist enough usable bits, e.g. the low bits in every object
reference. In the first pass (mark) the list of static (non-local)
variables is examined, and all referenced objects are marked (alive).
Then the active subroutine list (stack) is traversed, and all referenced
objects are marked. Finally the list of all objects is traversed and
split into a "life" and an "unknown" list. Every marked object is put
into the "life" list, and all objects that it references are marked
alive as well. In a simple implementation the remaining "unknown" object
list can be traversed until it contains no more alive members. More
sophisticated algorithms may do that recursively, where recursion is
limited by the number of yet unmarked objects. The key point is the
linear traversal of the lists, without searches. In the final pass
(sweep) the mark bits in the "life" list are cleared, and it becomes
(is) the list of all remaining objects (2), and the objects in the
"unknown" list are discarded.
This may sound complicated, but all that happens in linear time. When
swapping will occur during GC, the same will occur in your model, when
the reference lists are updated. Now it should be clear why frequent
updates of the management information should be avoided.
I'm not perfect, any comments on above are appreciated.
DoDi
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Improving Ref Counting
>> Why are you looking at GC/Refcounting when the problem is the >> try..finally? >> It is better to rewrite the try..finally code using the C++ ABI for >> exception handling. > > Where do you see improvements in the C++ ABI? Or even differences? > > Windows implements this ABI, and every language should use it in the > intended way. Perhaps FPC implements a different method on other > systems? FPC uses an platform independent method. The C++ ABI isn't used. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Improving Ref Counting
> You can finalize it, so that it releases all private resources. That's > common practice in a GC environment. But then you are responsible when > the interfaced object is referenced from one of the still remaining > references, and it fails to act properly due to the missing resources. I wish I could use an interface exactly how I could use an object (create..try..finally..free). But that it's another topic. > > Anyway, why you're saying that refcount is slow? Do you have any > > benchmark? > > > > If I recall you mentioned try..finally as the bottleneck, but AFAIK > > modern cpu should do it almost with zero overhead, differently from > > try..except. > > Please separate the overhead in normal operation from the overhead in > exception handling. I agree, but the ratio between try..finally without and with raised exception is something about 1/1, and often an exception break the loop anyway. IMHO from a performance pov only try..finally without exception matters. At least if you're not Chad! (pun to the author of Indy ;))) > We had a lengthy thread about exception handling in > a Delphi group there's one on Borland groups years ago, too. > In normal operation a Try block requires to push and pop some pointers, > comparable to the argument handling of a subroutine call. The invocation > of the Finally code may cost a few extra cycles, because it has to be > executable both in normal and abnormal program flow. The Exception code > executes only when an exception occurs, so it does not add any time > penalty in normal operation. Consequently one should eliminate chances > for exceptions, not try/except/finally blocks ;-) My experiences and that lengthy discussion differs here: in delphi try ... finally ... end; is almost negible, without exception raised at least. I've no data here but with or without the block the time is quite the same. try ... except ...end; instead is really really really slow. with kylix the times were a bit higher but restoring after a exception was faster. I want to crono fpc now! ;)) Bye Uberto ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
