Re: [fpc-devel] compile time memory leak detection
20.01.2010 0:29, JoshyFun: Pointers and safe code is mutually exclusive, like managed code and I use pointers quite a lot. The code then works 24/7 for months under some continuous load. (Still, I'd be even happier if compiler/RTL double-checked my code additionally.) fast execution, garbage collection and memory optimization, ... I'm no expert in GCs, but it seems too general. In example this will work in some machines and crash in others: GetMem(p,1024*1024*1024); p^:=#0; You can not detect it a compile time in any way. Sure, however this code is actually safe. (if allocation was successfull, then execution goes on normally, otherwise an exception will be raised on all reasonable platforms I suppose) ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] compile time memory leak detection
20.01.2010 0:22, Marco van de Voort: Totally different issue. The problem is that there was no implementation of memavail that would suit the usage of memavail in old dos code. That was the simple difference between single- and multiprocessing, which was ingrained in the use. Even the simple showing available memory like Turbo Vision apps sometimes did would lead to strange behaviour (one minute it would look like memory was running low, and then later when an additional block was allocated from memory, there would be plenty free memory again) It would become even more confusing when you tried to make decisions based on it. (like trying to allocate a big as possible block to be used as cache). When one would let memavail return the memory available in the heap currently allocated it was too little. If you'd let it return the memory globally available, your risk starving other apps for memory (*), and allocating huge chunks to former 64k apps that won't use more than a few 100ks. There was really no solution then. Well, you are right of course. I just meant that a sole existance of some code relying on some feature does not necessarily prove this feature is sane enough for all times. Standard pointer arithmetics is not inherently broken as memavail. It is even usefull. However, a small mistake in such code also leads to strange behaviour, and we humans tend to sometimes make mistakes... (*) not to speak of the fact that a lot of systems returned virtual memory as free memory. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re[2]: [fpc-devel] compile time memory leak detection
Hello Nikolai, Tuesday, January 19, 2010, 8:10:49 PM, you wrote: NZ> 19.01.2010 17:49, JoshyFun ?: >> var >>p: pchar; >> begin >>GetMem(p,10); >>inc(p,random(30)); NZ> Yes, this is among things which should probably be immediately NZ> disallowed at compile-time (as long as one wants reasonably safe NZ> pointers) because they are hard (or expensive) to validate later. NZ> I usually try to not use raw pointer arithmetics and either use NZ> new/dispose with typed pointers or constructs like GetMem(X, NZ> SizeOf(X^)). However, compiler does not force me to do so. Pointers and safe code is mutually exclusive, like managed code and fast execution, garbage collection and memory optimization, ... In example this will work in some machines and crash in others: GetMem(p,1024*1024*1024); p^:=#0; You can not detect it a compile time in any way. -- Best regards, JoshyFun ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] compile time memory leak detection
In our previous episode, Nikolai Zhubr said: > 19.01.2010 23:18, Marco van de Voort: > > FPC is not going to forbid standard pointer use, since it breaks a lot of > > code. > Of course. I think it all was about new features for new code. > By the way, it reminds me MemAvail (though not exactly the same). Lots > of code used to use it, however AFAIK it is not available anymore (for a > clear reason that it is inappropriate for any modern code) Totally different issue. The problem is that there was no implementation of memavail that would suit the usage of memavail in old dos code. That was the simple difference between single- and multiprocessing, which was ingrained in the use. Even the simple showing available memory like Turbo Vision apps sometimes did would lead to strange behaviour (one minute it would look like memory was running low, and then later when an additional block was allocated from memory, there would be plenty free memory again) It would become even more confusing when you tried to make decisions based on it. (like trying to allocate a big as possible block to be used as cache). When one would let memavail return the memory available in the heap currently allocated it was too little. If you'd let it return the memory globally available, your risk starving other apps for memory (*), and allocating huge chunks to former 64k apps that won't use more than a few 100ks. There was really no solution then. (*) not to speak of the fact that a lot of systems returned virtual memory as free memory. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] compile time memory leak detection
19.01.2010 23:18, Marco van de Voort: FPC is not going to forbid standard pointer use, since it breaks a lot of code. Of course. I think it all was about new features for new code. By the way, it reminds me MemAvail (though not exactly the same). Lots of code used to use it, however AFAIK it is not available anymore (for a clear reason that it is inappropriate for any modern code) I usually try to not use raw pointer arithmetics and either use new/dispose with typed pointers or constructs like GetMem(X, SizeOf(X^)). However, compiler does not force me to do so. In theory, of course a dialect, or dialect switch could be implemented. It could be even combined with an ISO Pascal mode. But the question is always who is going to do it, and who is going to support it long term? This is understood. I was just speculating theoretically. Nikolai ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] compile time memory leak detection
In our previous episode, Nikolai Zhubr said: > >p: pchar; > > begin > >GetMem(p,10); > >inc(p,random(30)); > > Yes, this is among things which should probably be immediately > disallowed at compile-time (as long as one wants reasonably safe > pointers) because they are hard (or expensive) to validate later. To the compiler random is just another procedure. It doesn't grasp that it gives back random values. FPC is not going to forbid standard pointer use, since it breaks a lot of code. > I usually try to not use raw pointer arithmetics and either use > new/dispose with typed pointers or constructs like GetMem(X, > SizeOf(X^)). However, compiler does not force me to do so. In theory, of course a dialect, or dialect switch could be implemented. It could be even combined with an ISO Pascal mode. But the question is always who is going to do it, and who is going to support it long term? ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] compile time memory leak detection
19.01.2010 17:49, JoshyFun пишет: var p: pchar; begin GetMem(p,10); inc(p,random(30)); Yes, this is among things which should probably be immediately disallowed at compile-time (as long as one wants reasonably safe pointers) because they are hard (or expensive) to validate later. I usually try to not use raw pointer arithmetics and either use new/dispose with typed pointers or constructs like GetMem(X, SizeOf(X^)). However, compiler does not force me to do so. Nikolai ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] possible 'dynlibs' unit issue
W dniu 2010-01-19 19:00, Daniël Mantione pisze: Op Tue, 19 Jan 2010, schreef Dariusz Mazur: Is this possible to forbid this at compile time? An easy solution is to wrap it inside a record. This is good practise as it prevents accidental bugs like accidentally writing parameters in the wrong order when calling a procedure. It's basically the point of type safety. Yes. I use this very often. But it is possible only for new types. When we want to test or protect some sort of operation on huge source its harder. Operator overloading is enough but lack something that detect invoke of this function. And its not only for this solution, appliance is much wider (f.e. generics) -- Darek ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] possible 'dynlibs' unit issue
Op Tue, 19 Jan 2010, schreef Dariusz Mazur: Is this possible to forbid this at compile time? An easy solution is to wrap it inside a record. This is good practise as it prevents accidental bugs like accidentally writing parameters in the wrong order when calling a procedure. It's basically the point of type safety. Some languages have "opaque" types designed for this purpose, the difference being that a record only protects against accidental calculations (you could modify the field in the record), while an actual opaque type could enforce it. Daniël___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] possible 'dynlibs' unit issue
W dniu 2010-01-19 13:53, Michael Van Canneyt pisze: On Tue, 19 Jan 2010, Daniël Mantione wrote: Op Tue, 19 Jan 2010, schreef Graeme Geldenhuys: Michael Van Canneyt wrote: Why should it be better ? It doesn't really matter anyway. PtrUInt has a larger range than PtrInt (allowing full access to memory address range). Plus, I don't think pointers can be negative values. The problem occurs when doing pointer arithmetic "ptrint+integer" is a bug, because it will overflow if the original pointer<$800 and the result pointer>=$800. It is especially dangerous in loops like: q:=p+distance; while p That is what I meant. The TLibhandle type is opaque. You're not supposed to know that it is a pointer, integer or whatnot. And definitely you should not do math on them. Is this possible to forbid this at compile time? I thinking about something like this: operator + (a : tLibHandle; b: integer) res : tLibHandle; begin abstract; //compiler can give error at place of invoke end; is this possible? -- Darek ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re[2]: [fpc-devel] compile time memory leak detection
Hello Nikolai, Tuesday, January 19, 2010, 1:38:53 PM, you wrote: NZ> This would probably include: NZ> - guaranree that dynamically allocated will not be silently lost (leak) NZ> - guaranree that dynamically allocated mem will not be disposed twice; NZ> - guarantee that dynamically allocated mem will not be accessed beyond NZ> the respective allocation size. That's impossible to the compiler, it is a runtime task (heaptrc): procedure weird(); var p: pchar; begin GetMem(p,10); inc(p,random(30)); p^:='A'; end; -- Best regards, JoshyFun ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] compile time memory leak detection
19.01.2010 16:16, Thaddy: Nikolai Zhubr wrote: I'd guess this would require huge work and substantial modifications to the language though. The compiler then has to assert all possible codepaths... Do you think that's a viable option? I don't have that much expertise to judge really, but I'd suppose this can't be done at compile time completely. On the other hand, refcounting at run-time could work, theoretically. (IIRC long strings use refcounting already) Nikolai ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] compile time memory leak detection
Nikolai Zhubr wrote: I'd guess this would require huge work and substantial modifications to the language though. The compiler then has to assert all possible codepaths... Do you think that's a viable option? ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] possible 'dynlibs' unit issue
On Tue, 19 Jan 2010, Daniël Mantione wrote: Op Tue, 19 Jan 2010, schreef Graeme Geldenhuys: Michael Van Canneyt wrote: Why should it be better ? It doesn't really matter anyway. PtrUInt has a larger range than PtrInt (allowing full access to memory address range). Plus, I don't think pointers can be negative values. The problem occurs when doing pointer arithmetic "ptrint+integer" is a bug, because it will overflow if the original pointer<$800 and the result pointer>=$800. It is especially dangerous in loops like: q:=p+distance; while pHowever, when using pointers as handles it is another matter... Handles are not to be used in any calculation, so they cannot overflow. Who cares if you get a negative handle? That is what I meant. The TLibhandle type is opaque. You're not supposed to know that it is a pointer, integer or whatnot. And definitely you should not do math on them. Michael.___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] possible 'dynlibs' unit issue
Daniël Mantione wrote: > However, when using pointers as handles it is another matter... Handles > are not to be used in any calculation, so they cannot overflow. Who cares > if you get a negative handle? OK, now it makes sense. Thanks for the clear explanation. Regards, - Graeme - -- fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal http://opensoft.homeip.net/fpgui/ ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] possible 'dynlibs' unit issue
Op Tue, 19 Jan 2010, schreef Graeme Geldenhuys: Michael Van Canneyt wrote: Why should it be better ? It doesn't really matter anyway. PtrUInt has a larger range than PtrInt (allowing full access to memory address range). Plus, I don't think pointers can be negative values. The problem occurs when doing pointer arithmetic "ptrint+integer" is a bug, because it will overflow if the original pointer<$800 and the result pointer>=$800. It is especially dangerous in loops like: q:=p+distance; while pHowever, when using pointers as handles it is another matter... Handles are not to be used in any calculation, so they cannot overflow. Who cares if you get a negative handle? Daniël___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] compile time memory leak detection
19.01.2010 12:31, Jonas Maebe: Does FPC know how to find this places on compile time and at least issue a warning about it ? No, it doesn't. In principle, the compiler can't even be sure that you haven't replaced the default memory manager with e.g. some garbage-collector-style memory manager similar to Boehm GC, in which case such warnings would be wrong. May I put it a bit other way? Would it be possible/feasable to introduce some (optional) restrictions and/or extensions to pointer-related syntax and/or pointer-related parts of RTL so as to completely ensure safety if pointer operations? This would probably include: - guaranree that dynamically allocated will not be silently lost (leak) - guaranree that dynamically allocated mem will not be disposed twice; - guarantee that dynamically allocated mem will not be accessed beyond the respective allocation size. I'd guess this would require huge work and substantial modifications to the language though. And probably, this is not possible without GC, and I don't like GCs anyway :) Nikolai Jonas ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] possible 'dynlibs' unit issue
Michael Van Canneyt wrote: > > Why should it be better ? It doesn't really matter anyway. PtrUInt has a larger range than PtrInt (allowing full access to memory address range). Plus, I don't think pointers can be negative values. Regards, - Graeme - -- fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal http://opensoft.homeip.net/fpgui/ ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] possible 'dynlibs' unit issue
On Tue, 19 Jan 2010, Graeme Geldenhuys wrote:
Hi,
In the file rtl/unix/dynlibs.inc there is the following definition.
Type
{ using PtrInt here is compliant with the other platforms }
TLibHandle = PtrInt;
Shouldn't that rather be PtrUInt type? After all, PtrUInt was recommended
on the wiki.
Why should it be better ? It doesn't really matter anyway.
Anyone relying on the actual type is doing something wrong in each case.
Michael.
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] possible 'dynlibs' unit issue
Hi,
In the file rtl/unix/dynlibs.inc there is the following definition.
Type
{ using PtrInt here is compliant with the other platforms }
TLibHandle = PtrInt;
Shouldn't that rather be PtrUInt type? After all, PtrUInt was recommended
on the wiki.
Regards,
- Graeme -
--
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://opensoft.homeip.net/fpgui/
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] compile time memory leak detection
On 19 Jan 2010, at 09:35, ik wrote: There are sometimes way to "see" that you have memory leak in a code without using any sort of profiler. For example using new/getmem on a local variable without releasing it etc... Does FPC know how to find this places on compile time and at least issue a warning about it ? No, it doesn't. In principle, the compiler can't even be sure that you haven't replaced the default memory manager with e.g. some garbage- collector-style memory manager similar to Boehm GC, in which case such warnings would be wrong. Jonas ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] compile time memory leak detection
http://ik.homelinux.org/ On Tue, Jan 19, 2010 at 10:59, Marco van de Voort wrote: > In our previous episode, ik said: > > There are sometimes way to "see" that you have memory leak in a code > without > > using any sort of profiler. > > For example using new/getmem on a local variable without releasing it > etc... > > > > Does FPC know how to find this places on compile time and at least issue > a > > warning about it ? > > Is this a suitable task for a compiler? Maybe it is a task better suited to > an external profiler. > Well I find myself forgetting from time to time to close allocated memory on a local variable, and a nice warning can make it easier to figure out then running code and finding it after x amount of execution time (and using the feature that uses that code). So, I do not know if it's really a compiler task, but it sure can help :) > ___ > fpc-devel maillist - fpc-devel@lists.freepascal.org > http://lists.freepascal.org/mailman/listinfo/fpc-devel > Ido ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] compile time memory leak detection
In our previous episode, ik said: > There are sometimes way to "see" that you have memory leak in a code without > using any sort of profiler. > For example using new/getmem on a local variable without releasing it etc... > > Does FPC know how to find this places on compile time and at least issue a > warning about it ? Is this a suitable task for a compiler? Maybe it is a task better suited to an external profiler. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] compile time memory leak detection
Hello list, There are sometimes way to "see" that you have memory leak in a code without using any sort of profiler. For example using new/getmem on a local variable without releasing it etc... Does FPC know how to find this places on compile time and at least issue a warning about it ? Thanks, Ido http://ik.homelinux.org/ ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
