Re: [fpc-pascal] Is such memory statistics possible?
On Mon, Aug 15, 2011 at 1:03 PM, Marco van de Voort mar...@stack.nl wrote: In our previous episode, Max Vlasov said: about collecting some statistics. The idea is similar to some disk utilities that collects and sort the sizes of directories. You know when every folder on the computer is scanned and all the resulting paths are sorted by the summed size. Such utilities usually help to find space on the hard drive to free. With memory for every memory allocation we can call the path the addresses of procedures in stack. So when a function funci() is called from the function parentfunc() and getmem is called inside func() then we should add this size to the corresponding entries both for func() and parentfunc(). ... Do tools(units) like this already exist? If not, is developing such unit technically possible/hard with the currently available debug information? This is basically what valgrind (or fulldebugmode of fastmm) does. But they do this by parsing the stack on each call to the memory manager, and then keep track of it. Note that all these techniques can be very, very slowing. E.g. I tried to debug the CHM support with valgrind, and I terminated the valgrind process after 5 hours because it was not even half way to where the bug was. Without valgrind the program reached the point in 1-2 minutes. Based on this discussion and the following research I made an attempt to implement something like this for fpc/lazarus. The final result are a couple of units and a dialog that allows to see the results in any time inside the program. You can see the real example in the screenshot: http://www.maxerist.net/downloads/procmemstat_ss.png The download link for pascal sources: http://www.maxerist.net/downloads/procmemstat.zip (5k) The module traditionally installs its memory manager replacement procs and collects statistics. To make things faster on this step it just detects and saves addresses on the stack that falls into code segment range of the main module. When a request for actual dialog is made then it parses all allocated data and resolves the addresses to symbols with GetLineInfo proc. This step is much longer, the dialog in the screenshot took 15 seconds to appear (on my 1.7 GHz Celeron). Currently the monitor is only win32-compatible, but the only platform-specific code is about getting current stack range (I took it according this information http://en.wikipedia.org/wiki/Win32_Thread_Information_Block) and getting code segment range that is made with toolhelp32 snapshots and virtualquery. Finding a way to do the same on linux will possibly make it compatible with linux also. The usage: - add uProcMemMon first in the lpr - define -dPROCMEMMON in the project options-other-custom options - add uProcMemDlg to unit (form) from where you want to show the statistics. - call ProcMemShowDialog from any click handler. Sure you can call the dialog as many times as you want looking what's wrong or right with different states of you program - if you don't want gui then don't use uProcMemDlg, just call PMMCollectStat that will return TStringList with objects as sizes (unsorted) Current limitations: - As I already said, currently it's win32-only - I assume it's currently not thread-safe because of global variable usage - Several first lines of the dialog are not very useful since they're either getmem related chain or main-related procedures, those are always on stack. - The speed of collecting is ok, but although it uses linked list so with many allocation it can drop. Resolving to symbols takes some time by default (GetLineInfo is not very good for thousands of queries) so in these areas some further optimization might possible. - No dynamic loading support. This is due to the fact that the monitor uses the address range of the main module. - Initially I got numbers for winners that were bigger than total allocated memory. This was because of exception blocks so from the point view of the parser this getmem were called at once from multiply lines of the same function. I fixed this by ignoring function duplicate while parsing the same memory block. It seems it worked, but maybe there are cases when some new trick will be necessary If someone finds time to test this approach on real projects, that would be great. As for usefulness the time will show, this time I'd like to know that it at least provide sane results :) Thanks Max Vlasov ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
[fpc-pascal] TTimeStamp and TBCD properties of a class cannot published?
Hi all,
I am developing a new app using a persistence framework initially developed
for Delphi.
The following code doesn't compile:
published
property AsTimeStamp: TTimeStamp read GetAsTimeStamp write
SetAsTimeStamp;
published
property AsBCD: TBCD read GetAsBCD write SetAsBCD;
The compiler error is: Error: This kind of property can't be published
If I move the properties declaration to the public section of the class
declaration,it's fine.
The class implements an interface of the type:
// Interface to provide access to attribute as TimeStamp
IAsTimeStampAccess = interface(IInterface)
['{257D2135-478A-4D46-918F-289678567B7F}']
function GetAsTimeStamp: TTimeStamp;
procedure SetAsTimeStamp(const aValue: TTimeStamp);
property AsTimeStamp: TTimeStamp read GetAsTimeStamp write
SetAsTimeStamp;
end;
But it works for string, integer and other basic type properties.
Is this a bug?
Best regards,
Gerard.
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
RE : [fpc-pascal] Is such memory statistics possible?
Based on this discussion and the following research I made an attempt to implement something like this for fpc/lazarus. The final result are a couple of units and a dialog that allows to see the results in any time inside the program. You can see the real example in the screenshot: http://www.maxerist.net/downloads/procmemstat_ss.png The download link for pascal sources: http://www.maxerist.net/downloads/procmemstat.zip (5k) Where can I find units uNtSnapshot, uMem32, uUniCompat? Looking at the screenshot, I'm confused. Memory allocated by FreeMem. How should that be interpreted? thanks, Ludo ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: RE : [fpc-pascal] Is such memory statistics possible?
On Fri, Aug 19, 2011 at 3:26 PM, Ludo Brands ludo.bra...@free.fr wrote: ** Based on this discussion and the following research I made an attempt to implement something like this for fpc/lazarus. The final result are a couple of units and a dialog that allows to see the results in any time inside the program. You can see the real example in the screenshot: http://www.maxerist.net/downloads/procmemstat_ss.png The download link for pascal sources: http://www.maxerist.net/downloads/procmemstat.zip (5k) Where can I find units uNtSnapshot, uMem32, uUniCompat? Looking at the screenshot, I'm confused. Memory allocated by FreeMem. How should that be interpreted? thanks, Ludo Sorry about that, I uploaded the archive once again with only uNtSnapshot dependency. http://www.maxerist.net/downloads/procmemstat.ziphttp://www.maxerist.net/downloads/procmemstat.zip As for the numbers, they're not supposed to be pieces to be summed. I mentioned utilities for finding space on drive, for example my FindFatFolder (http://www.maxerist.net/main/soft-for-win/find-fat-folder), every reachable folder (also nodes, not only leafs) is used and calculated, if you want to find the eater, you just look from the top to the bottom and wait for 'bingo'. In other words If you see a big number and the name of the function looks like a strange source of such big number (cumulative) you look deeper. For folders it works, but will it work for function I still don't know :) Max ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
RE : RE : RE : [fpc-pascal] Is such memory statistics possible?
OK compiles now. I tested this with 2 programs but when I include uProcMemMon as the first unit in lpr and define -dPROCMEMMON then they both crash with a sigsegv. Undefine PROCMEMMON and they run fine. I've attached a small testprogram that crashes in bufdataset. Just a small copy paste problem: change line 295 Result := OldMemMgr.GetMem(Size); into Result := OldMemMgr.AllocMem(Size); Ludo ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
[fpc-pascal] Suggestions on array allocation pattern/rules of thumb?
Hello, In Lazarus I found (pseudo)some code like this: procedure DoSomething(...) var v_Results: PInteger; v_ResultCount: Integer; begin v_Result := nil; PrepareData(..., v_Results, v_ResultCount); InternalDoSomething(v_Results, v_ResultCount); end; procedure PrepareData(..., var p_Results: PInteger; var p_ResultCount: Integer) begin for (...) if (...) begin Inc(p_ResultCount); ReallocMem(p_Results, p_ResultCount); end; end; As you might guess my main gripe is about the repeated calls to ReallocMem. The real case in the LCL is far from critical, it's infrequently and the array won't ever be bigger than 10 elements, I just thought it could a good example to hear from someone knowledgeable in FPC heap-allocator about any rules of thumb :-) Apart from making the parameters 'out' - ;) - what would you do? 1) This is this ok. 2) Allocate a buffer with maximum size and then trim it after the loop. 3) Declare a type for the maximum size, allocate it on the heap (in DoSomething). 4) Declare a type for the maximum size, allocate it on the stack (in DoSomething). 5) Any other options? Any remarks regarding different scenarios is also appreciated. Thanks, Flávio ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
