-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tim Bunce wrote: > On Sun, Jun 21, 2009 at 04:31:10PM +0200, Robin Smidsrød wrote: >> >> Is NYTProf made extensible to support this kind of thing, or is memory >> profiling something that wouldn't work with its current architecture? > > Fairly simple to do something fairly simple. But tricky to do something > worthwhile. > > Copy-on-write, for example, is a significant issue. Then there's the > issue of memory allocated to lexical pads that were only used once, for > example. As is memory allocated to the perl process but not currently > used by it. > > Your perlbloat.pl script, for example, doesn't account for modules that > use lots of memory *while loading* but then free much of it (back to > perl's own storage pool). The 'bloat' would be reported too high. > When the next module is loaded it'll use some of that freed memory > so its bloat would be reported too low. > > Lots of tricky detail here, in terms of what can be usefully measured > and what it actually means. Lots of thought would also need to be given > to how to report it in a meaningful way. > > Having said all that, I am interested in extending NYTProf in this > direction.
Well, my perlbloat script was snatched from somewhere else (can't remember where), but yes, it does only measure the total size of memory allocated to the process. But that in itself is a useful metric. If you run this script with several modules on the same run you'd get the _combined_ cost of using them in your program. If you run perlbloat several times, with only one module as the argument, you get the cost of that module alone. Actually there are two metrics that are interesting to measure: The size of the process (it's called VSS, Virtual Set Size, right?), which only increases until the process dies, and how much of that memory the perl interpreter is using of that memory at a given time. The GTop perl module based on glib from Gnome tries to get this kind of data in a portable way. That's why I used it in my small script. From its documentation it seems like it also has the ability to measure copy-on-write effects with the dirty_size method (AFAIK). If the system uses a recent linux kernel the perl module Linux::Smaps is also available to us. When it comes to measuring how much memory the interpreter is actually using at a given time (from the allocated) I don't know. Maybe someone with inside knowledge of perl can shed som light on this? Maybe using a combination of Devel::Gladiator and Devel::Size can give you an indication, but I would assume that perl itself keeps a free_process_map of some sort. If we go about this in an organized way I think we could make it useful. * First we need a way to measure the different metrics * Process size (the one that never decreases) * Current usage of memory inside above metric * Possibly shared memory usage aswell? * memory mapped files (part of the above?) * Then we need to add the ability to compute differences * Hook into statement or sub profiler * Then we need to figure out how to store it in the dumpfile * And finally we need to figure out how to report it Anything I've forgotten? Regards, Robin Smidsrød -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAko/c4gACgkQHAwEVD/in26MHACgn11CAOM46zPNSX/fjQh1mVpP ASEAn1V9XdxRinHK+cEnh4kB8vy3QX4M =RE86 -----END PGP SIGNATURE----- --~--~---------~--~----~------------~-------~--~----~ You've received this message because you are subscribed to the Devel::NYTProf Development User group. Group hosted at: http://groups.google.com/group/develnytprof-dev Project hosted at: http://perl-devel-nytprof.googlecode.com CPAN distribution: http://search.cpan.org/dist/Devel-NYTProf To post, email: [email protected] To unsubscribe, email: [email protected] -~----------~----~----~----~------~----~------~--~---
