Re: Carp vs Carp::Heavy was: Re: [PATCH] Put Carp into the Tarpit (No Carp #2 - Archive::Tar)
On Tue, Jun 28, 2005 at 11:24:59AM -0700, Michael G Schwern wrote: Its the principle of the thing. Also, not initializing $Verbose and $MaxEvalLen seems going a little overboard espcially when it violates the docs. Finally I threw in our onto the globals so one can turn on strict during testing. And a version bump. I've applied your patch apart from the bit that moved the POD to the top; as I still disagree :-( Useless statistic of the day: of the core .pm files, 133 have their pod before __END__ and 205 have it after, while 10 have bits both and after. -- The greatest achievement of the Austrians has been convincing the world that Hitler was German, and Mozart Austrian.
Re: Carp vs Carp::Heavy was: Re: [PATCH] Put Carp into the Tarpit (No Carp #2 - Archive::Tar)
On Fri, Jul 01, 2005 at 04:05:21PM +0100, Dave Mitchell wrote: On Tue, Jun 28, 2005 at 11:24:59AM -0700, Michael G Schwern wrote: Its the principle of the thing. Also, not initializing $Verbose and $MaxEvalLen seems going a little overboard espcially when it violates the docs. Finally I threw in our onto the globals so one can turn on strict during testing. And a version bump. I've applied your patch apart from the bit that moved the POD to the top; as I still disagree :-( That its inefficient or that it doesn't matter? -- Michael G Schwern [EMAIL PROTECTED] http://www.pobox.com/~schwern Just call me 'Moron Sugar'. http://www.somethingpositive.net/sp05182002.shtml
Re: Carp vs Carp::Heavy was: Re: [PATCH] Put Carp into the Tarpit (No Carp #2 - Archive::Tar)
On Fri, Jul 01, 2005 at 03:00:10PM -0700, Michael G Schwern wrote: On Fri, Jul 01, 2005 at 04:05:21PM +0100, Dave Mitchell wrote: On Tue, Jun 28, 2005 at 11:24:59AM -0700, Michael G Schwern wrote: Its the principle of the thing. Also, not initializing $Verbose and $MaxEvalLen seems going a little overboard espcially when it violates the docs. Finally I threw in our onto the globals so one can turn on strict during testing. And a version bump. I've applied your patch apart from the bit that moved the POD to the top; as I still disagree :-( That its inefficient or that it doesn't matter? I disagree that the POD should be at the top because * it's less efficient: Perl will stop reading a src file when it encounters an __END__; on my FC3 system, perl does a single read() syscall of the first 4K of the file; without the __END__ it has to do a second call to get the remaining 2K; and it seems to cause the process to malloc() more * on aethestic grounds: I like the fact that all the code fits in the first screen of an editor, terminated by __END__, so you immediatately know that that's all the code there is, rather than having to scroll through the whole file to make sure. -- This is a great day for France! -- Nixon at Charles De Gaulle's funeral
Re: Carp vs Carp::Heavy was: Re: [PATCH] Put Carp into the Tarpit (No Carp #2 - Archive::Tar)
On Fri, Jul 01, 2005 at 11:39:18PM +0100, Dave Mitchell wrote: I disagree that the POD should be at the top because * it's less efficient: Perl will stop reading a src file when it encounters an __END__; on my FC3 system, perl does a single read() syscall of the first 4K of the file; without the __END__ it has to do a second call to get the remaining 2K; and it seems to cause the process to malloc() more We're down to arguing individual mallocs. There's an old joke about hikers who would clip the tags off their teabags so save weight. * on aethestic grounds: I like the fact that all the code fits in the first screen of an editor, terminated by __END__, so you immediatately know that that's all the code there is, rather than having to scroll through the whole file to make sure. That's really a special case for Carp.pm at this single point in time. What I like about inline is the documentation for a given function or global is right there on the screen at the same time as you're looking at the code. It makes for less greping and scrolling. It also makes for less duplicating the documentation in comments (as illustrated in $MaxArg*). Curtis points out to me that one never wins coding arguments on aesthetics anyway. :) Also this is really all navel gazing as Carp.pm is now so small as to not matter. -- Michael G Schwern [EMAIL PROTECTED] http://www.pobox.com/~schwern 'All anyone gets in a mirror is themselves,' she said. 'But what you gets in a good gumbo is everything.' -- Witches Abroad by Terry Prachett
RE: Carp vs Carp::Heavy was: Re: [PATCH] Put Carp into the Tarpit (No Carp #2 - Archive::Tar)
cummulative changes, all running ./perl -Ilib -e 'use Carp; 1' mem used at end total malloced 178232 234839 before the changes 178232 230735 moving pod to end 178232 230735 removing comments 178232 228575 removing pod snip So moving and removing the pod both affect the accumulated mallocs(), but don't affect the residual unfreed memory at process exit. I wonder why POD altered the amount of memory used at all. Is Perl storing it somewhere? May be Perl is not storing POD somewhere, but in the middle of parsing it allocates some memory which is not returned to OS but anyways deallocated and will be reused on further memory demands?
Re: Carp vs Carp::Heavy was: Re: [PATCH] Put Carp into the Tarpit (No Carp #2 - Archive::Tar)
On Mon, Jun 27, 2005 at 03:57:23PM -0700, Michael G Schwern wrote: mem used at end total malloced command 159904 195136 ./perl -Ilib -MExporter -e 1 176330 224251 ./perl -Ilib -e 'use Carp; 1' it would appear that Carp uses now 15-30K, assuming that Exporter gets loaded anyway. How much did it use before? Without that information we don't know what was saved. It would also be nice to see each optimization individually to see if its saving anything. cummulative changes, all running ./perl -Ilib -e 'use Carp; 1' mem used at end total malloced 178232 234839 before the changes 178232 230735 moving pod to end 178232 230735 removing comments 178232 228575 removing pod 178023 226624 moving variable initialisations to Carp::Heavy 176330 224251 streamlining code So moving and removing the pod both affect the accumulated mallocs(), but don't affect the residual unfreed memory at process exit. The totality of these changes save somewhere between 2-10K -- In my day, we used to edit the inodes by hand. With magnets.
Re: Carp vs Carp::Heavy was: Re: [PATCH] Put Carp into the Tarpit (No Carp #2 - Archive::Tar)
Ok, I admit I was wr... wroo... wronnn... Otherwisely Correct Change 25006 by [EMAIL PROTECTED] on 2005/06/28 11:28:20 move Carp.pod back into Carp.pm -- A walk of a thousand miles begins with a single step... then continues for another 1,999,999 or so.
Re: Carp vs Carp::Heavy was: Re: [PATCH] Put Carp into the Tarpit (No Carp #2 - Archive::Tar)
-BEGIN PGP SIGNED MESSAGE- Moin Dave, On Tuesday 28 June 2005 13:56, Dave Mitchell wrote: Ok, I admit I was wr... wroo... wronnn... Otherwisely Correct Change 25006 by [EMAIL PROTECTED] on 2005/06/28 11:28:20 move Carp.pod back into Carp.pm Thanx for the streamlining. I'd say that the patch number #8 from me (no Carp #8 - SelfLoader, Text/Balanced and open.pm) should be still applied, because it works towards more consistent code. Best wishes, Tels PS: cummulative changes, all running ./perl -Ilib -e 'use Carp; 1' What do you get with ./perl -Ilib -e 'use Exporter; 1' (aka: w/o Carp at all)? How much memory does loading an empty module need? Best wishes, Tels - -- Signed on Tue Jun 28 18:05:26 2005 with key 0x93B84C15. Visit my photo gallery at http://bloodgate.com/photos/ PGP key on http://bloodgate.com/tels.asc or per email. Ok, a shiny copper piece to the first person who can tell me why Elan is running around naked. - Giant moth attack? - Maybe a bad guy casted 'Dispel clothes'? - The Order of the Stick issue #26 -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.4 (GNU/Linux) iQEVAwUBQsF14ncLPEOTuEwVAQFuYwf+LWpo7hEBIfNC10OugI6TL9S6QHeSzwxb TZD4RUtjaeNnQshh21g9c7dx23MNZJnZjPdfcg9lV8qfupDLpVycTMN/F+9c82DG UL4UTcfwTznAamC8NxtP+TTcN5ocNlvyUn6ZZ5zzKqbau3ceoQ0DGTco952GvIEV 2XLXJKf2XCBQM38ANhd8mFheIik/GSMBtNACD+OOb1gexS82TokpmfkOKolHbnm8 eazxLOhLNzTS15AMYx4ktlzzk5djGPp1DAg9qEgaWvtoU2zDMHiunI97Zm9DDn7f pDKwV6mHe33iInd3v+Fjs011KqlLCX5ZRdnlP/BUHujzXkJG5Hze6Q== =NWeY -END PGP SIGNATURE-
Re: Carp vs Carp::Heavy was: Re: [PATCH] Put Carp into the Tarpit (No Carp #2 - Archive::Tar)
On Tue, Jun 28, 2005 at 12:30:19PM +0100, Dave Mitchell wrote: cummulative changes, all running ./perl -Ilib -e 'use Carp; 1' mem used at end total malloced 178232234839 before the changes 178232230735 moving pod to end 178232230735 removing comments 178232228575 removing pod snip So moving and removing the pod both affect the accumulated mallocs(), but don't affect the residual unfreed memory at process exit. I wonder why POD altered the amount of memory used at all. Is Perl storing it somewhere? -- Michael G Schwern [EMAIL PROTECTED] http://www.pobox.com/~schwern 'All anyone gets in a mirror is themselves,' she said. 'But what you gets in a good gumbo is everything.' -- Witches Abroad by Terry Prachett
Re: Carp vs Carp::Heavy was: Re: [PATCH] Put Carp into the Tarpit (No Carp #2 - Archive::Tar)
On Tue, Jun 28, 2005 at 12:56:43PM +0100, Dave Mitchell wrote: Ok, I admit I was wr... wroo... wronnn... Otherwisely Correct Change 25006 by [EMAIL PROTECTED] on 2005/06/28 11:28:20 move Carp.pod back into Carp.pm Except now its at the end of the file instead of inline. -- Michael G Schwern [EMAIL PROTECTED] http://www.pobox.com/~schwern Ahh email, my old friend. Do you know that revenge is a dish that is best served cold? And it is very cold on the Internet!
Re: Carp vs Carp::Heavy was: Re: [PATCH] Put Carp into the Tarpit (No Carp #2 - Archive::Tar)
On Tue, Jun 28, 2005 at 09:57:11AM -0700, Michael G Schwern wrote: On Tue, Jun 28, 2005 at 12:56:43PM +0100, Dave Mitchell wrote: Ok, I admit I was wr... wroo... wronnn... Otherwisely Correct Change 25006 by [EMAIL PROTECTED] on 2005/06/28 11:28:20 move Carp.pod back into Carp.pm Except now its at the end of the file instead of inline. yes, all of 10 lines away from the functions its documenting :-) -- Little fly, thy summer's play my thoughtless hand has terminated with extreme prejudice. (with apologies to William Blake)
Re: Carp vs Carp::Heavy was: Re: [PATCH] Put Carp into the Tarpit (No Carp #2 - Archive::Tar)
On Tue, Jun 28, 2005 at 09:55:39AM -0700, Michael G Schwern wrote: On Tue, Jun 28, 2005 at 12:30:19PM +0100, Dave Mitchell wrote: cummulative changes, all running ./perl -Ilib -e 'use Carp; 1' mem used at end total malloced 178232 234839 before the changes 178232 230735 moving pod to end 178232 230735 removing comments 178232 228575 removing pod snip So moving and removing the pod both affect the accumulated mallocs(), but don't affect the residual unfreed memory at process exit. I wonder why POD altered the amount of memory used at all. Is Perl storing it somewhere? probably buffers used by the IO subsystem while reading in the src -- Lady Nancy Astor: If you were my husband, I would flavour your coffee with poison. Churchill: Madam - if I were your husband, I would drink it.
Re: Carp vs Carp::Heavy was: Re: [PATCH] Put Carp into the Tarpit (No Carp #2 - Archive::Tar)
On Tue, Jun 28, 2005 at 06:59:09PM +0100, Dave Mitchell wrote: mem used at end total malloced 178232234839 before the changes 178232230735 moving pod to end 178232230735 removing comments 178232228575 removing pod snip So moving and removing the pod both affect the accumulated mallocs(), but don't affect the residual unfreed memory at process exit. I wonder why POD altered the amount of memory used at all. Is Perl storing it somewhere? probably buffers used by the IO subsystem while reading in the src If that's true, why didn't removing the comments cause a memory drop? -- Michael G Schwern [EMAIL PROTECTED] http://www.pobox.com/~schwern You are wicked and wrong to have broken inside and peeked at the implementation and then relied upon it. -- tchrist in [EMAIL PROTECTED]
Re: Carp vs Carp::Heavy was: Re: [PATCH] Put Carp into the Tarpit (No Carp #2 - Archive::Tar)
On Tue, Jun 28, 2005 at 11:16:41AM -0700, Michael G Schwern wrote: On Tue, Jun 28, 2005 at 06:59:09PM +0100, Dave Mitchell wrote: mem used at end total malloced 178232 234839 before the changes 178232 230735 moving pod to end 178232 230735 removing comments 178232 228575 removing pod snip So moving and removing the pod both affect the accumulated mallocs(), but don't affect the residual unfreed memory at process exit. I wonder why POD altered the amount of memory used at all. Is Perl storing it somewhere? probably buffers used by the IO subsystem while reading in the src If that's true, why didn't removing the comments cause a memory drop? actually, removing just the comments or the pod on their own didn't drop; removing both of them did; I'm guessing it pushed the total file size over some limit that caused more/bigger buffers to be allocated. -- Standards (n). Battle insignia or tribal totems.
Re: Carp vs Carp::Heavy was: Re: [PATCH] Put Carp into the Tarpit (No Carp #2 - Archive::Tar)
On Tue, Jun 28, 2005 at 06:57:40PM +0100, Dave Mitchell wrote:
On Tue, Jun 28, 2005 at 09:57:11AM -0700, Michael G Schwern wrote:
On Tue, Jun 28, 2005 at 12:56:43PM +0100, Dave Mitchell wrote:
Ok, I admit I was wr... wroo... wronnn... Otherwisely Correct
Change 25006 by [EMAIL PROTECTED] on 2005/06/28 11:28:20
move Carp.pod back into Carp.pm
Except now its at the end of the file instead of inline.
yes, all of 10 lines away from the functions its documenting :-)
Its the principle of the thing. Also, not initializing $Verbose and
$MaxEvalLen seems going a little overboard espcially when it violates the
docs. Finally I threw in our onto the globals so one can turn on strict
during testing. And a version bump.
The next target for use less 'memory' should probably be Exporter.
--
Michael G Schwern [EMAIL PROTECTED] http://www.pobox.com/~schwern
Reality is that which, when you stop believing in it, doesn't go away.
-- Phillip K. Dick
--- lib/Carp.pm 2005/06/28 18:18:18 1.1
+++ lib/Carp.pm 2005/06/28 18:23:52
@@ -1,43 +1,7 @@
package Carp;
-our $VERSION = '1.04';
-# this file is an utra-lightweight stub. The first time a function is
-# called, Carp::Heavy is loaded, and the real short/longmessmess_jmp
-# subs are installed
-
-# $MaxEvalLen, $Verbose
-# are supposed to default to 0, but undef should be close enough
-$CarpLevel = 0;
-$MaxArgLen = 64;# How much of each argument to print. 0 = all.
-$MaxArgNums = 8;# How many arguments to print. 0 = all.
+our $VERSION = '1.05';
-require Exporter;
[EMAIL PROTECTED] = ('Exporter');
[EMAIL PROTECTED] = qw(confess croak carp);
[EMAIL PROTECTED] = qw(cluck verbose longmess shortmess);
[EMAIL PROTECTED] = qw(verbose);# hook to enable verbose mode
-
-# if the caller specifies verbose usage (perl -MCarp=verbose script.pl)
-# then the following method will be called by the Exporter which knows
-# to do this thanks to @EXPORT_FAIL, above. $_[1] will contain the word
-# 'verbose'.
-
-sub export_fail { shift; $Verbose = shift if $_[0] eq 'verbose'; @_ }
-
-# fixed hooks for stashes to point to
-sub longmess { goto longmess_jmp }
-sub shortmess { goto shortmess_jmp }
-# these two are replaced when Carp::Heavy is loaded
-sub longmess_jmp {{ local($@, $!); require Carp::Heavy} goto longmess_jmp}
-sub shortmess_jmp {{ local($@, $!); require Carp::Heavy} goto shortmess_jmp}
-
-sub croak { die shortmess @_ }
-sub confess { die longmess @_ }
-sub carp{ warn shortmess @_ }
-sub cluck { warn longmess @_ }
-
-1;
-__END__
=head1 NAME
@@ -187,6 +151,48 @@
Note, this is analogous to using Cuse Carp 'verbose'.
Defaults to C0.
+
+=cut
+
+
+# this file is an utra-lightweight stub. The first time a function is
+# called, Carp::Heavy is loaded, and the real short/longmessmess_jmp
+# subs are installed
+
+our $MaxEvalLen = 0;
+our $Verbose= 0;
+our $CarpLevel = 0;
+our $MaxArgLen = 64; # How much of each argument to print. 0 = all.
+our $MaxArgNums = 8;# How many arguments to print. 0 = all.
+
+require Exporter;
+our @ISA = ('Exporter');
+our @EXPORT = qw(confess croak carp);
+our @EXPORT_OK = qw(cluck verbose longmess shortmess);
+our @EXPORT_FAIL = qw(verbose);# hook to enable verbose mode
+
+# if the caller specifies verbose usage (perl -MCarp=verbose script.pl)
+# then the following method will be called by the Exporter which knows
+# to do this thanks to @EXPORT_FAIL, above. $_[1] will contain the word
+# 'verbose'.
+
+sub export_fail { shift; $Verbose = shift if $_[0] eq 'verbose'; @_ }
+
+# fixed hooks for stashes to point to
+sub longmess { goto longmess_jmp }
+sub shortmess { goto shortmess_jmp }
+# these two are replaced when Carp::Heavy is loaded
+sub longmess_jmp {{ local($@, $!); require Carp::Heavy} goto longmess_jmp}
+sub shortmess_jmp {{ local($@, $!); require Carp::Heavy} goto shortmess_jmp}
+
+sub croak { die shortmess @_ }
+sub confess { die longmess @_ }
+sub carp{ warn shortmess @_ }
+sub cluck { warn longmess @_ }
+
+1;
+
+=cut
=head1 BUGS
Re: Carp vs Carp::Heavy was: Re: [PATCH] Put Carp into the Tarpit (No Carp #2 - Archive::Tar)
On Tue, Jun 28, 2005 at 11:24:59AM -0700, Michael G Schwern wrote: Its the principle of the thing. Also, not initializing $Verbose and $MaxEvalLen seems going a little overboard espcially when it violates the docs. Finally I threw in our onto the globals so one can turn on strict during testing. And a version bump. your patch increases final memory usage by 460 bytes, and total mallocs by 1200 bytes. The next target for use less 'memory' should probably be Exporter. Exporter's already pretty tight. -- The perl5 internals are a complete mess. It's like Jenga - to get the perl5 tower taller and do something new you select a block somewhere in the middle, with trepidation pull it out slowly, and then carefully balance it somewhere new, hoping the whole edifice won't collapse as a result. - Nicholas Clark, based on an original by Simon Cozens.
Re: Carp vs Carp::Heavy was: Re: [PATCH] Put Carp into the Tarpit (No Carp #2 - Archive::Tar)
On Tue, Jun 28, 2005 at 10:21:20PM +0100, Dave Mitchell wrote: On Tue, Jun 28, 2005 at 11:24:59AM -0700, Michael G Schwern wrote: Its the principle of the thing. Also, not initializing $Verbose and $MaxEvalLen seems going a little overboard espcially when it violates the docs. Finally I threw in our onto the globals so one can turn on strict during testing. And a version bump. your patch increases final memory usage by 460 bytes, and total mallocs by 1200 bytes. Hopefully that's just for informational purposes and you're not actually proposing rejecting the patch to save 460 bytes. The next target for use less 'memory' should probably be Exporter. Exporter's already pretty tight. Exporter will load Exporter::Heavy if it sees foo in its import list. It will not for foo. This is because any work for symbols goes off into Exporter::Heavy. This always struck me as being a bit unnecessary. A bunch of modules use foo style exports. It would be nice if Exporter wouldn't do this. The obvious thing to do is strip off the before processing but the extreme memory saving code style in Exporter makes this kind of difficult. I'll take a whack at it. -- Michael G Schwern [EMAIL PROTECTED] http://www.pobox.com/~schwern 'All anyone gets in a mirror is themselves,' she said. 'But what you gets in a good gumbo is everything.' -- Witches Abroad by Terry Prachett
Re: Carp vs Carp::Heavy was: Re: [PATCH] Put Carp into the Tarpit (No Carp #2 - Archive::Tar)
I've taken the opportunity of make making Carp.pm even lighter (change #24995), by moving its pod into a separate file, deferring the creation of various package vars until Carp::Heavy is included, and stripping longmess and shortmess so they are just empty hooks that trigger 'require Carp::Heavy' Running valgrind to get an idea of: * malloc()ed memory not freed at end (a vague indication of process size), * total mallocs() - gives an upper bound on peak memory usage, although doesn't allow for memory that was free()ed and them re-malloc()ed - it would count that memory twice; I get: mem used at end total malloced command 159904 195136 ./perl -Ilib -MExporter -e 1 176330 224251 ./perl -Ilib -e 'use Carp; 1' it would appear that Carp uses now 15-30K, assuming that Exporter gets loaded anyway. (this is a debugging perl; without that it would probably be less) Dave. -- This email is confidential, and now that you have read it you are legally obliged to shoot yourself. Or shoot a lawyer, if you prefer. If you have received this email in error, place it in its original wrapping and return for a full refund. By opening this email, you accept that Elvis lives.
Re: Carp vs Carp::Heavy was: Re: [PATCH] Put Carp into the Tarpit (No Carp #2 - Archive::Tar)
On Mon, Jun 27, 2005 at 06:45:47PM +0100, Dave Mitchell wrote: I've taken the opportunity of make making Carp.pm even lighter (change #24995), by moving its pod into a separate file Ahh, the great POD stripping optimization arises from the past. Documentation should be as close to the code it documents as possible to increase the likelyhood they will be updated simultaneously. Moving the POD out to a separate file wrecks this so its an actively harmful optimization. Even moving the POD to an __END__ block, rather than inline, degrades its maintainability. How much load time does this save? By my benchmarks, nothing. Why? Because Perl can parse POD easier than a comment. Its just /^=/ and /^=cut/. How much memory does this save? None. USER PID %CPU %MEM VSZRSS TT STAT STARTED TIME COMMAND schwern 21170 0.0 0.219084 1676 std S 3:50PM 0:00.02 bleadperl -I. -wle use CarpNoPOD; sleep 100 schwern 21171 0.0 0.219084 1676 std S 3:50PM 0:00.02 bleadperl -I. -wle use CarpPOD; sleep 100 schwern 21175 0.0 0.119032 1056 std S 3:50PM 0:00.00 perl5.8.6 -I. -wle use CarpPOD; sleep 100 schwern 21176 0.0 0.119032 1056 std S 3:50PM 0:00.02 perl5.8.6 -I. -wle use CarpNoPOD; sleep 100 (Note, that bleadperl is a little old. 24148. This is on OS X 10.3) Granted I'm using the inaccurate process size method, but if the memory saved is so little that it doesn't register here its not worth the maintenance degredation that POD stripping entails. I get: mem used at end total malloced command 159904195136 ./perl -Ilib -MExporter -e 1 176330224251 ./perl -Ilib -e 'use Carp; 1' it would appear that Carp uses now 15-30K, assuming that Exporter gets loaded anyway. How much did it use before? Without that information we don't know what was saved. It would also be nice to see each optimization individually to see if its saving anything. -- Michael G Schwern [EMAIL PROTECTED] http://www.pobox.com/~schwern Ahh email, my old friend. Do you know that revenge is a dish that is best served cold? And it is very cold on the Internet!
Re: Carp vs Carp::Heavy was: Re: [PATCH] Put Carp into the Tarpit (No Carp #2 - Archive::Tar)
On Sun, Jun 26, 2005 at 01:58:26PM +0200, Tels wrote:
(Including vars, because that gives us strict, vars, and warnings as a
baseline):
# ./perl -Ilib -Mvars -le 'print join (\n, sort keys %INC); sleep (100)'
strict.pm
vars.pm
warnings.pm
warnings/register.pm
Carp doesn't use any of this. Just Exporter.
$ perl -MExporter -MCarp -wle 'END { print join \n, keys %INC }'
Carp.pm
Exporter.pm
The only module Carp uses is Exporter. You didn't include Exporter in your
baseline so what you're really testing is the cost of Carp + Exporter vs
strict + vars + warnings + warnings::register. ie. nonsense.
What you need to check is:
No modules (its important to retain all the testing code that will be used
later, it costs memory).
$ perl -wle 'print join \n, keys %INC; sleep 100;'
schwern 12654 0.0 2.0 3776 1256 pts/0SJun21 0:00 perl -wle print
join \n, keys %INC; sleep 100;
Exporter.
$ perl -MExporter -wle 'print join \n, keys %INC; sleep 100'
Exporter.pm
schwern 12618 0.0 2.4 3916 1460 pts/0SJun21 0:00 perl -MExporter
-wle print join \n, keys %INC; sleep 100
Carp.
$ perl -MCarp -wle 'print join \n, keys %INC; sleep 100'
Carp.pm
Exporter.pm
schwern 12626 0.0 2.5 3916 1516 pts/0SJun21 0:00 perl -MCarp -wle
print join \n, keys %INC; sleep 100
Carp + Carp::Heavy.
$ perl -MCarp -MCarp::Heavy -wle 'print join \n, keys %INC; sleep 100'
warnings.pm
Carp.pm
Carp/Heavy.pm
strict.pm
Exporter.pm
schwern 12629 0.0 2.8 4048 1724 pts/0SJun21 0:00 perl -MCarp
-MCarp::Heavy -wle print join \n, keys %INC; sleep 100
I never was any good at interpreting ps output but it looks like loading
Carp on top of Exporter adds a grand total of 56K on my Debian box (5.8.4).
Carp::Heavy slugs in 208K plus another 132K of virtual memory. Exporter
weighs in at 204K + 140K virtual.
I'd take the Exporter numbers with a grain of salt, its a very small module.
No where near the size of Carp::Heavy and shouldn't cost that much memory.
The above might be measuring some sort of first module loaded cost. Have
a look at loading strict vs strict + Exporter.
schwern 12659 0.0 2.2 3784 1376 pts/0SJun21 0:00 perl -Mstrict
-wle print join \n, keys %INC; sleep 100;
schwern 12662 0.0 2.4 3916 1492 pts/0SJun21 0:00 perl -MExporter
-Mstrict -wle print join \n, keys %INC; sleep 100;
strict costs 120K + 8K vm. strict + Exporter is 236K + 140K vm. Note that
if we add the cost of strict alone with the cost of Exporter alone the
numbers don't jive with loading both of them together. It should cost 324K
+ 148K vm, but it actually costs nearly 100K less. So checking the cost
of loading a module by subtracting the cost of loading all its dependencies
from the cost of loading the module might not be accurate.
--
Michael G Schwern [EMAIL PROTECTED] http://www.pobox.com/~schwern
Don't try the paranormal until you know what's normal.
-- Lords and Ladies by Terry Prachett
Re: Carp vs Carp::Heavy was: Re: [PATCH] Put Carp into the Tarpit (No Carp #2 - Archive::Tar)
-BEGIN PGP SIGNED MESSAGE- Moin, On Sunday 26 June 2005 14:32, Michael G Schwern wrote: On Sun, Jun 26, 2005 at 01:58:26PM +0200, Tels wrote: (Including vars, because that gives us strict, vars, and warnings as a baseline): # ./perl -Ilib -Mvars -le 'print join (\n, sort keys %INC); sleep (100)' strict.pm vars.pm warnings.pm warnings/register.pm Carp doesn't use any of this. Just Exporter. You are right, my bad. schwern 12629 0.0 2.8 4048 1724 pts/0SJun21 0:00 perl -MCarp -MCarp::Heavy -wle print join \n, keys %INC; sleep 100 I never was any good at interpreting ps output but it looks like loading Carp on top of Exporter adds a grand total of 56K on my Debian box (5.8.4). Carp::Heavy slugs in 208K plus another 132K of virtual memory. Exporter weighs in at 204K + 140K virtual. I'd take the Exporter numbers with a grain of salt, its a very small module. No where near the size of Carp::Heavy and shouldn't cost that much memory. The above might be measuring some sort of first module loaded cost. Have a look at loading strict vs strict + Exporter. schwern 12659 0.0 2.2 3784 1376 pts/0SJun21 0:00 perl -Mstrict -wle print join \n, keys %INC; sleep 100; schwern 12662 0.0 2.4 3916 1492 pts/0SJun21 0:00 perl -MExporter -Mstrict -wle print join \n, keys %INC; sleep 100; strict costs 120K + 8K vm. strict + Exporter is 236K + 140K vm. Note that if we add the cost of strict alone with the cost of Exporter alone the numbers don't jive with loading both of them together. It should cost 324K + 148K vm, but it actually costs nearly 100K less. So checking the cost of loading a module by subtracting the cost of loading all its dependencies from the cost of loading the module might not be accurate. See Dave's post about malloc granularity - that might explain a few things. So, in conclusion, I think it is worthwhile to apply these changes, because: * no matter how you measure it, memory consumption goes down * the changes are small and pure-perl only Does anybody disagree with me? Best wishes, Tels - -- Signed on Sun Jun 26 15:12:05 2005 with key 0x93B84C15. Visit my photo gallery at http://bloodgate.com/photos/ PGP key on http://bloodgate.com/tels.asc or per email. online porn is the most concerning thing to psychological health that I know of existing today. -- Mary Anne Layden in ttp://tinyurl.com/6a9cy (I fear what she will say about violence in computer games, evening news or Hollywood block-busters...) -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.4 (GNU/Linux) iQEVAwUBQr6qxXcLPEOTuEwVAQEzswf7B9rfG4+3ip3qGc3XCaMFy/4fNniZuV6I Q+SfEMRanjJWf86pgjHmW6qYJtTmT7fHVwXzfjqnZBcw2UNMp4vK21stPIgYZSaK mYcD+hMqghdrlO9P863YYwQ7Q5Igp2pOGjZCsGR0j9Sb0Gr2v/9O33VZiQoFB5DZ wvliZyVdrcnxtd7SBTANdjMk+TOcah640warwZ3sV/FBrZqxedO295eCl0lyfPWS GQh4JFoCO+li2k6xcB8wqQ5BmiAybtPx5rskwCSV2fq2ioPRGsKPQroE+wDa08oc edo3rkwe4ppINgKwMDzZQ8BNucX9cdsBqK6Jvm91Rds0Cc43gcBTnQ== =p4nh -END PGP SIGNATURE-
Re: Carp vs Carp::Heavy was: Re: [PATCH] Put Carp into the Tarpit (No Carp #2 - Archive::Tar)
On Sun, Jun 26, 2005 at 03:16:37PM +0200, Tels wrote: * no matter how you measure it, memory consumption goes down Only if the whole code body manages to avoid including Carp.pm. As soon as any module anywhere manages to include it, even indirectly (quite likely), its a net loss since as well as Carp.pm using memory, you have a bunch of extra wrapper functions in each of the modules that have been 'improved', all using memory too. -- To collect all the latest movies, simply place an unprotected ftp server on the Internet, and wait for the disk to fill
Re: Carp vs Carp::Heavy was: Re: [PATCH] Put Carp into the Tarpit (No Carp #2 - Archive::Tar)
-BEGIN PGP SIGNED MESSAGE- Moin, On Sunday 26 June 2005 18:35, Dave Mitchell wrote: On Sun, Jun 26, 2005 at 03:16:37PM +0200, Tels wrote: * no matter how you measure it, memory consumption goes down Only if the whole code body manages to avoid including Carp.pm. As soon as any module anywhere manages to include it, even indirectly (quite likely), its a net loss since as well as Carp.pm using memory, you have a bunch of extra wrapper functions in each of the modules that have been 'improved', all using memory too. Yes, the grand masterplan was to do all modules in the core. I just didn't want to send in one big monster patch, but do it step by step. I had to start somewhere :) And in any event, nobody loades _all_ modules at the same time, so even if there is one module still loading Carp, it still would be a gain for all modules that use anything beside the left-over module(s). For instance, Benchmark.pm probably doesn't need to worry about loading Carp, if you do benchmarks, that bit of memory hardly matters. I am more worried about everyday scripts and the most-often used modules like File::Path etc. Best wishes, Tels - -- Signed on Sun Jun 26 19:57:28 2005 with key 0x93B84C15. Visit my photo gallery at http://bloodgate.com/photos/ PGP key on http://bloodgate.com/tels.asc or per email. Some spammers have this warped idea that their freedom of speech is guaranteed all the way into my hard drive, but it is my firm belief that their rights end at my firewall. -- Nigel Featherston -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.4 (GNU/Linux) iQEVAwUBQr7tzXcLPEOTuEwVAQGE+wf+LvQZh8dNpvZfaKHuGynKZjlRtVKYDt/9 rqhe+Azy9GcIzvjC7dg+G/RAuIyv14SLYIaxXBmQ15RZEPG7Eyz1KMLh3AYbSDl1 9Zp4xyEExUEkn8WHKo+GSYQMGCc+CMLTcZmo7KPX5n5qoskyqlvgC/JxhCsWKITS XcaPOqdoZpgjOBO68uo0huhtMXy8WN5eKtuv0US/UmMO2RGJb924dhimnv8/i3Wt aSXJ1Chia5S6rEu5bfvq8yxegWD7aqieVKcAzIyRG86LqbcEFEEuyKl2/aiN/AcU O5PcRljM6Dpl06+y7ODjyab5UjvjkwqkZ8ZmI/pfxRFlpHc5CWo/fQ== =Eq/4 -END PGP SIGNATURE-
Re: Carp vs Carp::Heavy was: Re: [PATCH] Put Carp into the Tarpit (No Carp #2 - Archive::Tar)
-BEGIN PGP SIGNED MESSAGE- Moin, On Sunday 26 June 2005 14:32, Michael G Schwern wrote: On Sun, Jun 26, 2005 at 01:58:26PM +0200, Tels wrote: (Including vars, because that gives us strict, vars, and warnings as a baseline): strict costs 120K + 8K vm. strict + Exporter is 236K + 140K vm. Note that if we add the cost of strict alone with the cost of Exporter alone the numbers don't jive with loading both of them together. It should cost 324K + 148K vm, but it actually costs nearly 100K less. So checking the cost of loading a module by subtracting the cost of loading all its dependencies from the cost of loading the module might not be accurate. I had an idea: It might be that loading a module allocates some memory and then frees it again, but doesn't return it to the OS. So if you do: use A; briefly uses 200 Kb, but frees 100. Then you do: use A; use B; (both show 200 Kbyte process growth when loaded independently). But since loading A free 100 Kbyte, B only needs 100 more and the final process size is a bit smaller. AFAIK, ps and top don't show how much memory Perl is actually using from the size it claimed. Best wishes, Tels - -- Signed on Sun Jun 26 19:59:50 2005 with key 0x93B84C15. Visit my photo gallery at http://bloodgate.com/photos/ PGP key on http://bloodgate.com/tels.asc or per email. I'm a Wei-wei-wei-wei-wei-wow-wow-wow-wow-wow-wow-wizzzaahrd... -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.4 (GNU/Linux) iQEVAwUBQr7tmncLPEOTuEwVAQGlWwf+J0Mo+dX1K7oTmDuYQbFzBR0QZtKREvJ6 qNImwBOkboy/Fnp1PXbjLc8DQGapQ7IXjgPt9xZFH3Veq+XMAgF8+3ve5qLZKYUQ +J5GTcFixpQmCjnBoQeVLNO4RWfnVN75D4XhybtBvWxNeIWQEG5qjM9Bp+L9nlU1 VNhj3V0VkSFZ+U41t2eQRoeDRhBLZf1iJUg55Zz/NIXhjggQq0ZlmtDgYH5ElPSz QIULbFfwLrFU5SKu0pjow91tfxxQYZPf5Ea0sQq4zlkk0eTrsRGvmyNUSgWqMwsd j+Rge1Vq+d4v/6buMWah7+pU8vXsWp3H1x0BgTpt0gk0bzUpa6HV1w== =cnKU -END PGP SIGNATURE-
Re: Carp vs Carp::Heavy was: Re: [PATCH] Put Carp into the Tarpit (No Carp #2 - Archive::Tar)
Dave Mitchell [EMAIL PROTECTED] wrote: :On Sun, Jun 26, 2005 at 03:16:37PM +0200, Tels wrote: : * no matter how you measure it, memory consumption goes down : :Only if the whole code body manages to avoid including Carp.pm. As soon as :any module anywhere manages to include it, even indirectly (quite likely), :its a net loss since as well as Carp.pm using memory, you have a bunch of :extra wrapper functions in each of the modules that have been 'improved', :all using memory too. I had a round of this not so long ago on the work application: it is much easier to get away with locally modifying CPAN modules to avoid such things than to modify the ones bundled with perl, which are likely to be depended on by far more things. Carp usually gets invoked for two reasons: to report a fatal error (in which case if that's the only use, that's soon enough to be loading it), and to lie about where an error/warning was found (which I'd almost always rather it didn't do). I find it great to get a stack trace for the occasional hard debugging problem, but I'd love to see an end to the habit of loading it unnecessarily, and I think the core is the only possible place to start improving that. (I hate Exporter too, but that's a rant for another day.) Hugo
Re: Carp vs Carp::Heavy was: Re: [PATCH] Put Carp into the Tarpit (No Carp #2 - Archive::Tar)
On Sun, Jun 26, 2005 at 03:16:37PM +0200, Tels wrote: So, in conclusion, I think it is worthwhile to apply these changes, because: * no matter how you measure it, memory consumption goes down * the changes are small and pure-perl only Does anybody disagree with me? Seems ok to me in the its not a bad thing and someone else will do the work sense. -- Michael G Schwern [EMAIL PROTECTED] http://www.pobox.com/~schwern Just call me 'Moron Sugar'. http://www.somethingpositive.net/sp05182002.shtml
Re: Carp vs Carp::Heavy was: Re: [PATCH] Put Carp into the Tarpit (No Carp #2 - Archive::Tar)
On Sun, 2005-06-26 at 15:16 +0200, Tels wrote: So, in conclusion, I think it is worthwhile to apply these changes, because: * no matter how you measure it, memory consumption goes down ... except for all of the duplicate subroutines in all of the patched modules. * the changes are small and pure-perl only ... and duplicate code. I have serious doubts about the efficacy of copying and pasting code to reduce memory use. -- c
