On Fri, Sep 05, 2003 at 09:34:04AM -0400, Dan Sugalski wrote:
On Fri, 5 Sep 2003, Peter Haworth wrote:
With the seen hash approach, I wouldn't expect the hash itself to take nearly as much space as the frozen structure;
My experience tracing variable graphs in Perl 5 indicates otherwise.
I think it may be worth listening to Dan on this one.
Appended patch to Storable implements an option to count the size of the seen hashes. (There seem to be two, one for objects, one for classes) It's disabled by default - compile Storable with -DDEBUGME or change the #if 0 at line 23 of Storable.xs to enable. You'll also need Devel::Size installed.
Taking a relatively complex structure, such as the POSIX exports:
$ /usr/local/bin/perl5.8.0 -Mblib -MStorable -MPOSIX -MDevel::Size -le 'package POSIX; foreach ([EMAIL PROTECTED], [EMAIL PROTECTED], \%EXPORT_TAGS) { print Devel::Size::total_size $_; print length Storable::freeze $_; print $Storable::SEENSIZE; print ""}'
reformatted, this give:
size in memory size frozen seen hash size @EXPORT 22680 5477 19068 @EXPORT_OK 1943 422 1864 %EXPORT_TAGS 23686 5963 20472
Not very surprised by this result at all. Hash table entries are going to be roughly comparable in size to the original object graph, since so many objects in Perl are really quite tiny. The issues with the alternatives are stunningly painful, though. They've been hashed out already, though. I'm less concerned by transient memory consumption than by the sabotage of threading.
Perhaps it would make sense to use a specialized data structure if memory consumption is really at the top of anybody's list of concerns. Remember the hash tables without linked lists chained off of them from undergrad CS? They're much more compact (literally a single table in memory), and most of their flukes are avoidable when entries can't be removed.
—
Gordon Henriksen [EMAIL PROTECTED]
