On Fri, 8 Jul 2005, Daisuke Maki wrote:
I'm happy to announce that I just commited the first cut of the new XS
implementation of DateTime::TimeZone to CVS.
For now the only thing I ported are a few methods and the $spans structure,
which used to be a AoA. This is now a list of C structs, and are loaded on
demand.
Everything compiled and the pure Perl bits work, but it appears that the
XS version is actually _more_ memory intensive than the Perl version!
Here's a script I wrote to test this:
use strict;
use blib;
use lib './lib';
use Devel::Size;
use DateTime::TimeZone;
use Number::Format ();
print "started\n";
{my $x = <STDIN>;}
print $ENV{PERL_DATETIME_TIMEZONE_PP} ? "Pure Perl" : "XS";
print "\n";
print "loading all zones ...\n";
my @z;
my $total = 0;
for my $z ( DateTime::TimeZone::all_names )
{
push @z, DateTime::TimeZone->new( name => $z );
my $size = Devel::Size::total_size($z[-1]);
$size += Devel::Size::total_size( $z[-1]->_spans ) if
$z[-1]->can('_spans');
print "Size for $z:$size\n";
$total += $size;
}
print scalar @z . " zones loaded\n";
print "Total size: ", Number::Format::format_number( $total, 0, 0 ), "\n";
{my $x = <STDIN>;}
The "my $x = <STDIN>" is there to give me a chance to look at the
process in top or via /proc.
If I run this with the Pure Perl version, /proc/$$/status gives me:
VmSize: 19104 kB
VmLck: 0 kB
VmRSS: 17564 kB
VmData: 16216 kB
VmStk: 84 kB
VmExe: 1000 kB
VmLib: 1692 kB
VmPTE: 32 kB
With the XS version I get:
VmSize: 26864 kB
VmLck: 0 kB
VmRSS: 25280 kB
VmData: 14516 kB
VmStk: 84 kB
VmExe: 1000 kB
VmLib: 9660 kB
VmPTE: 36 kB
So it's almost 8 MB _bigger_. The total size reported by Devel::Size
is much smaller, but I think that's cause it just can't see the C
level data structure.
I also tried a variation where I constructed a DateTime with
DateTime->now() with the timezone and added 20 years to force future span
generation. It was still about 8-9MB bigger with XS.
Finally, I tried just the Pacific zones and added _200_ years. This time
the XS version was 1 MB smaller.
So what I _think_ is happening is that perhaps the XS code is allocating
_way_ too much memory initially for some reason, and then only gradually
using it up.
I thought that maybe this was the culprit:
static void
push_span(dtz_span **list, dtz_span *s)
{
if (spanset_size <= spanset_count) {
spanset_size *= 2;
Renew(__america_chicago_spans, spanset_size, dtz_span *);
}
__america_chicago_spans[spanset_count++] = s;
}
I think that "spanset_size *= 2" may be very problematic, since many
spans start out rather huge already, and will _never_ double in size
under normal use. But I changed this to "+= 8" and even changed the code
so the initial allocation is the exact size needed (before additional
spans are generated), and that didn't seem to help at all.
There is also some inefficiency in the generated XS, where some repeated
code bits could be moved out to shared files.
But I'm still rather mystified. How can the C structs possibly be bigger
than Perl's data structures?
Any C gurus want to take a look at what's in CVS now?
-dave
/*===================================================
VegGuide.Org www.BookIRead.com
Your guide to all that's veg. My book blog
===================================================*/
