On Thursday 12 October 2006 23:47, Paul Johnson wrote: > Not really a beginners topic, but ... Hmm, is there a more appropriate list? I just posted here because I've been subscribed since I started learning perl, quite some time ago :)
> In general, perl ops are expensive. Assuming you have enough memory and
> are CPU bound you'll want to be reducing the number of ops. This whole
Yeah, this stuff is mostly CPU bound, I've tried to optimise the disk as much
as I know how. For reference, this is how the disk reads are done, coming
from pre-built binary files and an index stored in a hash:
sub get_user_details {
my $userid = shift;
my ($pos, $count) = @{$byuserindex{$userid}};
my $buffer;
seek $byuserfh, $pos*$recsize, 0;
read $byuserfh, $buffer, $count*$recsize;
my @res = unpack("($template)*", $buffer);
return [EMAIL PROTECTED];
}
$template='LC' (which fits my data type, being a large number and a small
number). The files these are coming from are 500Mb each (there's two of
them), and my machine has 2Gb RAM, so if something else in the script needs
to build large in-memory indexes, then there's not really enough left over
for that if I load these into RAM. If I had NZ$400 handy to buy 2Gb more, I'd
probably have less of a problem :)
> I'd probably try replacing the middle of your loop with something like:
> my ($sum, $count, $i) = (0, @$details / 2, 1);
> $sum += @$_ for grep $i ^= 1, @$details;
As I mentioned in my other post, this gives a not-insignificant speedup. I can
see I'm going to have to put some effort in to devising crafty uses of map
and grep to do this stuff well.
> You might even find newer versions of the talk floating around somewhere.
Hmm, when i started looking on the net, that was one of the first things that
came up. Also this:
http://www-128.ibm.com/developerworks/library/l-optperl.html
They're helpful, but unfortunately not largely applicable. Although, I may
need to read them closer.
--
Robin <[EMAIL PROTECTED]> JabberID: <[EMAIL PROTECTED]>
Hostes alienigeni me abduxerunt. Qui annus est?
PGP Key 0xA99CEB6D = 5957 6D23 8B16 EFAB FEF8 7175 14D3 6485 A99C EB6D
pgpd6VWSS3uLj.pgp
Description: PGP signature
