On Jul 9, 2010, at 1:37 PM, David Mertens wrote:
On Fri, Jul 9, 2010 at 2:32 AM, Craig DeForest <[email protected]
> wrote:
Wow, Puneet really stirred us all up (again). Puneet, as David
said, your PDL code is slow because you are using a complicated
expression, which forced PDL to create and destroy intermediate PDLs
(every binary operation has to have a complete temporary PDL
allocated and then freed to store its result!).
Would this be a problem if we had lazy evaluation implemented?
Well, yes and no. The problem is that (from an efficiency-of-computer
standpoint) you want to grab a piece of data, do everything you are
going to do to it, and then put it back in RAM -- but the obvious
optimization for a vectorized language like PDL is to loop over all
the data for each single step. It is not obvious whether lazy
evaluation would invert the loop as you describe. Even if it did, it
would impose its own penalty -- reconstructing the sequence of
operations for each point has a computational overhead, and it would
now happen once for every point rather than once per large array of
operations. It is not clear whether that sort of generalization would
be faster than naked Perl (it would be very easy for it to end up
slower).
The Right thing to do is probably to have a separate parser that can
recognize arithmetic expressions and optimize for your particular
machine on-the-fly by cutting up the threaded expression into smaller
cache-compatible chunks, and evaluating them one at a time. That was
talked about a bit on this list sometime in (it seems) the late
Jurassic, but it does require access to the expression tree, perhaps
via a source filter, if the syntax is not to change. Source filters
do leave something to be desired since Perl is so hard to parse, and
there was a perception that Perl 6, with its cleaner built-in parser
access, would make the problem cleaner to solve. Perl 6 has been
moving at a glacial pace, though, as perlish types keep making perl5
better and everyone else is working on Pyrubhaskocamel (or whatever
the flavor of the day happens to be).
So, er, for the moment we are left with what has faced programmers
since the first FORTRAN compiler: if you really want your code to run
fast, you have to know a bit about the machine that is executing it,
and modify the program to play to its strengths rather than its
weaknesses.
_______________________________________________
Perldl mailing list
[email protected]
http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
