On Mon, Jul 2, 2012 at 12:10 AM, David Mertens <[email protected]>wrote:
> Maggie - > > For sparse matrix support, see http://p3rl.org/PDL::CCS::Nd. This was one > of Bryan Jurish's many modules posted late last year/early this year. I > have not examined the internals to see how it works or what sort of hit on > speed one has to take to use the library, but it's at least available as a > starting point. > > As for cross-machine parallelization, the only solution of which I am > aware is http://p3rl.org/PDL::Parallel::MPI. That requires an MPI setup, > though, and I had to tweak the Makefile.PL in order to get it to run on > UIUC's local cluster. But the underlying code worked. As a matter of fact, > I tried to contact Darin a few years ago to see if I could take over > maintenance of the module (because the changes to Makefile.PL are pretty > trivial), but he never got back to me. I wonder if I should follow up on > that... > I spend some time hunting around LinkedIn and found Darin. He no longer uses Per and was happy to transfer CPAN ownership of PDL::Parallel::MPI to me, so now I can finally start fixing Makefile.PL! :-D > David > > On Sun, Jul 1, 2012 at 8:18 PM, Maggie X <[email protected]> wrote: > >> For "big data" we need sparse matrix and maybe even more importantly, the >> ability to parallelize across machines. PDL is great if the data can fit in >> memory on a single machine. For big data, the assumption should be that the >> data will NOT fit in memory and multiple machines are necessary to finish >> processing in a reasonable amount of time. I would love to see more work in >> that direction, if possible. >> >> >> Best, >> Maggie >> > > -- > "Debugging is twice as hard as writing the code in the first place. > Therefore, if you write the code as cleverly as possible, you are, > by definition, not smart enough to debug it." -- Brian Kernighan > > -- "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." -- Brian Kernighan
_______________________________________________ Perldl mailing list [email protected] http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
