Awesome. Congrats! On Jul 2, 2012 10:49 PM, "David Mertens" <[email protected]> wrote:
> On Mon, Jul 2, 2012 at 12:10 AM, David Mertens > <[email protected]>wrote: > >> Maggie - >> >> For sparse matrix support, see http://p3rl.org/PDL::CCS::Nd. This was >> one of Bryan Jurish's many modules posted late last year/early this year. I >> have not examined the internals to see how it works or what sort of hit on >> speed one has to take to use the library, but it's at least available as a >> starting point. >> >> As for cross-machine parallelization, the only solution of which I am >> aware is http://p3rl.org/PDL::Parallel::MPI. That requires an MPI setup, >> though, and I had to tweak the Makefile.PL in order to get it to run on >> UIUC's local cluster. But the underlying code worked. As a matter of fact, >> I tried to contact Darin a few years ago to see if I could take over >> maintenance of the module (because the changes to Makefile.PL are pretty >> trivial), but he never got back to me. I wonder if I should follow up on >> that... >> > > I spend some time hunting around LinkedIn and found Darin. He no longer > uses Per and was happy to transfer CPAN ownership of PDL::Parallel::MPI to > me, so now I can finally start fixing Makefile.PL! :-D > > >> David >> >> On Sun, Jul 1, 2012 at 8:18 PM, Maggie X <[email protected]> wrote: >> >>> For "big data" we need sparse matrix and maybe even more importantly, >>> the ability to parallelize across machines. PDL is great if the data can >>> fit in memory on a single machine. For big data, the assumption should be >>> that the data will NOT fit in memory and multiple machines are necessary to >>> finish processing in a reasonable amount of time. I would love to see more >>> work in that direction, if possible. >>> >>> >>> Best, >>> Maggie >>> >> >> -- >> "Debugging is twice as hard as writing the code in the first place. >> Therefore, if you write the code as cleverly as possible, you are, >> by definition, not smart enough to debug it." -- Brian Kernighan >> >> > > > -- > "Debugging is twice as hard as writing the code in the first place. > Therefore, if you write the code as cleverly as possible, you are, > by definition, not smart enough to debug it." -- Brian Kernighan > >
_______________________________________________ Perldl mailing list [email protected] http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
