Thanks! You are right. After changing to "->", it all worked. :) Nan
From: David Mertens [mailto:[email protected]] Sent: Wednesday, October 31, 2012 1:46 PM To: Nan Yu Cc: Craig DeForest; [email protected] Subject: Re: [Perldl] how to work with large 2D matrix Try PDL->new_from_specification(dim1, dim2, ...) instead of PDL::new_from_specification(dim1, dim2, ...) Apparently it's a class method. Maybe that's why I never got the hang of it. :-) David On Wed, Oct 31, 2012 at 1:05 PM, Nan Yu <[email protected]<mailto:[email protected]>> wrote: Hi, Craig: When I use "new_from_specification", I have an error messag: "Can't call method "initialize" without a package or object reference at Basic/Core/Core.pm.PL<http://Core.pm.PL> (i.e. PDL::Core.pm) line 2508." So, it means I should install some package? Thanks! Nan From: Craig DeForest [mailto:[email protected]<mailto:[email protected]>] Sent: Wednesday, October 31, 2012 1:02 PM To: Nan Yu Cc: Craig DeForest; [email protected]<mailto:[email protected]> Subject: Re: [Perldl] how to work with large 2D matrix The most processor efficient way is $matrix = PDL::new_from_specification(4000,50000); $matrix .= -99; but you can also do $matrix = -99 + zeroes(4000,50000). The latter makes two passes through memory, the former only makes one. On Oct 31, 2012, at 11:33 AM, Nan Yu <[email protected]<mailto:[email protected]>> wrote: Hi, Craig: Thanks for helping me. With PDL::BIGPDL, things seems to be working. When I use $matrix = zeros(4000,50000), I will get the matrix with all 0 values. If I would like to make it all "-99" as default (instead of 0), is there are a quick way for me to do that? Thanks! Nan From: Craig DeForest [mailto:deforest@<mailto:deforest@>boulder.swri.edu<http://boulder.swri.edu>] Sent: Monday, October 29, 2012 11:38 PM To: Nan Yu Cc: Craig DeForest; [email protected]<mailto:[email protected]> Subject: Re: [Perldl] how to work with large 2D matrix You want to allocate the matrix all at once, then populate it. The ->glue call makes a copy of the matrix, so by the time you're up to 4000x10000, you're copying between 40MB and 320MB with each glue call. That can eat up some time. Far better do do something like: $PDL::BIGPDL = 1; $matrix = PDL::new_from_specification(byte,4000,50000); # use the smallest type you can! print "Loading rows"; for $row(0..$maxrow) { $matrix->(:,($row)) .= get_a_row(); print "." unless($row%500); } On Oct 29, 2012, at 10:21 PM, Nan Yu <[email protected]<mailto:[email protected]>> wrote: Hi, I'm working with a large 2D matrix: about 4000 x 50000. I used to use perl 2D array to do the job (when working with smaller dimensions) but the memory became an issue when the dimension grows into this size. So I decided to use PDL. After I finished my code and started testing it, I found out that just to initiate the 2D matrix takes a long time. I start with an 1D array of 4000 elements and then use "$matrix = $matrix->glue(1,$oneDarray)" within a loop (from 1 to 50000) to create the 2D matrix. The first 5000-10000 were reasonable quick (within 10 mins) then the speed slows down a lot. I'm wondering if I used a wrong way to do the initiate of the matrix. Could someone please tell me how to work with the large 2D matrix? Thanks! Nan Yu Please note that my email address has changed to [email protected]<mailto:[email protected]> ________________________________ *** The information contained in this communication may be confidential, is intended only for the use of the recipient(s) named above, and may be legally privileged. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication, or any of its contents, is strictly prohibited. If you have received this communication in error, please return it to the sender immediately and delete the original message and any copies of it. If you have any questions concerning this message, please contact the sender. *** _______________________________________________ Perldl mailing list [email protected]<mailto:[email protected]> http://mailman.jach.hawaii.edu/mailman/listinfo/perldl Please note that my email address has changed to [email protected]<mailto:[email protected]> ________________________________ *** The information contained in this communication may be confidential, is intended only for the use of the recipient(s) named above, and may be legally privileged. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication, or any of its contents, is strictly prohibited. If you have received this communication in error, please return it to the sender immediately and delete the original message and any copies of it. If you have any questions concerning this message, please contact the sender. *** Please note that my email address has changed to [email protected]<mailto:[email protected]> ________________________________ *** The information contained in this communication may be confidential, is intended only for the use of the recipient(s) named above, and may be legally privileged. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication, or any of its contents, is strictly prohibited. If you have received this communication in error, please return it to the sender immediately and delete the original message and any copies of it. If you have any questions concerning this message, please contact the sender. *** _______________________________________________ Perldl mailing list [email protected]<mailto:[email protected]> http://mailman.jach.hawaii.edu/mailman/listinfo/perldl -- "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." -- Brian Kernighan Please note that my email address has changed to [email protected] ________________________________ *** The information contained in this communication may be confidential, is intended only for the use of the recipient(s) named above, and may be legally privileged. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication, or any of its contents, is strictly prohibited. If you have received this communication in error, please return it to the sender immediately and delete the original message and any copies of it. If you have any questions concerning this message, please contact the sender. ***
_______________________________________________ Perldl mailing list [email protected] http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
