The most processor efficient way is $matrix = PDL::new_from_specification(4000,50000); $matrix .= -99;
but you can also do $matrix = -99 + zeroes(4000,50000). The latter makes two passes through memory, the former only makes one. On Oct 31, 2012, at 11:33 AM, Nan Yu <[email protected]> wrote: > Hi, Craig: > Thanks for helping me. With PDL::BIGPDL, things seems to be > working. > When I use $matrix = zeros(4000,50000), I will get the matrix > with all 0 values. > If I would like to make it all “-99” as default (instead of 0), is there are > a quick way for me to do that? > Thanks! > Nan > > From: Craig DeForest [mailto:[email protected]] > Sent: Monday, October 29, 2012 11:38 PM > To: Nan Yu > Cc: Craig DeForest; [email protected] > Subject: Re: [Perldl] how to work with large 2D matrix > > You want to allocate the matrix all at once, then populate it. The ->glue > call makes a copy of > the matrix, so by the time you're up to 4000x10000, you're copying between > 40MB and 320MB with each glue call. That can eat up some time. > > Far better do do something like: > > $PDL::BIGPDL = 1; > $matrix = PDL::new_from_specification(byte,4000,50000); # use the smallest > type you can! > print "Loading rows"; > for $row(0..$maxrow) { > $matrix->(:,($row)) .= get_a_row(); > print "." unless($row%500); > } > > > > On Oct 29, 2012, at 10:21 PM, Nan Yu <[email protected]> wrote: > > > Hi, > I’m working with a large 2D matrix: about 4000 x 50000. > I used to use perl 2D array to do the job (when working with > smaller dimensions) but the memory became an issue when the dimension grows > into this size. So I decided to use PDL. > After I finished my code and started testing it, I found out that just to > initiate the 2D matrix takes a long time. > I start with an 1D array of 4000 elements and then use “$matrix = > $matrix->glue(1,$oneDarray)” within a loop (from 1 to 50000) to create the 2D > matrix. > The first 5000-10000 were reasonable quick (within 10 mins) then the speed > slows down a lot. > I’m wondering if I used a wrong way to do the initiate of the > matrix. > Could someone please tell me how to work with the large 2D > matrix? > Thanks! > Nan Yu > > > Please note that my email address has changed to [email protected] > > *** The information contained in this communication may be confidential, is > intended only for the use of the recipient(s) named above, and may be legally > privileged. If the reader of this message is not the intended recipient, you > are hereby notified that any dissemination, distribution, or copying of this > communication, or any of its contents, is strictly prohibited. If you have > received this communication in error, please return it to the sender > immediately and delete the original message and any copies of it. If you have > any questions concerning this message, please contact the sender. *** > _______________________________________________ > Perldl mailing list > [email protected] > http://mailman.jach.hawaii.edu/mailman/listinfo/perldl > > > Please note that my email address has changed to [email protected] > > *** The information contained in this communication may be confidential, is > intended only for the use of the recipient(s) named above, and may be legally > privileged. If the reader of this message is not the intended recipient, you > are hereby notified that any dissemination, distribution, or copying of this > communication, or any of its contents, is strictly prohibited. If you have > received this communication in error, please return it to the sender > immediately and delete the original message and any copies of it. If you have > any questions concerning this message, please contact the sender. ***
_______________________________________________ Perldl mailing list [email protected] http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
