Thank you very much for your help! I am just starting with PDL. I see that I have to get used to matrix perspective. Thanks again! VE [email protected] 02.09.2013
----- Folgende Nachricht wurde empfangen ----- Absender: Craig DeForest Empfänger: wanderdoc Zeit: 2013-09-02, 16:42:52 Betreff: Re: [Perldl] Run out of memory while modifying the piddle There are two big memory hogs that spring to mind. The first is that you're using (the default) double precision type instead of something shorter -- you could save a factor of 8, for example, by saying "$table = byte rint $table" up near the top there. The second, and more egregious, is that you're keeping the intermediate tables in memory. Dicing with a PDL index works differently than slicing with a scalar index -- it actually allocates a new copy of the source array, and maintains the connection by copying values between the old and new variables whenever one or the other is modified. With your iterative dicing, you're multiplying memory usage by <nrows/2>. You can avoid that by severing the new copy of the table: "$table = $table(,$rkeep)->sever". That loses the intermediate copies, but it looks like you're not using them anyway. (Dataflow links count as perl references, so they prevent the old copies from being garbage-collected). Best of luck! On Sep 1, 2013, at 10:27 AM, "VE"<[email protected]> wrote: Dear All, I played further with this example http://mailman.jach.hawaii.edu/pipermail/perldl/2013-August/008102.html and run into the following issue now. The below code should generate a 2D piddle of random 0 and 1, then take a random row, calculate a sum of it, then cut this row off the piddle and do so again until there are rows. I run "out of memory" if the initial piddle has more than 900 rows. In fact, I wanted to take a lot more :) Where is my mistake? Thank you in advance! VE The code: #!/perl use strict; use warnings; use PDL; use PDL::NiceSlice; use PDL::Math; my $size = 900; # the initial number of rows. my $table = random 10, $size; $table = rint $table; my @sums; for ( 1 .. $size) { my ( $col, $row ) = dims $table; my $row_x = $row > 1 ? int(rand($row)) : 0; my $chosen = $table->dice_axis(1, $row_x); push @sums, sum $chosen; my $rows_to_remove = pdl($row_x); # rows to be removed my $rvals = sequence($table->dim(1)); # all rows indexes my $rkeep = setops($rows_to_remove, 'XOR', $rvals); # rows to keep $table = $table(,$rkeep); } # print scalar @sums, $/; # or whatever... --- [email protected] 01.09.2013 _______________________________________________ Perldl mailing list [email protected] http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
_______________________________________________ Perldl mailing list [email protected] http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
