I recommend the mapflex() in PDL::IO::FlexRaw instead
of the legacy mapfraw() in PDL::IO::FastRaw. You might
not have enough virtual memory in a 1GB VM to make
use of memory mapped pdls.
--Chris
On Tue, Sep 9, 2014 at 1:08 PM, Diab Jerius <djer...@cfa.harvard.edu> wrote:
> Yet another alternative is PDL::IO::FastRaw's mapfraw function. It uses
> disk space as the primary storage for the data, and automatically maps
> required chunks of data into memory via the OS's memory mapping capability.
> It would allow you to use piddles larger than your memory, but there's a
> performance hit. For example:
>
> use PDL;
> use PDL::IO::FastRaw;
>
> # create a gigantic piddle on disk
> $PDL::BIGPDL = 1;
> my $bigpdl = mapfraw( 'bigpdl', { Dims => [ 15_000, 15_000 ],
> Datatype => double,
> Creat => 1
> },
> );
>
> $bigpdl .= sequence( $bigpdl->dims );
>
>
>
>
>
>> On Tue, Sep 9, 2014 at 9:28 AM, Chris Marshall <devel.chm...@gmail.com>
>> wrote:
>> > On Tue, Sep 9, 2014 at 5:30 AM, Ronak Agrawal <ronagra...@gmail.com>
>> > wrote:
>> >>
>> >> ...snip...
>> >>
>> >> I am using Virtual Machine with 1 GB Memory and therby I am getting Out
>> >> of
>> >> Memory for $a = sequnece 15000 * 15000
>> >> Is there any way to tackle it...Currently i need to compute a Spare
>> >> matrix
>> >> of 12000 * 8000 and find A'A ( A transpose A )
>> >
>> > If you have very small memory, you'll need to do something
>> > like a disk-to-disk computation.
>>
>> Take a look a PDL::DiskCache where you could use readflex/writeflex
>> to read/write the blocks of the matrix and then write a blocked matrix
>> multiply on the PDL::DiskCache object with the IO handled transparently.
_______________________________________________
Perldl mailing list
Perldl@jach.hawaii.edu
http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
