Depending on your granularity, storing PDLs as either blobs or as files whose names are stored in the DB could work for you.
Also check out PDL::DiskCache, which is a tied array object for quasi- transparently keeping PDLs on disk. (DiskCache uses FITS files by default but can use Storable with minimal effort). On Jun 22, 2010, at 2:49 PM, P Kishor wrote: > On Tue, Jun 22, 2010 at 3:32 PM, David Mertens <[email protected] > > wrote: >> >> >> On Tue, Jun 22, 2010 at 3:10 PM, P Kishor <[email protected]> >> wrote: >>> >>> On Tue, Jun 22, 2010 at 3:01 PM, Craig DeForest >>> <[email protected]> wrote: >>>> Looks like you're storing integer data in a double-precision >>>> floating-point >>>> PDL. (default data type is 'double'). Try using type >>>> declarations in >>>> your >>>> constructors -- e.g. "ones(short, $x)->dummy * pdl(short, @dm)" >>>> >>> >>> ahhh... >>> >>> -rw-r--r-- 1 punkish punkish 1289616 Jun 22 14:42 bin.dat >>> -rw-r--r-- 1 punkish punkish 1280000 Jun 22 15:07 pdl.dat >>> -rw-r--r-- 1 punkish punkish 14 Jun 22 15:07 pdl.dat.hdr >>> >>> >>> So much better. So, a modest technical question -- is a piddle, >>> internally, just like a packed value with some magic dust >>> sprinkled on >>> it? >> >> Yes. Actually quite a bit of magic dust, and some lint. You >> shouldn't go >> around creating thousands of 5-element piddles because they would >> probably >> be more efficiently stored as perl arrays. However, if you cat'd >> those >> piddles together, then you'd be in business. >> > > > Thanks David. For now, I am trying different things out, and, > hopefully, learning how to "PDL walk" in the process. Actually, even > my little-trainer-wheels test code is actually creating a "few hundred > X few hundred piddle" of 6 element piddles. Since I potentially need > to use PDL to work on those data once I take them out and off of the > disk, I am guessing PDL will be way more efficient than working with > array of arrays and Storable. RIght now I am researching potential > alternatives to db storage as well as piddle+db storage... just trying > out different approaches. > > >> In terms of storage, however, the stuff stored to the disk is just >> the data. >> >> David >> >> -- >> Sent via my carrier pigeon. >> > > > > -- > Puneet Kishor http://www.punkish.org > Carbon Model http://carbonmodel.org > Charter Member, Open Source Geospatial Foundation http://www.osgeo.org > Science Commons Fellow, http://sciencecommons.org/about/whoweare/ > kishor > Nelson Institute, UW-Madison http://www.nelson.wisc.edu > ----------------------------------------------------------------------- > Assertions are politics; backing up assertions with evidence is > science > = > ====================================================================== > > _______________________________________________ > Perldl mailing list > [email protected] > http://mailman.jach.hawaii.edu/mailman/listinfo/perldl > _______________________________________________ Perldl mailing list [email protected] http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
