It sounds Ike you are losing some bytes at the end of the IDL output. Is your colleague closing his file properly?
(Mobile) On Aug 27, 2010, at 7:34 AM, P Kishor <[email protected]> wrote: > Oh... so many questions. But first, many, many thanks Craig for > sending this module. I will try it out. > > On Fri, Aug 27, 2010 at 1:03 AM, Craig DeForest <[email protected]> wrote: >> Whups - forgot to include it. Here. >> > ... >> On Aug 26, 2010, at 11:46 PM, Craig DeForest wrote: >> >>> Why not tell him to write out all the data in a .sav file, and read it in >>> using PDL::IO::IDL? > > 1. I don't see this module on the PDL web site? Is it not for public > consumption? > > 2. He is working on creating the .sav files, but hasn't done so yet. > Will test it out soon as he is ready. Part of the problem is, he is > experiencing some problems on the IDL side. If he writes out 503 x 539 > x 100 array, he gets a 108446800 bytes in his binary file. I am able > to read those files in PDL properly (although there are still data gap > problems). However, if he writes out a file with 271117 x 100 array > (that is, collapses 503 x 539 into a single dim), he gets 108445696 > bytes in his binary file. That is, of course, 1104 bytes less than how > much the file should be (27111700 * 4 bytes for floats). Any idea why > that might be happening? I know this is a long shot... why would any > one of you know what is wrong with my colleague's program and output, > but hey, you all are brainiacs and mind-readers... so... > > 3. I am trying to read up on PDL datatypes, but methinks, those docs > need to be clarified and organized better. For example, I can't seem > to find the definitions and sizes of each of the PDL datatypes. The > first hit I get just tells me what they are... byte, short, long, > float, double, etc., but not how much space they take. So, what is the > difference between a float and a double, and a long? I can wikipedia > this stuff, but it really belongs write there in PDL docs, at least, > imho. > > The piddle I created yesterday from the text file ended up being a > Double, and was, of course, double in size from that of a float! > > Nevertheless, any idea why my avgs are turning out to be nans? > > > >>> >>> On Aug 26, 2010, at 11:41 PM, P Kishor wrote: >>> >>>> I am running into a strange problem converting from data coming from >>>> IDL. A colleague is generating the IDL output as 100 values each for >>>> 271117 cells. This is what I would think of as a 271117 x 100 array of >>>> arrays. That is, an array with 271K elements, each element being a 100 >>>> element array. >>>> >>>> Except, when I read into PDL via FlexRaw, using a header of "Float >>>> 271117 100", I get the error "PDL: Couldn't read enough data from >>>> <file name>..." Strangely, when I change the header to 271116, 99, it >>>> works, but the output is all wrong... lots of gaps. I can't figure out >>>> why. >>>> >>>> Ok. To get to the bottom of this mystery, I asked him to write out all >>>> the values as a text file so I could see the darned data. Well, he >>>> didn't know how to format text output from IDL so he wrote out one >>>> value per line and gave me a text file with 27111700 (27 million >>>> lines... please control your snickering and laughter). >>>> >>>> I coaxed that file into a PDL which looks like so >>>> >>>> PDL: Double D [100,271117] >>>> >>>> The header generated by FlexRaw looks like so >>>> >>>> Double >>>> 2 >>>> 100 >>>> 271117 >>>> >>>> So, I read this piddle back into PDL. This is what it looks like >>>> >>>> print $pdl->info . "\n"; >>>>> >>>>> PDL: Float D [100,271117] >>>> >>>> Now I want to average the numbers along the 0th dimension (those are >>>> 100 years), that is, I want yearly averages for all the cells. So I do >>>> the following >>>> >>>> my $xpdl = $pdl->xchg(0, 1); >>>> my $yavgs = $xpdl->average; >>>> print $yavgs->info . "\n"; >>>>> >>>>> PDL: Float D [100] >>>> >>>> Now I print the entire avg piddle, and I get garbage. >>>> >>>> [nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan >>>> nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan >>>> nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan >>>> nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan >>>> nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan >>>> nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan] >>>> >>>> Any idea what I could be doing wrong? >>>> > > > > -- > Puneet Kishor http://www.punkish.org > Carbon Model http://carbonmodel.org > Charter Member, Open Source Geospatial Foundation http://www.osgeo.org > Science Commons Fellow, http://sciencecommons.org/about/whoweare/kishor > Nelson Institute, UW-Madison http://www.nelson.wisc.edu > ----------------------------------------------------------------------- > Assertions are politics; backing up assertions with evidence is science > ======================================================================= > _______________________________________________ Perldl mailing list [email protected] http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
