Re: [Pytables-users] Writing to CArray

2013-03-11 Thread Tim Burgess
The netCDF library gives me a masked array so I have to explicitly transform that into a regular numpy array.Ahh interesting.  Depending on the netCDF version the file was made with, you should be able to read the file directly from PyTables.  You could thus directly get a normal numpy array.  Th

Re: [Pytables-users] Writing to CArray

2013-03-11 Thread Anthony Scopatz
On Sun, Mar 10, 2013 at 8:47 PM, Tim Burgess wrote: > > On 08/03/2013, at 2:51 AM, Anthony Scopatz wrote: > > > Hey Tim, > > > > Awesome dataset! And neat image! > > > > As per your request, a couple of minor things I noticed were that you > probably don't need to do the sanity check each time (g

Re: [Pytables-users] Writing to CArray

2013-03-10 Thread Tim Burgess
On 08/03/2013, at 2:51 AM, Anthony Scopatz wrote: > Hey Tim, > > Awesome dataset! And neat image! > > As per your request, a couple of minor things I noticed were that you > probably don't need to do the sanity check each time (great for debugging, > but not needed always), you are using mas

Re: [Pytables-users] Writing to CArray

2013-03-07 Thread Anthony Scopatz
Hey Tim, Awesome dataset! And neat image! As per your request, a couple of minor things I noticed were that you probably don't need to do the sanity check each time (great for debugging, but not needed always), you are using masked arrays which while sometimes convenient are generally slower than

[Pytables-users] Writing to CArray

2013-03-06 Thread Tim Burgess
I'm producing a large chunked HDF5 using CArray and want to clarify that the performance I'm getting is what would normally be expected.The source data is a large annual satellite dataset - 365 days x 4320 latitiude by 8640 longitude of 32bit floats. I'm only interested in pixels of a certain quali