Thanks David, I'll look into it now.
Regarding the allocation/deallocation times I think that is not an issue for
me. The chunks are generated by a fortran routine that takes several minutes to
run (I am collecting a few thousand points before saving to disk). They are
approximately the same siz
Hi, I finally decided by the pytables approach because will be easier later to
work with the data. Now, I know is not the right place but may be I can get
some quick pointers. I've calculated a numpy array of about 20 columns and a
few thousands rows at each time. I'd like to append all the rows
= np.load(fi)
y1 = np.load(fi)
fi.close()
#-
--- On Tue, 8/11/09, Juan Fiol wrote:
> From: Juan Fiol
> Subject: Re: [Numpy-discussion] saving incrementally numpy arrays
> To: "Discussion of Numerical Python"
> Date: Tuesday, August 11, 2009, 8:28 PM
> Hi, th
11:05 AM,
> Robert Kern
> wrote:
> > On Mon, Aug 10, 2009 at 22:29, Juan Fiol
> wrote:
> >> Hi, I am creating numpy arrays in chunks and I
> want to save the chunks while my program creates them. I
> tried to use numpy.save but it failed (because it is not
> intend
Hi, I am creating numpy arrays in chunks and I want to save the chunks while my
program creates them. I tried to use numpy.save but it failed (because it is
not intended to append data). I'd like to know what is, in your opinion, the
best way to go. I will put a few thousands every time but buil
