On Tue, Aug 11, 2009 at 11:05 AM, Robert Kern<robert.k...@gmail.com> wrote:
> On Mon, Aug 10, 2009 at 22:29, Juan Fiol<fi...@yahoo.com> wrote:
>> Hi, I am creating numpy arrays in chunks and I want to save the chunks while
>> my program creates them. I tried to use numpy.save but it failed (because it
>> is not intended to append data). I'd like to know what is, in your opinion,
>> the best way to go. I will put a few thousands every time but building up a
>> file of several Gbytes. I do not want to put into memory
>> all previous data each time. Also I cannot wait until the program finishes,
>> I must save partial results periodically. Thanks, any help will be
>> appreciated
>
> As others mentioned, PyTables is an excellent, complete solution. If
> you still want to write your own, then you can pass an open file
> object to numpy.save() in order to append. Just open it with the mode
> 'a+b' and seek to the end.
>
> f = open('myfile.npy', 'a+b')
> f.seek(0, 2)
> numpy.save(f, chunk)
> f.close()
That looks nice. What am I doing wrong?
>> x = np.array([1,2,3])
>> y = np.array([4,5,6])
>>
>> f = open('myfile.npy', 'a+b')
>> np.save(f, x)
>> f.seek(0, 2)
>> np.save(f, y)
>> f.close()
>>
>> xy = np.load('myfile.npy')
>> xy
array([1, 2, 3])
I was expecting something like array([1, 2, 3, 4, 5, 6]).
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
