Re: [Numpy-discussion] ragged array implimentation

2011-03-12 Thread Christopher Barker
Francesc Alted wrote: Just a C pointer to a malloc'ed area (it cannot be an ndarray, as the data area might be compressed). got it. Ok. Thanks. The code is really simple, and that's great. However, carray is quite more sophisticated, as it supports not only enlarging arrays, but also

Re: [Numpy-discussion] ragged array implimentation

2011-03-11 Thread Francesc Alted
A Thursday 10 March 2011 22:25:27 Christopher Barker escrigué: On 3/10/11 12:01 PM, Francesc Alted wrote: 3) when the extra space is used up, it re-allocates the entire array, with some more extra room again, carray works exactly the same: the extra room is just a new chunk does it

Re: [Numpy-discussion] ragged array implimentation

2011-03-10 Thread Christopher Barker
On 3/7/11 5:51 PM, Sturla Molden wrote: Den 07.03.2011 18:28, skrev Christopher Barker: 1, 2, 3, 4 5, 6 7, 8, 9, 10, 11, 12 13, 14, 15 ... A ragged array, as implemented in C++, Java or C# is just an array of arrays (or 'a pointer to an array of pointers'). Sure, but as a rule I don't

Re: [Numpy-discussion] ragged array implimentation

2011-03-10 Thread Francesc Alted
A Thursday 10 March 2011 18:05:11 Christopher Barker escrigué: NOTE: this looks like it could use a growable numpy array, much like one I've written before -- maybe it's time to revive that project and use it here, fixing some performance issues while I'm at it. A growable array would be

Re: [Numpy-discussion] ragged array implimentation

2011-03-10 Thread Christopher Barker
On 3/10/11 9:51 AM, Francesc Alted wrote: A Thursday 10 March 2011 18:05:11 Christopher Barker escrigué: NOTE: this looks like it could use a growable numpy array, much like one I've written before -- maybe it's time to revive that project and use it here, fixing some performance issues while

Re: [Numpy-discussion] ragged array implimentation

2011-03-10 Thread Christopher Barker
On 3/10/11 11:29 AM, Christopher Barker wrote: By the way, it would be great to have a growable array that could be efficiently used in Cython code -- so you could accumulate a bunch of native C datatype data easily. I just noticed that carray is written in Cython, so that part should be

Re: [Numpy-discussion] ragged array implimentation

2011-03-10 Thread Francesc Alted
A Thursday 10 March 2011 20:29:00 Christopher Barker escrigué: I don't think so -- my approach is a lot simpler that I think carray is: it acts much like a python list: 1) it pre-allocates extra space when an array is created. same for carray 2) when you append, it uses that extra space

[Numpy-discussion] ragged array implimentation

2011-03-07 Thread Christopher Barker
Hi folks, I'm setting out to write some code to access and work with ragged arrays stored in netcdf files. It dawned on me that ragged arrays are not all that uncommon, so I'm wondering if any of you have any code you've developed that I could learn-from borrow from, etc. note that when I say

Re: [Numpy-discussion] ragged array implimentation

2011-03-07 Thread Francesc Alted
A Monday 07 March 2011 18:28:11 Christopher Barker escrigué: Hi folks, I'm setting out to write some code to access and work with ragged arrays stored in netcdf files. It dawned on me that ragged arrays are not all that uncommon, so I'm wondering if any of you have any code you've developed

Re: [Numpy-discussion] ragged array implimentation

2011-03-07 Thread Jeff Whitaker
On 3/7/11 10:28 AM, Christopher Barker wrote: Hi folks, I'm setting out to write some code to access and work with ragged arrays stored in netcdf files. It dawned on me that ragged arrays are not all that uncommon, so I'm wondering if any of you have any code you've developed that I could

Re: [Numpy-discussion] ragged array implimentation

2011-03-07 Thread John Salvatier
@Jeff I need to work with ragged arrays too. Are object arrays of 1d numpy arrays slower than lists of 1d numpy arrays? @ Christopher I'd be interested in hearing if you come up with any better solutions. On Mon, Mar 7, 2011 at 9:37 AM, Jeff Whitaker jsw...@fastmail.fm wrote: On 3/7/11 10:28

Re: [Numpy-discussion] ragged array implimentation

2011-03-07 Thread Christopher Barker
On 3/7/11 9:33 AM, Francesc Alted wrote: A Monday 07 March 2011 18:28:11 Christopher Barker escrigué: I'm setting out to write some code to access and work with ragged arrays stored in netcdf files. It dawned on me that ragged arrays are not all that uncommon, so I'm wondering if any of you

Re: [Numpy-discussion] ragged array implimentation

2011-03-07 Thread Jeff Whitaker
On 3/7/11 11:42 AM, Christopher Barker wrote: On 3/7/11 9:33 AM, Francesc Alted wrote: A Monday 07 March 2011 18:28:11 Christopher Barker escrigué: I'm setting out to write some code to access and work with ragged arrays stored in netcdf files. It dawned on me that ragged arrays are not all

Re: [Numpy-discussion] ragged array implimentation

2011-03-07 Thread Francesc Alted
A Monday 07 March 2011 19:42:00 Christopher Barker escrigué: But now that you've entered the conversation, does HDF and/or pytables have a standard way of dealing with this? Well, I don't think there is such a 'standard' way for dealing with ragged arrays, but yes, pytables has support for

Re: [Numpy-discussion] ragged array implimentation

2011-03-07 Thread Christopher Barker
On 3/7/11 11:18 AM, Francesc Alted wrote: Well, I don't think there is such a 'standard' way for dealing with ragged arrays, but yes, pytables has support for them. Creating them is easy: # Create a VLArray: fileh = tables.openFile('vlarray1.h5', mode='w') vlarray =

Re: [Numpy-discussion] ragged array implimentation

2011-03-07 Thread Anne Archibald
On 7 March 2011 15:29, Christopher Barker chris.bar...@noaa.gov wrote: On 3/7/11 11:18 AM, Francesc Alted wrote: but, instead of returning a numpy array of 'object' elements, plain python lists are returned instead. which gives you the append option -- I can see how that would be usefull.

Re: [Numpy-discussion] ragged array implimentation

2011-03-07 Thread Sturla Molden
Den 07.03.2011 18:28, skrev Christopher Barker: 1, 2, 3, 4 5, 6 7, 8, 9, 10, 11, 12 13, 14, 15 ... In my case, these will only be 2-d, though I suppose one could have a n-d version where the last dimension was ragged (or any dimension, I suppose, though I'm having trouble wrapping my