Re: [Numpy-discussion] pickling/unpickling numpy.void and numpy.record for multiprocessing

2010-02-26 Thread Martin Spacek
On 2010-02-26 15:26, Pauli Virtanen wrote: > No, the unpickled void scalar will own its data. The problem is that > either the data is not saved correctly (unlikely), or it is unpickled > incorrectly. > > The relevant code path to look at is multiarraymodule:array_scalar -> > scalarapi.c:PyArray_Sc

Re: [Numpy-discussion] pickling/unpickling numpy.void and numpy.record for multiprocessing

2010-02-26 Thread Martin Spacek
On 2010-02-26 15:02, Robert Kern wrote: >> Is this a known limitation? > > Nope. New bug! Thanks! Good. I'm not crazy after all :) > Pickling of complete arrays works. A quick workaround would be to send > rank-0 scalars: > >Pool.map(map(np.asarray, x)) > > Or just tuples: > >Pool.map(map

Re: [Numpy-discussion] pickling/unpickling numpy.void and numpy.record for multiprocessing

2010-02-26 Thread Pauli Virtanen
pe, 2010-02-26 kello 14:41 -0800, Martin Spacek kirjoitti: [clip: pickling/unpickling numpy.void scalar objects] > I suppose numpy.void is as it suggests, a pointer to a specific place in > memory. > I'm just surprised that this pointer isn't dereferenced before pickling Or is > it? I'm not skil

Re: [Numpy-discussion] pickling/unpickling numpy.void and numpy.record for multiprocessing

2010-02-26 Thread Robert Kern
On Fri, Feb 26, 2010 at 16:41, Martin Spacek wrote: > I have a 1D structured ndarray with several different fields in the dtype. I'm > using multiprocessing.Pool.map() to iterate over this structured ndarray, > passing one entry (of type numpy.void) at a time to the function to be called > by > e

[Numpy-discussion] pickling/unpickling numpy.void and numpy.record for multiprocessing

2010-02-26 Thread Martin Spacek
I have a 1D structured ndarray with several different fields in the dtype. I'm using multiprocessing.Pool.map() to iterate over this structured ndarray, passing one entry (of type numpy.void) at a time to the function to be called by each process in the pool. After much confusion about why this