Re: [Numpy-discussion] subclassing array in c

[mark florisson]

On 30 March 2012 21:40, mark florisson  wrote:
> On 30 March 2012 21:38, mark florisson  wrote:
>> On 30 March 2012 19:53, Chris Barker  wrote:
>>> On Fri, Mar 30, 2012 at 10:57 AM, mark florisson
>>>  wrote:
 Although the segfault was caused by a bug in NumPy, you should
 probably also consider using Cython, which can make a lot of this pain
 and boring stuff go away.
>>>
>>> Is there a good demo/sample somewhere of an ndarray subclass in Cython?
>>>
>>> Some quick googling turned up a number of people asking about it, but
>>> I didn't find (quickly) a wiki page or demo about it.
>>>
>>> -Chris
>>>
>>> --
>>>
>>> Christopher Barker, Ph.D.
>>> Oceanographer
>>>
>>> Emergency Response Division
>>> NOAA/NOS/OR&R            (206) 526-6959   voice
>>> 7600 Sand Point Way NE   (206) 526-6329   fax
>>> Seattle, WA  98115       (206) 526-6317   main reception
>>>
>>> chris.bar...@noaa.gov
>>> ___
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion@scipy.org
>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>> It's not common to do, I tried the following:
>>
>> cimport numpy
>>
>> cdef extern from "Python.h":
>>    ctypedef struct PyTypeObject:
>>        void *tp_alloc
>>
>>    object PyType_GenericAlloc(PyTypeObject *type, Py_ssize_t nitems)
>>
>> cdef myalloc(PyTypeObject *type, Py_ssize_t nitems):
>>    print "allocating"
>>    return PyType_GenericAlloc(type, nitems)
>>
>> cdef class MyClass(numpy.ndarray) :
>>    cdef int array[1000]
>>
>> ( MyClass).tp_alloc =  myalloc # This works
>> around the NumPy bug
>> cdef MyClass obj = MyClass((10,))
>> obj.array[99] = 20
>>
>> The array attribute is quite large here to cause a segfault if our
>> trick to replace the tp_alloc isn't working. It's kind of a hack, but
>> the only alternative is to use composition instead.
>
> (So remove the array attribute, it's just for demonstration :)

And you can also directly assign PyType_GenericAlloc instead of
writing your own (again, demonstration to see if it works).
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] subclassing array in c

[mark florisson]

On 30 March 2012 21:38, mark florisson  wrote:
> On 30 March 2012 19:53, Chris Barker  wrote:
>> On Fri, Mar 30, 2012 at 10:57 AM, mark florisson
>>  wrote:
>>> Although the segfault was caused by a bug in NumPy, you should
>>> probably also consider using Cython, which can make a lot of this pain
>>> and boring stuff go away.
>>
>> Is there a good demo/sample somewhere of an ndarray subclass in Cython?
>>
>> Some quick googling turned up a number of people asking about it, but
>> I didn't find (quickly) a wiki page or demo about it.
>>
>> -Chris
>>
>> --
>>
>> Christopher Barker, Ph.D.
>> Oceanographer
>>
>> Emergency Response Division
>> NOAA/NOS/OR&R            (206) 526-6959   voice
>> 7600 Sand Point Way NE   (206) 526-6329   fax
>> Seattle, WA  98115       (206) 526-6317   main reception
>>
>> chris.bar...@noaa.gov
>> ___
>> NumPy-Discussion mailing list
>> NumPy-Discussion@scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
> It's not common to do, I tried the following:
>
> cimport numpy
>
> cdef extern from "Python.h":
>    ctypedef struct PyTypeObject:
>        void *tp_alloc
>
>    object PyType_GenericAlloc(PyTypeObject *type, Py_ssize_t nitems)
>
> cdef myalloc(PyTypeObject *type, Py_ssize_t nitems):
>    print "allocating"
>    return PyType_GenericAlloc(type, nitems)
>
> cdef class MyClass(numpy.ndarray) :
>    cdef int array[1000]
>
> ( MyClass).tp_alloc =  myalloc # This works
> around the NumPy bug
> cdef MyClass obj = MyClass((10,))
> obj.array[99] = 20
>
> The array attribute is quite large here to cause a segfault if our
> trick to replace the tp_alloc isn't working. It's kind of a hack, but
> the only alternative is to use composition instead.

(So remove the array attribute, it's just for demonstration :)
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] subclassing array in c

[mark florisson]

On 30 March 2012 19:53, Chris Barker  wrote:
> On Fri, Mar 30, 2012 at 10:57 AM, mark florisson
>  wrote:
>> Although the segfault was caused by a bug in NumPy, you should
>> probably also consider using Cython, which can make a lot of this pain
>> and boring stuff go away.
>
> Is there a good demo/sample somewhere of an ndarray subclass in Cython?
>
> Some quick googling turned up a number of people asking about it, but
> I didn't find (quickly) a wiki page or demo about it.
>
> -Chris
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R            (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
>
> chris.bar...@noaa.gov
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion

It's not common to do, I tried the following:

cimport numpy

cdef extern from "Python.h":
ctypedef struct PyTypeObject:
void *tp_alloc

object PyType_GenericAlloc(PyTypeObject *type, Py_ssize_t nitems)

cdef myalloc(PyTypeObject *type, Py_ssize_t nitems):
print "allocating"
return PyType_GenericAlloc(type, nitems)

cdef class MyClass(numpy.ndarray) :
cdef int array[1000]

( MyClass).tp_alloc =  myalloc # This works
around the NumPy bug
cdef MyClass obj = MyClass((10,))
obj.array[99] = 20

The array attribute is quite large here to cause a segfault if our
trick to replace the tp_alloc isn't working. It's kind of a hack, but
the only alternative is to use composition instead.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] subclassing array in c

[Chris Barker]

On Fri, Mar 30, 2012 at 10:57 AM, mark florisson
 wrote:
> Although the segfault was caused by a bug in NumPy, you should
> probably also consider using Cython, which can make a lot of this pain
> and boring stuff go away.

Is there a good demo/sample somewhere of an ndarray subclass in Cython?

Some quick googling turned up a number of people asking about it, but
I didn't find (quickly) a wiki page or demo about it.

-Chris

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

chris.bar...@noaa.gov
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] subclassing array in c

[mark florisson]

On 29 March 2012 09:07, Christoph Gohle  wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
>
> Am 08.03.2012 um 20:39 schrieb Pauli Virtanen:
>
>> 08.03.2012 17:37, Christoph Gohle kirjoitti:
>>> thanks for testing. I have now tried on different platforms. I get
>>> all kinds of crashes on os x (now with numpy 1.6.1) and windows
>>> with numpy 1.6.0. On Ubuntu with numpy 1.3.0 I get a hughe memory
>>> leak...
>>>
>>> Any hints would be welcome.
>>
>> The type object inherits `tp_alloc` from Numpy. This routine always
>> allocates memory of size NPY_SIZEOF_PYARRAYOBJECT for the
>> PyArrayObject. Therefore, the write to new->unit in your
>> __array_finalize__ goes to unallocated memory.
> do you think, I should submit a bugreport?
>
> Cheers,
> Christoph
>

Although the segfault was caused by a bug in NumPy, you should
probably also consider using Cython, which can make a lot of this pain
and boring stuff go away.

> -BEGIN PGP SIGNATURE-
> Version: GnuPG/MacGPG2 v2.0.14 (Darwin)
>
> iEYEARECAAYFAk90GEQACgkQLYu25rCEIztHGwCgi4asYpscBFYp6yfYAc1wioZG
> 9EAAoIZft1WOiiAj+cyJi+RVuT2U7BzF
> =e9cn
> -END PGP SIGNATURE-
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] subclassing array in c

[Christoph Gohle]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1


Am 08.03.2012 um 20:39 schrieb Pauli Virtanen:

> 08.03.2012 17:37, Christoph Gohle kirjoitti:
>> thanks for testing. I have now tried on different platforms. I get
>> all kinds of crashes on os x (now with numpy 1.6.1) and windows
>> with numpy 1.6.0. On Ubuntu with numpy 1.3.0 I get a hughe memory
>> leak...
>> 
>> Any hints would be welcome.
> 
> The type object inherits `tp_alloc` from Numpy. This routine always
> allocates memory of size NPY_SIZEOF_PYARRAYOBJECT for the
> PyArrayObject. Therefore, the write to new->unit in your
> __array_finalize__ goes to unallocated memory.
do you think, I should submit a bugreport?

Cheers,
Christoph


-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.14 (Darwin)

iEYEARECAAYFAk90GEQACgkQLYu25rCEIztHGwCgi4asYpscBFYp6yfYAc1wioZG
9EAAoIZft1WOiiAj+cyJi+RVuT2U7BzF
=e9cn
-END PGP SIGNATURE-
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] subclassing array in c

[Christoph Gohle]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1


Am 08.03.2012 um 20:39 schrieb Pauli Virtanen:

> 08.03.2012 17:37, Christoph Gohle kirjoitti:
>> thanks for testing. I have now tried on different platforms. I get
>> all kinds of crashes on os x (now with numpy 1.6.1) and windows
>> with numpy 1.6.0. On Ubuntu with numpy 1.3.0 I get a hughe memory
>> leak...
>> 
>> Any hints would be welcome.
> 
> The type object inherits `tp_alloc` from Numpy. This routine always
> allocates memory of size NPY_SIZEOF_PYARRAYOBJECT for the
> PyArrayObject. Therefore, the write to new->unit in your
> __array_finalize__ goes to unallocated memory.
> 
> This is probably a bug in Numpy --- arrayobject.c:array_alloc should
> respect the size specified by the subtype.
> 
> A workaround is probably to specify a suitable tp_alloc routine yourself:
> 
>   PyType_GenericAlloc,/* tp_alloc */
>unitArray_new,  /* tp_new */
>_PyObject_Del   /* tp_free */
Now I have been playing around with this configuration and can't seem to get 
rid of the memory leak I was talking about earlier. Reference counting seems to 
be fine as far as I can tell. I am using the following tp_dealloc:

static void unitArray_dealloc(UnitArrayObject *obj) {
Py_XDECREF(obj->unit);
obj->unit = NULL;
if (obj->base.ob_type->tp_base != NULL) {
//call base dealloc
obj->base.ob_type->tp_base->tp_dealloc((PyObject*)obj);
}
}

I was also trying to provide my own tp_alloc like below and using the inherited 
tp_free. This also crashes with a segmentation fault upon freeing saying that 
the memory block was not allocated. Does this make sense?

static PyObject *
unitArray_alloc(PyTypeObject *type, Py_ssize_t NPY_UNUSED(nitems))
{
PyObject *obj;
/* nitems will always be 0 */
obj = (PyObject *)PyObject_Malloc(type->tp_basicsize);
PyObject_Init(obj, type);
return obj;
}

Cheers,
Christoph

-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.14 (Darwin)

iEYEARECAAYFAk90FvkACgkQLYu25rCEIzvd+gCeNRgsv44g8kFJut5OQNXvK9zv
XckAoKWEjj3A34i4H+POOU/EIzzSU1EX
=kJPT
-END PGP SIGNATURE-
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] subclassing array in c

[Pauli Virtanen]

09.03.2012 08:00, Christoph Gohle kirjoitti:
> I don't want to look as if I want other people do my work, 
> so I would like to ask if there is a simple way of tracing
> memory leaks (without recompiling the python interpreter)?

The easiest way probably is to compile Numpy with debug symbols on, set
breakpoints to relevant routines in your class and inside Numpy, and
then step through the execution with (c)gdb.

Automatic memory leak detection tools maybe don't help so much here, as
the problem is likely in wrong reference counting.

-- 
Pauli Virtanen

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] subclassing array in c

[Francesc Alted]

Sure.  Check the memcheck tool of Valgrind:

 http://valgrind.org/info/tools.html#memcheck

It is a really amazing tool.

Francesc

On Mar 8, 2012, at 11:00 PM, Christoph Gohle wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> Hi again,
> 
> I don't want to look as if I want other people do my work, so I would like to 
> ask if there is a simple way of tracing memory leaks (without recompiling the 
> python interpreter)?
> 
> Cheers,
> Christoph
> Am 09.03.2012 um 01:22 schrieb Christoph Gohle:
> 
>> -BEGIN PGP SIGNED MESSAGE-
>> Hash: SHA1
>> 
>> Hi
>> Am 08.03.2012 um 20:39 schrieb Pauli Virtanen:
>> 
>>> 08.03.2012 17:37, Christoph Gohle kirjoitti:
 thanks for testing. I have now tried on different platforms. I get
 all kinds of crashes on os x (now with numpy 1.6.1) and windows
 with numpy 1.6.0. On Ubuntu with numpy 1.3.0 I get a hughe memory
 leak...
 
 Any hints would be welcome.
>>> 
>>> The type object inherits `tp_alloc` from Numpy. This routine always
>>> allocates memory of size NPY_SIZEOF_PYARRAYOBJECT for the
>>> PyArrayObject. Therefore, the write to new->unit in your
>>> __array_finalize__ goes to unallocated memory.
>>> 
>>> This is probably a bug in Numpy --- arrayobject.c:array_alloc should
>>> respect the size specified by the subtype.
>>> 
>>> A workaround is probably to specify a suitable tp_alloc routine yourself:
>>> 
>>> PyType_GenericAlloc,/* tp_alloc */
>>>  unitArray_new,  /* tp_new */
>>>  _PyObject_Del   /* tp_free */
>>> 
>> OK, I did that. And I get no more segfaults as far as I can tell. But there 
>> is still a memory leak:
>> 
>> In [1]: import spampub
>> 
>> In [2]: a=[spampub.UnitArray(i,{'s':i}) for i in xrange(10)]
>> 
>> In [3]: del a
>> 
>> after the last two statements, python uses ~60MB more memory than before.
>> 
>> Thanks for your help
>> Christoph
>>> -- 
>>> Pauli Virtanen
>>> 
>>> ___
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion@scipy.org
>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>> 
>> 
>> Christoph Gohle
>> - --
>> Max-Planck-Institut für Quantenoptik
>> Abteilung Quantenvielteilchensysteme
>> Hans-Kopfermann-Strasse 1
>> 85748 Garching
>> 
>> christoph.go...@mpq.mpg.de
>> tel: +49 89 32905 283
>> fax: +49 89 32905 313
>> 
>> 
>> 
>> -BEGIN PGP SIGNATURE-
>> Version: GnuPG/MacGPG2 v2.0.14 (Darwin)
>> 
>> iEYEARECAAYFAk9ZTVgACgkQLYu25rCEIztbcwCfcyeQ+FtKTOwFUGbleX/CrjPi
>> nZcAnj86kejcAO45YbX+I+rxhU9kq4PU
>> =KGdt
>> -END PGP SIGNATURE-
>> ___
>> NumPy-Discussion mailing list
>> NumPy-Discussion@scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 
> 
> Christoph Gohle
> - --
> Max-Planck-Institut für Quantenoptik
> Abteilung Quantenvielteilchensysteme
> Hans-Kopfermann-Strasse 1
> 85748 Garching
> 
> christoph.go...@mpq.mpg.de
> tel: +49 89 32905 283
> fax: +49 89 32905 313
> 
> 
> 
> -BEGIN PGP SIGNATURE-
> Version: GnuPG/MacGPG2 v2.0.14 (Darwin)
> 
> iEYEARECAAYFAk9ZqnQACgkQLYu25rCEIzthWACgi0dYy2nh83w57Ho8emkvJZ8z
> KrkAnistJfaU29tzul8nrJBYsrdmksJk
> =Iyr4
> -END PGP SIGNATURE-
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion

-- Francesc Alted



___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] subclassing array in c

[Christoph Gohle]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi again,

I don't want to look as if I want other people do my work, so I would like to 
ask if there is a simple way of tracing memory leaks (without recompiling the 
python interpreter)?

Cheers,
Christoph
Am 09.03.2012 um 01:22 schrieb Christoph Gohle:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> Hi
> Am 08.03.2012 um 20:39 schrieb Pauli Virtanen:
> 
>> 08.03.2012 17:37, Christoph Gohle kirjoitti:
>>> thanks for testing. I have now tried on different platforms. I get
>>> all kinds of crashes on os x (now with numpy 1.6.1) and windows
>>> with numpy 1.6.0. On Ubuntu with numpy 1.3.0 I get a hughe memory
>>> leak...
>>> 
>>> Any hints would be welcome.
>> 
>> The type object inherits `tp_alloc` from Numpy. This routine always
>> allocates memory of size NPY_SIZEOF_PYARRAYOBJECT for the
>> PyArrayObject. Therefore, the write to new->unit in your
>> __array_finalize__ goes to unallocated memory.
>> 
>> This is probably a bug in Numpy --- arrayobject.c:array_alloc should
>> respect the size specified by the subtype.
>> 
>> A workaround is probably to specify a suitable tp_alloc routine yourself:
>> 
>>  PyType_GenericAlloc,/* tp_alloc */
>>   unitArray_new,  /* tp_new */
>>   _PyObject_Del   /* tp_free */
>> 
> OK, I did that. And I get no more segfaults as far as I can tell. But there 
> is still a memory leak:
> 
> In [1]: import spampub
> 
> In [2]: a=[spampub.UnitArray(i,{'s':i}) for i in xrange(10)]
> 
> In [3]: del a
> 
> after the last two statements, python uses ~60MB more memory than before.
> 
> Thanks for your help
> Christoph
>> -- 
>> Pauli Virtanen
>> 
>> ___
>> NumPy-Discussion mailing list
>> NumPy-Discussion@scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 
> 
> Christoph Gohle
> - --
> Max-Planck-Institut für Quantenoptik
> Abteilung Quantenvielteilchensysteme
> Hans-Kopfermann-Strasse 1
> 85748 Garching
> 
> christoph.go...@mpq.mpg.de
> tel: +49 89 32905 283
> fax: +49 89 32905 313
> 
> 
> 
> -BEGIN PGP SIGNATURE-
> Version: GnuPG/MacGPG2 v2.0.14 (Darwin)
> 
> iEYEARECAAYFAk9ZTVgACgkQLYu25rCEIztbcwCfcyeQ+FtKTOwFUGbleX/CrjPi
> nZcAnj86kejcAO45YbX+I+rxhU9kq4PU
> =KGdt
> -END PGP SIGNATURE-
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion


Christoph Gohle
- --
Max-Planck-Institut für Quantenoptik
Abteilung Quantenvielteilchensysteme
Hans-Kopfermann-Strasse 1
85748 Garching

christoph.go...@mpq.mpg.de
tel: +49 89 32905 283
fax: +49 89 32905 313



-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.14 (Darwin)

iEYEARECAAYFAk9ZqnQACgkQLYu25rCEIzthWACgi0dYy2nh83w57Ho8emkvJZ8z
KrkAnistJfaU29tzul8nrJBYsrdmksJk
=Iyr4
-END PGP SIGNATURE-
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] subclassing array in c

[Christoph Gohle]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi
Am 08.03.2012 um 20:39 schrieb Pauli Virtanen:

> 08.03.2012 17:37, Christoph Gohle kirjoitti:
>> thanks for testing. I have now tried on different platforms. I get
>> all kinds of crashes on os x (now with numpy 1.6.1) and windows
>> with numpy 1.6.0. On Ubuntu with numpy 1.3.0 I get a hughe memory
>> leak...
>> 
>> Any hints would be welcome.
> 
> The type object inherits `tp_alloc` from Numpy. This routine always
> allocates memory of size NPY_SIZEOF_PYARRAYOBJECT for the
> PyArrayObject. Therefore, the write to new->unit in your
> __array_finalize__ goes to unallocated memory.
> 
> This is probably a bug in Numpy --- arrayobject.c:array_alloc should
> respect the size specified by the subtype.
> 
> A workaround is probably to specify a suitable tp_alloc routine yourself:
> 
>   PyType_GenericAlloc,/* tp_alloc */
>unitArray_new,  /* tp_new */
>_PyObject_Del   /* tp_free */
> 
OK, I did that. And I get no more segfaults as far as I can tell. But there is 
still a memory leak:

In [1]: import spampub

In [2]: a=[spampub.UnitArray(i,{'s':i}) for i in xrange(10)]

In [3]: del a

after the last two statements, python uses ~60MB more memory than before.

Thanks for your help
Christoph
> -- 
> Pauli Virtanen
> 
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion


Christoph Gohle
- --
Max-Planck-Institut für Quantenoptik
Abteilung Quantenvielteilchensysteme
Hans-Kopfermann-Strasse 1
85748 Garching

christoph.go...@mpq.mpg.de
tel: +49 89 32905 283
fax: +49 89 32905 313



-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.14 (Darwin)

iEYEARECAAYFAk9ZTVgACgkQLYu25rCEIztbcwCfcyeQ+FtKTOwFUGbleX/CrjPi
nZcAnj86kejcAO45YbX+I+rxhU9kq4PU
=KGdt
-END PGP SIGNATURE-
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] subclassing array in c

[Pauli Virtanen]

08.03.2012 17:37, Christoph Gohle kirjoitti:
> thanks for testing. I have now tried on different platforms. I get
> all kinds of crashes on os x (now with numpy 1.6.1) and windows
> with numpy 1.6.0. On Ubuntu with numpy 1.3.0 I get a hughe memory
> leak...
> 
> Any hints would be welcome.

The type object inherits `tp_alloc` from Numpy. This routine always
allocates memory of size NPY_SIZEOF_PYARRAYOBJECT for the
PyArrayObject. Therefore, the write to new->unit in your
__array_finalize__ goes to unallocated memory.

This is probably a bug in Numpy --- arrayobject.c:array_alloc should
respect the size specified by the subtype.

A workaround is probably to specify a suitable tp_alloc routine yourself:

   PyType_GenericAlloc,/* tp_alloc */
unitArray_new,  /* tp_new */
_PyObject_Del   /* tp_free */

-- 
Pauli Virtanen

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] subclassing array in c

[Christoph Gohle]

Dear Val,

thanks for testing. I have now tried on different platforms. I get all kinds of 
crashes on os x (now with numpy 1.6.1) and windows with numpy 1.6.0. On Ubuntu 
with numpy 1.3.0 I get a hughe memory leak...

Any hints would be welcome.

Thanks,
Christoph
Am 08.03.2012 um 09:08 schrieb Val Kalatsky:

> Hi Christoph,
> 
> I've just tried 
> a=[spampub.UnitArray(i,{'s':i}) for i in xrange(1000)] 
> and everything looks fine on my side.
> Probably my test environment is too different to give comparable results:
> 
> In [3]: call(["uname", "-a"])
> Linux ubuntu 2.6.35-22-generic #33-Ubuntu SMP Sun Sep 19 20:32:27 UTC 2010 
> x86_64 GNU/Linux
> In [4]: np.__version__
> Out[4]: '1.5.1'
> 
> I am afraid I will not be able to help testing it on super-nova versions of 
> numpy.
> Cheers
> Val
> 
> 
> On Thu, Mar 8, 2012 at 1:49 AM, Christoph Gohle  
> wrote:
> Dear Val,
> 
> I agree that more detail is needed. Sorry for that it was late yesterday.
> 
>  I am running Python 2.6.1, numpy development branch 
> (numpy-2.0.0.dev_20101104-py2.6-macosx-10.6-universal.egg). maybe I should 
> switch to release?
> 
> I compile with your setup.py using 'python setup.py build_ext -i' and run the 
> commands below in ipython. As you can see, I don't get a crash for a single 
> call to the constructor, consistent with your observation. And it looks like, 
> one has to use dict in the unit to make it crash.
> 
> Can you make sense out of that?
> 
> 
> In [1]: import spampub
> 
> In [4]: spampub.UnitArray(1,{'s':1})
> Out[4]: UnitArray(1)
> 
> In [6]: a=[spampub.UnitArray(i,{'s':i}) for i in xrange(1000)]
> Segmentation fault
> 
> backtrace is the following:
> 
> Exception Type:  EXC_BAD_ACCESS (SIGSEGV)
> Exception Codes: KERN_INVALID_ADDRESS at 0x0021
> Crashed Thread:  0  Dispatch queue: com.apple.main-thread
> 
> Thread 0 Crashed:  Dispatch queue: com.apple.main-thread
> 0   org.python.python 0x0001000b0b8e PyObject_GC_Track + 
> 1473
> 1   org.python.python 0x0001000b1255 _PyObject_GC_Malloc 
> + 191
> 2   org.python.python 0x0001000b12d0 _PyObject_GC_New + 21
> 3   org.python.python 0x00010003a856 PyDict_New + 116
> 4   org.python.python 0x00010003a99c _PyDict_NewPresized 
> + 24
> 5   org.python.python 0x0001000880cb PyEval_EvalFrameEx + 
> 11033
> 6   org.python.python 0x00010008acce PyEval_EvalCodeEx + 
> 1803
> 7   org.python.python 0x00010008735d PyEval_EvalFrameEx + 
> 7595
> 8   org.python.python 0x00010008acce PyEval_EvalCodeEx + 
> 1803
> 9   org.python.python 0x00010008935e PyEval_EvalFrameEx + 
> 15788
> 10  org.python.python 0x00010008acce PyEval_EvalCodeEx + 
> 1803
> 11  org.python.python 0x00010008935e PyEval_EvalFrameEx + 
> 15788
> 12  org.python.python 0x0001000892e1 PyEval_EvalFrameEx + 
> 15663
> 13  org.python.python 0x00010008acce PyEval_EvalCodeEx + 
> 1803
> 14  org.python.python 0x00010008935e PyEval_EvalFrameEx + 
> 15788
> 15  org.python.python 0x00010008acce PyEval_EvalCodeEx + 
> 1803
> 16  org.python.python 0x00010008935e PyEval_EvalFrameEx + 
> 15788
> 17  org.python.python 0x00010008acce PyEval_EvalCodeEx + 
> 1803
> 18  org.python.python 0x00010008935e PyEval_EvalFrameEx + 
> 15788
> 19  org.python.python 0x00010008acce PyEval_EvalCodeEx + 
> 1803
> 20  org.python.python 0x00010008935e PyEval_EvalFrameEx + 
> 15788
> 21  org.python.python 0x00010008acce PyEval_EvalCodeEx + 
> 1803
> 22  org.python.python 0x00010008ad61 PyEval_EvalCode + 54
> 23  org.python.python 0x0001000a265a Py_CompileString + 78
> 24  org.python.python 0x0001000a2723 PyRun_FileExFlags + 
> 150
> 25  org.python.python 0x0001000a423d 
> PyRun_SimpleFileExFlags + 704
> 26  org.python.python 0x0001000b0286 Py_Main + 2718
> 27  org.python.python.app 0x00010e6c start + 52
> 
> Am 08.03.2012 um 02:36 schrieb Val Kalatsky:
> 
>> 
>> Tried it on my Ubuntu 10.10 box, no problem:
>> 
>> 1) Saved as spampub.c
>> 2) Compiled with (setup.py attached): python setup.py build_ext -i
>> 3) Tested from ipython:
>> In [1]: import spampub
>> In [2]: ua=spampub.UnitArray([0,1,2,3.0],'liter')
>> In [3]: ua
>> Out[3]: UnitArray([ 0.,  1.,  2.,  3.])
>> In [4]: ua.unit
>> Out[4]: 'liter'
>> 
>> 
>> On Wed, Mar 7, 2012 at 7:15 PM, Val Kalatsky  wrote:
>> 
>> Seeing the backtrace would be helpful. 
>> Can you do whatever leads to the segfault 
>> from python run from gdb?
>> Val
>> 
>> 
>> On Wed, Mar 7, 2012 at 7:04 PM, Christoph Gohle  
>> wrote:
>> -BEGIN PGP SIGNED MESS

Re: [Numpy-discussion] subclassing array in c

[Val Kalatsky]

Hi Christoph,

I've just tried
a=[spampub.UnitArray(i,{'s':i}) for i in xrange(1000)]
and everything looks fine on my side.
Probably my test environment is too different to give comparable results:

In [3]: call(["uname", "-a"])
Linux ubuntu 2.6.35-22-generic #33-Ubuntu SMP Sun Sep 19 20:32:27 UTC 2010
x86_64 GNU/Linux
In [4]: np.__version__
Out[4]: '1.5.1'

I am afraid I will not be able to help testing it on super-nova versions of
numpy.
Cheers
Val


On Thu, Mar 8, 2012 at 1:49 AM, Christoph Gohle
wrote:

> Dear Val,
>
> I agree that more detail is needed. Sorry for that it was late yesterday.
>
>  I am running Python 2.6.1, numpy development branch
> (numpy-2.0.0.dev_20101104-py2.6-macosx-10.6-universal.egg). maybe I should
> switch to release?
>
> I compile with your setup.py using 'python setup.py build_ext -i' and run
> the commands below in ipython. As you can see, I don't get a crash for a
> single call to the constructor, consistent with your observation. And it
> looks like, one has to use dict in the unit to make it crash.
>
> Can you make sense out of that?
>
>
> In [1]: import spampub
>
> In [4]: spampub.UnitArray(1,{'s':1})
> Out[4]: UnitArray(1)
>
> In [6]: a=[spampub.UnitArray(i,{'s':i}) for i in xrange(1000)]
> Segmentation fault
>
> backtrace is the following:
>
> Exception Type:  EXC_BAD_ACCESS (SIGSEGV)
> Exception Codes: KERN_INVALID_ADDRESS at 0x0021
> Crashed Thread:  0  Dispatch queue: com.apple.main-thread
>
> Thread 0 Crashed:  Dispatch queue: com.apple.main-thread
> 0   org.python.python 0x0001000b0b8e PyObject_GC_Track +
> 1473
> 1   org.python.python 0x0001000b1255 _PyObject_GC_Malloc
> + 191
> 2   org.python.python 0x0001000b12d0 _PyObject_GC_New + 21
> 3   org.python.python 0x00010003a856 PyDict_New + 116
> 4   org.python.python 0x00010003a99c _PyDict_NewPresized
> + 24
> 5   org.python.python 0x0001000880cb PyEval_EvalFrameEx +
> 11033
> 6   org.python.python 0x00010008acce PyEval_EvalCodeEx +
> 1803
> 7   org.python.python 0x00010008735d PyEval_EvalFrameEx +
> 7595
> 8   org.python.python 0x00010008acce PyEval_EvalCodeEx +
> 1803
> 9   org.python.python 0x00010008935e PyEval_EvalFrameEx +
> 15788
> 10  org.python.python 0x00010008acce PyEval_EvalCodeEx +
> 1803
> 11  org.python.python 0x00010008935e PyEval_EvalFrameEx +
> 15788
> 12  org.python.python 0x0001000892e1 PyEval_EvalFrameEx +
> 15663
> 13  org.python.python 0x00010008acce PyEval_EvalCodeEx +
> 1803
> 14  org.python.python 0x00010008935e PyEval_EvalFrameEx +
> 15788
> 15  org.python.python 0x00010008acce PyEval_EvalCodeEx +
> 1803
> 16  org.python.python 0x00010008935e PyEval_EvalFrameEx +
> 15788
> 17  org.python.python 0x00010008acce PyEval_EvalCodeEx +
> 1803
> 18  org.python.python 0x00010008935e PyEval_EvalFrameEx +
> 15788
> 19  org.python.python 0x00010008acce PyEval_EvalCodeEx +
> 1803
> 20  org.python.python 0x00010008935e PyEval_EvalFrameEx +
> 15788
> 21  org.python.python 0x00010008acce PyEval_EvalCodeEx +
> 1803
> 22  org.python.python 0x00010008ad61 PyEval_EvalCode + 54
> 23  org.python.python 0x0001000a265a Py_CompileString + 78
> 24  org.python.python 0x0001000a2723 PyRun_FileExFlags +
> 150
> 25  org.python.python 0x0001000a423d
> PyRun_SimpleFileExFlags + 704
> 26  org.python.python 0x0001000b0286 Py_Main + 2718
> 27  org.python.python.app 0x00010e6c start + 52
>
> Am 08.03.2012 um 02:36 schrieb Val Kalatsky:
>
>
> Tried it on my Ubuntu 10.10 box, no problem:
>
> 1) Saved as spampub.c
> 2) Compiled with (setup.py attached): python setup.py build_ext -i
> 3) Tested from ipython:
> In [1]: import spampub
> In [2]: ua=spampub.UnitArray([0,1,2,3.0],'liter')
> In [3]: ua
> Out[3]: UnitArray([ 0.,  1.,  2.,  3.])
> In [4]: ua.unit
> Out[4]: 'liter'
>
>
> On Wed, Mar 7, 2012 at 7:15 PM, Val Kalatsky  wrote:
>
>>
>> Seeing the backtrace would be helpful.
>> Can you do whatever leads to the segfault
>> from python run from gdb?
>> Val
>>
>>
>> On Wed, Mar 7, 2012 at 7:04 PM, Christoph Gohle <
>> christoph.go...@mpq.mpg.de> wrote:
>>
>>> -BEGIN PGP SIGNED MESSAGE-
>>> Hash: SHA1
>>>
>>> Hi,
>>>
>>> I have been struggeling for quite some time now. Desperate as I am, now
>>> I need help.
>>>
>>> I was trying to subclass ndarrays in a c extension (see code below) and
>>> do constantly get segfaults. I have been checking my INCREF and DECREF
>>> stuff up and down but can't find the error. Probably I got something
>>> completely wrong... anybody able to help?
>>>
>>> Thanks,
>>> Christoph
>>> - -
>>> #include 
>>> #include 
>>>
>>> #inc

Re: [Numpy-discussion] subclassing array in c

[Christoph Gohle]

Dear Val,

I agree that more detail is needed. Sorry for that it was late yesterday.

 I am running Python 2.6.1, numpy development branch 
(numpy-2.0.0.dev_20101104-py2.6-macosx-10.6-universal.egg). maybe I should 
switch to release?

I compile with your setup.py using 'python setup.py build_ext -i' and run the 
commands below in ipython. As you can see, I don't get a crash for a single 
call to the constructor, consistent with your observation. And it looks like, 
one has to use dict in the unit to make it crash.

Can you make sense out of that?


In [1]: import spampub

In [4]: spampub.UnitArray(1,{'s':1})
Out[4]: UnitArray(1)

In [6]: a=[spampub.UnitArray(i,{'s':i}) for i in xrange(1000)]
Segmentation fault

backtrace is the following:

Exception Type:  EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x0021
Crashed Thread:  0  Dispatch queue: com.apple.main-thread

Thread 0 Crashed:  Dispatch queue: com.apple.main-thread
0   org.python.python   0x0001000b0b8e PyObject_GC_Track + 
1473
1   org.python.python   0x0001000b1255 _PyObject_GC_Malloc 
+ 191
2   org.python.python   0x0001000b12d0 _PyObject_GC_New + 21
3   org.python.python   0x00010003a856 PyDict_New + 116
4   org.python.python   0x00010003a99c _PyDict_NewPresized 
+ 24
5   org.python.python   0x0001000880cb PyEval_EvalFrameEx + 
11033
6   org.python.python   0x00010008acce PyEval_EvalCodeEx + 
1803
7   org.python.python   0x00010008735d PyEval_EvalFrameEx + 
7595
8   org.python.python   0x00010008acce PyEval_EvalCodeEx + 
1803
9   org.python.python   0x00010008935e PyEval_EvalFrameEx + 
15788
10  org.python.python   0x00010008acce PyEval_EvalCodeEx + 
1803
11  org.python.python   0x00010008935e PyEval_EvalFrameEx + 
15788
12  org.python.python   0x0001000892e1 PyEval_EvalFrameEx + 
15663
13  org.python.python   0x00010008acce PyEval_EvalCodeEx + 
1803
14  org.python.python   0x00010008935e PyEval_EvalFrameEx + 
15788
15  org.python.python   0x00010008acce PyEval_EvalCodeEx + 
1803
16  org.python.python   0x00010008935e PyEval_EvalFrameEx + 
15788
17  org.python.python   0x00010008acce PyEval_EvalCodeEx + 
1803
18  org.python.python   0x00010008935e PyEval_EvalFrameEx + 
15788
19  org.python.python   0x00010008acce PyEval_EvalCodeEx + 
1803
20  org.python.python   0x00010008935e PyEval_EvalFrameEx + 
15788
21  org.python.python   0x00010008acce PyEval_EvalCodeEx + 
1803
22  org.python.python   0x00010008ad61 PyEval_EvalCode + 54
23  org.python.python   0x0001000a265a Py_CompileString + 78
24  org.python.python   0x0001000a2723 PyRun_FileExFlags + 
150
25  org.python.python   0x0001000a423d 
PyRun_SimpleFileExFlags + 704
26  org.python.python   0x0001000b0286 Py_Main + 2718
27  org.python.python.app   0x00010e6c start + 52

Am 08.03.2012 um 02:36 schrieb Val Kalatsky:

> 
> Tried it on my Ubuntu 10.10 box, no problem:
> 
> 1) Saved as spampub.c
> 2) Compiled with (setup.py attached): python setup.py build_ext -i
> 3) Tested from ipython:
> In [1]: import spampub
> In [2]: ua=spampub.UnitArray([0,1,2,3.0],'liter')
> In [3]: ua
> Out[3]: UnitArray([ 0.,  1.,  2.,  3.])
> In [4]: ua.unit
> Out[4]: 'liter'
> 
> 
> On Wed, Mar 7, 2012 at 7:15 PM, Val Kalatsky  wrote:
> 
> Seeing the backtrace would be helpful. 
> Can you do whatever leads to the segfault 
> from python run from gdb?
> Val
> 
> 
> On Wed, Mar 7, 2012 at 7:04 PM, Christoph Gohle  
> wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> Hi,
> 
> I have been struggeling for quite some time now. Desperate as I am, now I 
> need help.
> 
> I was trying to subclass ndarrays in a c extension (see code below) and do 
> constantly get segfaults. I have been checking my INCREF and DECREF stuff up 
> and down but can't find the error. Probably I got something completely 
> wrong... anybody able to help?
> 
> Thanks,
> Christoph
> - -
> #include 
> #include 
> 
> #include 
> 
> static PyObject *SpamError;
> 
> typedef struct {
>  PyArrayObject base;
>  PyDictObject* unit;
> } UnitArrayObject;
> 
> PyTypeObject unitArrayObjectType;
> 
> static int
> checkunit(PyObject *unit1, PyObject *unit2) {
>  return PyObject_Compare(unit1, unit2);
> }
> 
> static PyObject *
> unitArray_new(PyTypeObject *cls, PyObject *args, PyObject *kwargs) {
>PyObject *data = NULL,
> *unit = NULL;
>  PyArray_Descr* dtype = NULL;
>  PyObject *res = NULL, *tmp = NULL;
> 
>if (!PyArg_P

Re: [Numpy-discussion] subclassing array in c

[Christoph Gohlke]

FWIW, this crashes on Windows with numpy 1.6.1 but not numpy 1.7-git 
debug build.

Christoph Gohlke


On 3/7/2012 5:36 PM, Val Kalatsky wrote:
>
> Tried it on my Ubuntu 10.10 box, no problem:
>
> 1) Saved as spampub.c
> 2) Compiled with (setup.py attached): python setup.py build_ext -i
> 3) Tested from ipython:
> In [1]: import spampub
> In [2]: ua=spampub.UnitArray([0,1,2,3.0],'liter')
> In [3]: ua
> Out[3]: UnitArray([ 0.,  1.,  2.,  3.])
> In [4]: ua.unit
> Out[4]: 'liter'
>
>
> On Wed, Mar 7, 2012 at 7:15 PM, Val Kalatsky  > wrote:
>
>
> Seeing the backtrace would be helpful.
> Can you do whatever leads to the segfault
> from python run from gdb?
> Val
>
>
> On Wed, Mar 7, 2012 at 7:04 PM, Christoph Gohle
> mailto:christoph.go...@mpq.mpg.de>> wrote:
>
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Hi,
>
> I have been struggeling for quite some time now. Desperate as I
> am, now I need help.
>
> I was trying to subclass ndarrays in a c extension (see code
> below) and do constantly get segfaults. I have been checking my
> INCREF and DECREF stuff up and down but can't find the error.
> Probably I got something completely wrong... anybody able to help?
>
> Thanks,
> Christoph
> - -
> #include 
> #include 
>
> #include 
>
> static PyObject *SpamError;
>
> typedef struct {
>   PyArrayObject base;
>   PyDictObject* unit;
> } UnitArrayObject;
>
> PyTypeObject unitArrayObjectType;
>
> static int
> checkunit(PyObject *unit1, PyObject *unit2) {
>   return PyObject_Compare(unit1, unit2);
> }
>
> static PyObject *
> unitArray_new(PyTypeObject *cls, PyObject *args, PyObject *kwargs) {
> PyObject *data = NULL,
>  *unit = NULL;
>   PyArray_Descr* dtype = NULL;
>   PyObject *res = NULL, *tmp = NULL;
>
> if (!PyArg_ParseTuple(args, "OO|O&", &data, &unit,
> PyArray_DescrConverter, &dtype)) {
> Py_XDECREF(dtype);
> return NULL;
> }
>
>   res = PyArray_FromAny(data, dtype, 0, 0, NPY_ENSURECOPY, NULL);
> if (res == NULL) {
> Py_XDECREF(dtype);
> //TODO: raise exception?
> return NULL;
> }
>
> if (PyObject_IsInstance(data, (PyObject*)cls)) {
> if (unit!=NULL &&
> !checkunit((PyObject*)((UnitArrayObject*)data)->unit,unit)) {
> Py_XDECREF(res);
> //TODO: raise exception
> return NULL;
> }
> } else {
> if (PyObject_IsTrue(unit)) {
>   tmp = res;
> res = PyArray_View((PyArrayObject*)res,
> NULL, &unitArrayObjectType);
>   if (tmp!=res) {
> Py_XDECREF(tmp);
>   }
>   ((UnitArrayObject*)res)->unit = (PyDictObject*)unit;
>   Py_INCREF(unit);
>   if (unit!=NULL) {
>   }
> }
> }
> return res;
> }
>
> static PyObject*
> unitArray__array_finalize__(PyObject* new, PyObject* args) {
> PyObject *attr = NULL, *tmp = NULL;
>   PyObject *parent = NULL;
>
>   if (!PyArg_ParseTuple(args, "O", &parent)) {
> return NULL;
>   }
>if (parent!=NULL) {
>  attr = PyObject_GetAttrString(parent, "unit");
>  if (attr == NULL) {
> //parent has no 'unit' so we make a new empty one
>attr = PyDict_New();
>PyErr_Clear();
>  }
>}
>   tmp = (PyObject*)((UnitArrayObject*)new)->unit;
> ((UnitArrayObject*)new)->unit = (PyDictObject*)attr;
>
>   Py_INCREF(Py_None);
>   return Py_None;
> }
>
> static PyObject*
> unitArray__array_wrap__(PyObject *self, PyObject *args) {
> PyObject *array = NULL, *context = NULL;
>
> if (!PyArg_ParseTuple(args, "OO", array, context)) {
> //TODO: raise exception
> return NULL;
> }
>
> printf("%s",PyString_AsString(PyObject_Str(context)));
>
>   Py_INCREF(array);
>   return array;
> }
>
>
> static PyMethodDef unitArrayMethods[] = {
>   {"__array_finalize__", unitArray__array_finalize__,
> METH_VARARGS, "array finalize method"},
>   {"__array_wrap__", unitArray__array_wrap__, METH_VARARGS,
> "arr

Re: [Numpy-discussion] subclassing array in c

[Val Kalatsky]

Tried it on my Ubuntu 10.10 box, no problem:

1) Saved as spampub.c
2) Compiled with (setup.py attached): python setup.py build_ext -i
3) Tested from ipython:
In [1]: import spampub
In [2]: ua=spampub.UnitArray([0,1,2,3.0],'liter')
In [3]: ua
Out[3]: UnitArray([ 0.,  1.,  2.,  3.])
In [4]: ua.unit
Out[4]: 'liter'


On Wed, Mar 7, 2012 at 7:15 PM, Val Kalatsky  wrote:

>
> Seeing the backtrace would be helpful.
> Can you do whatever leads to the segfault
> from python run from gdb?
> Val
>
>
> On Wed, Mar 7, 2012 at 7:04 PM, Christoph Gohle <
> christoph.go...@mpq.mpg.de> wrote:
>
>> -BEGIN PGP SIGNED MESSAGE-
>> Hash: SHA1
>>
>> Hi,
>>
>> I have been struggeling for quite some time now. Desperate as I am, now I
>> need help.
>>
>> I was trying to subclass ndarrays in a c extension (see code below) and
>> do constantly get segfaults. I have been checking my INCREF and DECREF
>> stuff up and down but can't find the error. Probably I got something
>> completely wrong... anybody able to help?
>>
>> Thanks,
>> Christoph
>> - -
>> #include 
>> #include 
>>
>> #include 
>>
>> static PyObject *SpamError;
>>
>> typedef struct {
>>  PyArrayObject base;
>>  PyDictObject* unit;
>> } UnitArrayObject;
>>
>> PyTypeObject unitArrayObjectType;
>>
>> static int
>> checkunit(PyObject *unit1, PyObject *unit2) {
>>  return PyObject_Compare(unit1, unit2);
>> }
>>
>> static PyObject *
>> unitArray_new(PyTypeObject *cls, PyObject *args, PyObject *kwargs) {
>>PyObject *data = NULL,
>> *unit = NULL;
>>  PyArray_Descr* dtype = NULL;
>>  PyObject *res = NULL, *tmp = NULL;
>>
>>if (!PyArg_ParseTuple(args, "OO|O&", &data, &unit,
>> PyArray_DescrConverter, &dtype)) {
>>Py_XDECREF(dtype);
>>return NULL;
>>}
>>
>>  res = PyArray_FromAny(data, dtype, 0, 0, NPY_ENSURECOPY, NULL);
>>if (res == NULL) {
>>Py_XDECREF(dtype);
>>//TODO: raise exception?
>>return NULL;
>>}
>>
>>if (PyObject_IsInstance(data, (PyObject*)cls)) {
>>if (unit!=NULL &&
>> !checkunit((PyObject*)((UnitArrayObject*)data)->unit,unit)) {
>>Py_XDECREF(res);
>>//TODO: raise exception
>>return NULL;
>>}
>>} else {
>>if (PyObject_IsTrue(unit)) {
>>  tmp = res;
>>res = PyArray_View((PyArrayObject*)res, NULL,
>> &unitArrayObjectType);
>>  if (tmp!=res) {
>>Py_XDECREF(tmp);
>>  }
>>  ((UnitArrayObject*)res)->unit = (PyDictObject*)unit;
>>  Py_INCREF(unit);
>>  if (unit!=NULL) {
>>  }
>>}
>>}
>>return res;
>> }
>>
>> static PyObject*
>> unitArray__array_finalize__(PyObject* new, PyObject* args) {
>>PyObject *attr = NULL, *tmp = NULL;
>>  PyObject *parent = NULL;
>>
>>  if (!PyArg_ParseTuple(args, "O", &parent)) {
>>return NULL;
>>  }
>>   if (parent!=NULL) {
>> attr = PyObject_GetAttrString(parent, "unit");
>> if (attr == NULL) {
>>//parent has no 'unit' so we make a new empty one
>>   attr = PyDict_New();
>>   PyErr_Clear();
>> }
>>   }
>>  tmp = (PyObject*)((UnitArrayObject*)new)->unit;
>>((UnitArrayObject*)new)->unit = (PyDictObject*)attr;
>>
>>  Py_INCREF(Py_None);
>>  return Py_None;
>> }
>>
>> static PyObject*
>> unitArray__array_wrap__(PyObject *self, PyObject *args) {
>>PyObject *array = NULL, *context = NULL;
>>
>>if (!PyArg_ParseTuple(args, "OO", array, context)) {
>>//TODO: raise exception
>>return NULL;
>>}
>>
>>printf("%s",PyString_AsString(PyObject_Str(context)));
>>
>>  Py_INCREF(array);
>>  return array;
>> }
>>
>>
>> static PyMethodDef unitArrayMethods[] = {
>>  {"__array_finalize__", unitArray__array_finalize__, METH_VARARGS, "array
>> finalize method"},
>>  {"__array_wrap__", unitArray__array_wrap__, METH_VARARGS, "array wrap
>> method"},
>>  {NULL, NULL, 0, NULL}
>> };
>>
>> static PyMemberDef unitArrayMembers[] = {
>>  {"unit", T_OBJECT, offsetof(UnitArrayObject, unit),  0, "dictionary
>> containing unit info."},
>>  {NULL, 0, 0, 0, NULL}
>> };
>>
>> PyTypeObject unitArrayObjectType = {
>>PyObject_HEAD_INIT(NULL)
>>0,  /* ob_size*/
>>"spam.UnitArray",   /* tp_name*/
>>sizeof(UnitArrayObject),/* tp_basicsize   */
>>0,  /* tp_itemsize*/
>>0,  /* tp_dealloc */
>>0,  /* tp_print   */
>>0,  /* tp_getattr */
>>0,  /* tp_setattr */
>>0,  /* tp_compare */
>>0,  /* tp_repr*/
>>0,  

Re: [Numpy-discussion] subclassing array in c

[Val Kalatsky]

Seeing the backtrace would be helpful.
Can you do whatever leads to the segfault
from python run from gdb?
Val

On Wed, Mar 7, 2012 at 7:04 PM, Christoph Gohle
wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Hi,
>
> I have been struggeling for quite some time now. Desperate as I am, now I
> need help.
>
> I was trying to subclass ndarrays in a c extension (see code below) and do
> constantly get segfaults. I have been checking my INCREF and DECREF stuff
> up and down but can't find the error. Probably I got something completely
> wrong... anybody able to help?
>
> Thanks,
> Christoph
> - -
> #include 
> #include 
>
> #include 
>
> static PyObject *SpamError;
>
> typedef struct {
>  PyArrayObject base;
>  PyDictObject* unit;
> } UnitArrayObject;
>
> PyTypeObject unitArrayObjectType;
>
> static int
> checkunit(PyObject *unit1, PyObject *unit2) {
>  return PyObject_Compare(unit1, unit2);
> }
>
> static PyObject *
> unitArray_new(PyTypeObject *cls, PyObject *args, PyObject *kwargs) {
>PyObject *data = NULL,
> *unit = NULL;
>  PyArray_Descr* dtype = NULL;
>  PyObject *res = NULL, *tmp = NULL;
>
>if (!PyArg_ParseTuple(args, "OO|O&", &data, &unit,
> PyArray_DescrConverter, &dtype)) {
>Py_XDECREF(dtype);
>return NULL;
>}
>
>  res = PyArray_FromAny(data, dtype, 0, 0, NPY_ENSURECOPY, NULL);
>if (res == NULL) {
>Py_XDECREF(dtype);
>//TODO: raise exception?
>return NULL;
>}
>
>if (PyObject_IsInstance(data, (PyObject*)cls)) {
>if (unit!=NULL &&
> !checkunit((PyObject*)((UnitArrayObject*)data)->unit,unit)) {
>Py_XDECREF(res);
>//TODO: raise exception
>return NULL;
>}
>} else {
>if (PyObject_IsTrue(unit)) {
>  tmp = res;
>res = PyArray_View((PyArrayObject*)res, NULL,
> &unitArrayObjectType);
>  if (tmp!=res) {
>Py_XDECREF(tmp);
>  }
>  ((UnitArrayObject*)res)->unit = (PyDictObject*)unit;
>  Py_INCREF(unit);
>  if (unit!=NULL) {
>  }
>}
>}
>return res;
> }
>
> static PyObject*
> unitArray__array_finalize__(PyObject* new, PyObject* args) {
>PyObject *attr = NULL, *tmp = NULL;
>  PyObject *parent = NULL;
>
>  if (!PyArg_ParseTuple(args, "O", &parent)) {
>return NULL;
>  }
>   if (parent!=NULL) {
> attr = PyObject_GetAttrString(parent, "unit");
> if (attr == NULL) {
>//parent has no 'unit' so we make a new empty one
>   attr = PyDict_New();
>   PyErr_Clear();
> }
>   }
>  tmp = (PyObject*)((UnitArrayObject*)new)->unit;
>((UnitArrayObject*)new)->unit = (PyDictObject*)attr;
>
>  Py_INCREF(Py_None);
>  return Py_None;
> }
>
> static PyObject*
> unitArray__array_wrap__(PyObject *self, PyObject *args) {
>PyObject *array = NULL, *context = NULL;
>
>if (!PyArg_ParseTuple(args, "OO", array, context)) {
>//TODO: raise exception
>return NULL;
>}
>
>printf("%s",PyString_AsString(PyObject_Str(context)));
>
>  Py_INCREF(array);
>  return array;
> }
>
>
> static PyMethodDef unitArrayMethods[] = {
>  {"__array_finalize__", unitArray__array_finalize__, METH_VARARGS, "array
> finalize method"},
>  {"__array_wrap__", unitArray__array_wrap__, METH_VARARGS, "array wrap
> method"},
>  {NULL, NULL, 0, NULL}
> };
>
> static PyMemberDef unitArrayMembers[] = {
>  {"unit", T_OBJECT, offsetof(UnitArrayObject, unit),  0, "dictionary
> containing unit info."},
>  {NULL, 0, 0, 0, NULL}
> };
>
> PyTypeObject unitArrayObjectType = {
>PyObject_HEAD_INIT(NULL)
>0,  /* ob_size*/
>"spam.UnitArray",   /* tp_name*/
>sizeof(UnitArrayObject),/* tp_basicsize   */
>0,  /* tp_itemsize*/
>0,  /* tp_dealloc */
>0,  /* tp_print   */
>0,  /* tp_getattr */
>0,  /* tp_setattr */
>0,  /* tp_compare */
>0,  /* tp_repr*/
>0,  /* tp_as_number   */
>0,  /* tp_as_sequence */
>0,  /* tp_as_mapping  */
>0,  /* tp_hash*/
>0,  /* tp_call*/
>0,  /* tp_str */
>0,  /* tp_getattro*/
>0,  /* tp_setattro*/
>0,  /* tp_as_buffer   */
>Py_TPFLAGS_DEFAULT,