Re: [Numpy-discussion] numpy CAPI questions
Lane Brooks wrote: I am using the numpy CAPI to write an extension module that returns a numpy Array from an imaging data source. I collect the image into a buffer that I allocate. I then create numpy Array using the PyArray_New(..) function and pass it the buffer. I then set the NPY_OWNDATA flag on the Array because I want the Array to deallocate the buffer when it is deleted. Is that the correct way to do it? The code snippet below is what I wrote, and it seems to be working just fine, but I wanted to verify that I am doing things correctly. NPY_OWNDATA means the object will try to deallocate the memory (make sure it was allocated with the same allocator as NumPy uses). Otherwise, you will need to set up another approach as I showed in my blog posting several weeks ago. Also, don't use Py_BuildValue with O as it will create another reference so that img will have an extra reference to it when it is returned to Python. Use N instead. However, in this case you don't need to use Py_BuildValue at all because you are returning only one array. The PyArray_UpdateFlags call is not used for changing NPY_OWNDATA. It is only useful for changing FORTRAN, CONTIGUOUS, ALIGNED, and WRITEABLE flags which are convenience flags. This call does the check first and then sets the state of the flag to reflect the actual situation for the array. Instead use PyArray_FLAGS(arr) |= NPY_OWNDATA; -Travis ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] numpy CAPI questions
Travis E. Oliphant wrote: Lane Brooks wrote: I am using the numpy CAPI to write an extension module that returns a numpy Array from an imaging data source. I collect the image into a buffer that I allocate. I then create numpy Array using the PyArray_New(..) function and pass it the buffer. I then set the NPY_OWNDATA flag on the Array because I want the Array to deallocate the buffer when it is deleted. Is that the correct way to do it? The code snippet below is what I wrote, and it seems to be working just fine, but I wanted to verify that I am doing things correctly. NPY_OWNDATA means the object will try to deallocate the memory (make sure it was allocated with the same allocator as NumPy uses). Otherwise, you will need to set up another approach as I showed in my blog posting several weeks ago. Also, don't use Py_BuildValue with O as it will create another reference so that img will have an extra reference to it when it is returned to Python. Use N instead. However, in this case you don't need to use Py_BuildValue at all because you are returning only one array. The PyArray_UpdateFlags call is not used for changing NPY_OWNDATA. It is only useful for changing FORTRAN, CONTIGUOUS, ALIGNED, and WRITEABLE flags which are convenience flags. This call does the check first and then sets the state of the flag to reflect the actual situation for the array. Instead use PyArray_FLAGS(arr) |= NPY_OWNDATA; -Travis Thanks for all this valuable feedback. I read your blog post and like the idea but not the overhead. I guess my initial approach of doing a memory handoff to the numpy Array was a bit naive. It seems to be working, but I guess that is because numpy uses free to deallocate memory? ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] numpy CAPI questions
On Mon, Oct 20, 2008 at 16:13, Lane Brooks [EMAIL PROTECTED] wrote: Thanks for all this valuable feedback. I read your blog post and like the idea but not the overhead. I guess my initial approach of doing a memory handoff to the numpy Array was a bit naive. It seems to be working, but I guess that is because numpy uses free to deallocate memory? It uses PyMem_Free() (except for a few places that I've just noticed) which itself uses free() *except* if Python is compiled with -DPYMALLOC_DEBUG, in which case it uses its own debugging implementations. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] numpy CAPI questions
I am using the numpy CAPI to write an extension module that returns a numpy Array from an imaging data source. I collect the image into a buffer that I allocate. I then create numpy Array using the PyArray_New(..) function and pass it the buffer. I then set the NPY_OWNDATA flag on the Array because I want the Array to deallocate the buffer when it is deleted. Is that the correct way to do it? The code snippet below is what I wrote, and it seems to be working just fine, but I wanted to verify that I am doing things correctly. uint16_t *data; // malloc data and fill it with some image data... img = PyArray_New(PyArray_Type, 2, dims, NPY_UINT16, NULL, data, 2, NPY_C_CONTIGUOUS | NPY_WRITEABLE | NPY_ALIGNED, NULL); PyArray_UpdateFlags((PyArrayObject *) img, NPY_OWNDATA | NPY_C_CONTIGUOUS | NPY_WRITEABLE | NPY_ALIGNED); return Py_BuildValue(O, img); Here are my questions: 1. Does NPY_OWNDATA mean that the object will deallocate the memory when the object is deleted? The manual seems to indicate that as such but it is not explicitly stated. 2. Is my reference counting correct? Do I need to call the PyArray_INCREF() on img? Thanks, Lane Brooks ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] numpy CAPI questions
On Sun, Oct 19, 2008 at 14:28, Lane Brooks [EMAIL PROTECTED] wrote: I am using the numpy CAPI to write an extension module that returns a numpy Array from an imaging data source. I collect the image into a buffer that I allocate. I then create numpy Array using the PyArray_New(..) function and pass it the buffer. I then set the NPY_OWNDATA flag on the Array because I want the Array to deallocate the buffer when it is deleted. Is that the correct way to do it? The code snippet below is what I wrote, and it seems to be working just fine, but I wanted to verify that I am doing things correctly. Preferably, you should create the array and then use the memory it allocated as the buffer. Generally, malloc()/free() implementations are paired, and if you mix-and-match, you're begging for trouble. Here are my questions: 1. Does NPY_OWNDATA mean that the object will deallocate the memory when the object is deleted? The manual seems to indicate that as such but it is not explicitly stated. Yes. 2. Is my reference counting correct? Do I need to call the PyArray_INCREF() on img? Personally, I always need to double-check my refcounting with sys.getrefcount() (which, it should be noted, adds its own reference, so the correct result for sys.getrefcount(foo()) should be 1). In your actual code, do you have anything else in the Py_BuildValue()? If you don't, then you don't need to use Py_BuildValue(); you can just return img. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
