Brian Granger wrote:
> Hi,
Hi Brian,
> (dual posted to sage and cython)
>
> A few of us (ipython and mpi4py devs) are wondering what the
> best/safest way of allocating dynamic memory in a local scope
> (method/function) is when using cython. An example would be if you
> need an array of c ints that is locally scoped.
>
> The big question is how to make sure that the memory gets freed - even
> if something goes wrong in the function/method. That is, you want to
> prevent memory leaks. It looks like in sage, the
> sage_malloc/sage_free functions are used for this purpose:
They generally aren't used in most of the code. The idea for those
functions is that in the future we can wrap other allocators like slab
allocators.
> from sage/graphs/graph_isom.pyx:
>
> 176 def incorporate_permutation(self, gamma):
> 202 cdef int *_gamma = <int *> sage_malloc( n * sizeof(int) )
> 203 if not _gamma:
> 204 raise MemoryError("Error allocating memory.")
> 205 for k from 0 <= k < n:
> 206 _gamma[k] = gamma[k]
> 207 self._incorporate_permutation(_gamma, n)
> 208 sage_free(_gamma)
>
> Because sage_malloc is #defined to malloc in stdsage.h, I think there
> is a significant potential for memory leaks in code like this. Are we
> thinking correctly on this issue? Isn't this a huge problem?
Well, I don't see an advantage in using Python's allocator there. It is
likely slower for large allocations and make debugging memory issues
much more complicated since issues like pointer corruption is
significantly harder to debug.
> Lisandro Dalcin (author of mpi4py) came up with the following trick
> that, while more complicated, prevents memory leaks:
>
> cdef extern from "Python.h":
> object PyString_FromStringAndSize(char*,Py_ssize_t)
> char* PyString_AS_STRING(object)
>
> cdef inline object pyalloc_i(int size, int **i):
> if size < 0: size = 0
> cdef Py_ssize_t n = size * sizeof(int)
> cdef object ob = PyString_FromStringAndSize(NULL, n)
> i[0] = <int*> PyString_AS_STRING(ob)
> return ob
>
> and now
>
> def foo(sequence):
> cdef int size = len(sequence),
> cdef int *buf = NULL
> cdef object tmp = pyalloc_i(size, &buf)
>
> This could probably be adapted into a malloc-like function. What do
> people think?
We valgrind the complete test suite at least weekly and with ever
increasing doctest coverage I don't see that we have a problem. Python's
memory management has also some serious issues and I doubt it will offer
any advantage for anything but loads of small allocs. And by switching
to a slab allocator on our end will fix that problem.
> Thanks,
>
> Brian
Cheers,
Michael
> _______________________________________________
> Cython-dev mailing list
> [EMAIL PROTECTED]
> http://codespeak.net/mailman/listinfo/cython-dev
>
--~--~---------~--~----~------------~-------~--~----~
To post to this group, send email to sage-devel@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at http://groups.google.com/group/sage-devel
URLs: http://www.sagemath.org
-~----------~----~----~----~------~----~------~--~---
