Hello,
I'm one of the maintainers of the gevent concurrency library, which has
recently been ported to run on PyPy. Under PyPy, a small portion of the code is
compiled with Cython in order to get the desired atomic semantics
(specifically, a semaphore class). We recently had a user report an easily
reproducible leak of tuples of one element. Tracking it down, it appears that
__Pyx_PyObject_CallOneArg creates a new tuple under PyPy, but neglects to free
it. This was tested with Cython 0.23.3 and PyPy 2.6.1.
Our Cython code contained a loop like this, and every iteration of the loop
leaked a tuple:
for link in links:
link(self)
The C output for that last line looked like this:
__Pyx_PyObject_CallOneArg(__pyx_t_10, ((PyObject *)__pyx_v_self));...
And, under PyPy, the implementation of __Pyx_PyObject_CallOneArg is different
than it is under CPython. Specifically, with Cython 0.23.3 this is the code:
#if CYTHON_COMPILING_IN_CPYTHON
...
#else
static CYTHON_INLINE PyObject* __Pyx_PyObject_CallOneArg(PyObject *func,
PyObject *arg) {
PyObject* args = PyTuple_Pack(1, arg);
return (likely(args)) ? __Pyx_PyObject_Call(func, args, NULL) : NULL;
}
#endif
PyTuple_Pack is documented as returning a new reference
(https://docs.python.org/2/c-api/tuple.html#c.PyTuple_Pack), so it seems to me
like this code should be decrementing the reference to the tuple before
returning (much like the __Pyx__PyObject_CallOneArg function does under
CPython).
Changing the loop to avoid the use of CallOneArg seemed to resolve the tuple
leak, presumably at some execution time cost:
args = (self,)
for link in links:
link(*self)
There's some additional background at
https://bitbucket.org/pypy/pypy/issues/2149/memory-leak-for-python-subclass-of-cpyext#comment-22347393
Am I interpreting this correctly to be a bug, or could there be something wrong
in the way we're handling callbacks? Please let me know if there's any further
information I can provide.
Thanks,
Jason
_______________________________________________
cython-devel mailing list
[email protected]
https://mail.python.org/mailman/listinfo/cython-devel
