Robert Bradshaw wrote: > On Nov 30, 2008, at 4:33 AM, Dag Sverre Seljebotn wrote: > >> Here's an idea: >> >> Have a babysit mode for refcounting. >> >> a) When entering an exiting from a function, "__Pyx_PushRefCountFrame" >> and "__Pyx_PopRefcountFrame(__pyx_r)" are called. >> >> b) Use __Pyx_INCREF and __Pyx_DECREF instead >> >> c) Whenever a function call returns an object with a reference, >> __Pyx_GOTREF should be called on the returned object. >> >> (Point c) could be done with refactoring "the call of a Python C API >> function" to code.call_func or similar -- I'm noticing that a lot of >> code is duplicated all over the place currently.) >> >> With these, I think it should be possible to (conditionally on a >> #define) define things in such a way that a) __Pyx_DECREF can log >> useful >> information and exit more gracefully than a segfault, b) >> __Pyx_PopRefcountFrame can report any memory leaks, locally from the >> perspective of the function. >> >> Am I overlooking anything obvious? What do you think? I'm merely >> wondering if it is a good enough idea to warrant creating a ticket for >> it, not volunteering at this stage. >> >> If this was done before moving more temps it could perhaps help with >> gaining confidence in a change quickly, and it would be a useful >> addition to the regression test framework for the future as well. > > > This is to ease development, not for end-users, right? I think this
Right. It's something to enable while running runtests.py, in order to get more proper regression tests on refcounting than a segfault. (Also leaks are currently not regression tested as far as I know.) It wouldn't help for leaks in end-user code, it would help ensure that Cython itself doesn't introduce leaks. > could be a good idea, though it's unclear exactly what > "__Pyx_PushRefCountFrame" and "__Pyx_PopRefcountFrame" would do. And > would __Pyx_DECREF be more useful than running under gdb? The former would construct a new object which is stored in a global stack (basically a dict of pointers-casted-to-integer to a "refcount that the curren function holds"), and which __Pyx_???REF would record information in. (Hmm. Perhaps it would be better to conditinally define a local variable with a fixed name. But these are details unecesarry to discuss at this point.) The latter would count up everything and make sure that there was exactly zero references (that the function knows that it holds) to everything but the parameter, __pyx_r, which should have exactly one reference (which the function holds). Schemes using the Python ob_refcnt could work as well, but in general I thought something that is unit-testing the refcount things that happen in the same function only, independent of possible wrong reference counting going on in other functions, is more in line with the goals I wanted to achieve. The test implementations could be written quickly in Cython and compiled and imported by runtests.py. Of course, when not regression testing, they'd be #defined as noops/Py_INCREF/Py_DECREF. Anyway, I always have too many ideas and too little time. But I thought I'd at least archive it somewhere if it seemed ok. -- Dag Sverre _______________________________________________ Cython-dev mailing list [email protected] http://codespeak.net/mailman/listinfo/cython-dev
