On 3/27/06, Neal Norwitz <[EMAIL PROTECTED]> wrote:
I added this test, but I didn't remove test_tee, because it still leaks. I also checked in a test of code that used to leak, but doesn't anymore. The cycle this nested generator creates, which is also involved in the test_tee leak, is not cleanable by the cycle-gc, and it looks like it hasn't been since the yield-expr/coroutine patch was included in the trunk. I believe the culprit to be this part of that patch:
Index: Modules/gcmodule.cOn 3/27/06, Thomas Wouters <[EMAIL PROTECTED]> wrote:
>
> The teeobject has GC (hence the word 'and' in 'itertools.tee and its
> internal teedataobject' ;-) The problem with test_generators is that this
> also leaks:
>
> def leak():
> def gen():
> while True:
> yield g
> g = gen()
>
> leak()
Please add a test for this to the leakers and remove the tee test in
leakers if that no longer leaks.
I added this test, but I didn't remove test_tee, because it still leaks. I also checked in a test of code that used to leak, but doesn't anymore. The cycle this nested generator creates, which is also involved in the test_tee leak, is not cleanable by the cycle-gc, and it looks like it hasn't been since the yield-expr/coroutine patch was included in the trunk. I believe the culprit to be this part of that patch:
===================================================================
--- Modules/gcmodule.c (revision 39238)
+++ Modules/gcmodule.c (revision 39239)
@@ -413,10 +413,8 @@
assert(delstr != NULL);
return _PyInstance_Lookup(op, delstr) != NULL;
}
- else if (PyType_HasFeature(op->ob_type, Py_TPFLAGS_HEAPTYPE))
+ else
return op->ob_type->tp_del != NULL;
- else
- return 0;
}
/* Move the objects in unreachable with __del__ methods into `finalizers`.
Now, reverting that part of the patch in revision 39239 triggers an assertion failure, but removing the assertion makes it work right; the above nested-generator cycle gets cleaned up again. Later in the trunk, the assertion was changed anyway, and it no longer fails if I just revert the gcmodule change. Of course, the reason the cycle is uncleanable is because generators grew a tp_del method, and that throws cycle-gc in a hissy fit; reverting the above patch just makes cycle-gc ignore the tp_del of heaptypes. I don't know enough about the cycle-gc to say whether that's good or bad, but not having generators be cleanable is not very good. For what it's worth, reverting the gcmodule patch doesn't seem to break any tests.
However, fixing _both_ those things (itertools.tee lack of GC, and GC's inability to clean generators) *still does not fix the 254 refleaks in test_generators*. When I backport the itertools.tee-GC changes to r39238 (one checkin before coroutines), test_generators doesn't leak, neither the r39238 version of test_generators, nor the trunk version. One revision later, r39239, either test_generators leaks 15 references. 'Fixing' gcmodule, which fixes the nested-generator leak, does not change the number of leaks. And, as you all may be aware of, in the trunk, test_generators leaks 254 references; this is also the case with the gcmodule fix. So there's more huntin' afoot.
In the mean time, if people knowledgeable about the cycle-GC care to comment about the gcmodule change wrt. heaptypes, please do.
--
Thomas Wouters <[EMAIL PROTECTED] >
Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
