On 4/16/06, Thomas Wouters <[EMAIL PROTECTED]> wrote:
This is caused by _threading_local.local's __del__ method, or rather the fact that it's part of a closure enclosing threading.enumerate . Fixing the inner __del__ to call enumerate (which is 'threading.enumerate') directly, rather than through the cellvar 'threading_enumerate', makes the leak go away. The fact that the leakage is inconsistent is actually easily explained: the 'local' instances are stored on the 'currentThread' object indexed by 'key', and keys sometimes get reused (since they're basically id(self)), so sometimes an old reference is overwritten. It doesn't quite explain why using the cellvar causes the cycle, nor does it explain why gc.garbage remains empty. I guess some Thread objects linger in threading's _active or _limbo dicts, but I have no idea why having a cellvar in the cycle matters; they seem to be participating in GC just fine, and I cannot reproduce the leak with a simpler test.
test_threading_local is not entirely consistent, but it looks a lot more reliable on my box than on Neal's automated mails:
test_threading_local
beginning 11 repetitions
12345678901
...........
test_threading_local leaked [34, 34, 34, 34, 34, 26, 26, 22, 34] references
This is caused by _threading_local.local's __del__ method, or rather the fact that it's part of a closure enclosing threading.enumerate . Fixing the inner __del__ to call enumerate (which is 'threading.enumerate') directly, rather than through the cellvar 'threading_enumerate', makes the leak go away. The fact that the leakage is inconsistent is actually easily explained: the 'local' instances are stored on the 'currentThread' object indexed by 'key', and keys sometimes get reused (since they're basically id(self)), so sometimes an old reference is overwritten. It doesn't quite explain why using the cellvar causes the cycle, nor does it explain why gc.garbage remains empty. I guess some Thread objects linger in threading's _active or _limbo dicts, but I have no idea why having a cellvar in the cycle matters; they seem to be participating in GC just fine, and I cannot reproduce the leak with a simpler test.
And on top of that, I'm not sure *why* _threading_local.local is doing the song and dance to get a cellvar. If the global 'enumerate' (which is threading.enumerate) disappears, it will be because Python is cleaning up. Even if we had a need to clean up the local dict at that time (which I don't believe we do), using a cellvar doesn't guarantee anything more than using a global name. Chances are very good that the 'threading' module has also been cleared, meaning that while we still have a reference to threading.enumerate, it cannot use the three globals it uses (_active, _limbo, _active_limbo_lock.) All in all, I think matters improve significantly if it just deals with the NameError it'll get at cleanup (which it already does.)
--
Thomas Wouters <[EMAIL PROTECTED]>
Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
