[issue17927] Argument copied into cell still referenced by frame
Guido van Rossum added the comment: Would you mind doing the backport then? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17927 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17927] Argument copied into cell still referenced by frame
Roundup Robot added the comment: New changeset 2b4b289c1abb by Benjamin Peterson in branch '3.3': when arguments are cells clear the locals slot (backport of #17927) http://hg.python.org/cpython/rev/2b4b289c1abb -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17927 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17927] Argument copied into cell still referenced by frame
Changes by Benjamin Peterson benja...@python.org: -- status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17927 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17927] Argument copied into cell still referenced by frame
Martin Morrison added the comment: The latest diff (cellfree4.diff) applies correctly to 3.3 (one hunk fails, but it's just the one where you've removed a blank line). The tests also pass successfully with the diff applied. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17927 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17927] Argument copied into cell still referenced by frame
Roundup Robot added the comment: New changeset f6223bab5657 by Benjamin Peterson in branch 'default': when an argument is a cell, set the local copy to NULL (see #17927) http://hg.python.org/cpython/rev/f6223bab5657 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17927 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17927] Argument copied into cell still referenced by frame
Guido van Rossum added the comment: Benjamin, what do you think of backporting this? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17927 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17927] Argument copied into cell still referenced by frame
Benjamin Peterson added the comment: I think with the way I rewrote it, it would be okay. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17927 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17927] Argument copied into cell still referenced by frame
Roundup Robot added the comment: New changeset d331e14cae42 by Guido van Rossum in branch 'default': #17927: Keep frame from referencing cell-ified arguments. http://hg.python.org/cpython/rev/d331e14cae42 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17927 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17927] Argument copied into cell still referenced by frame
Guido van Rossum added the comment: Ok, the deed is done. Thanks for your review, Benjamin! I've reassigned it to you so you can fix up the test if you want to. What would you think of a backport to 3.3? -- assignee: gvanrossum - benjamin.peterson resolution: - fixed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17927 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17927] Argument copied into cell still referenced by frame
Martin Morrison added the comment: Our usage of Python would certainly benefit from the backport. We are embedding Python 3.3 in a system with requirements that lead us to disable the gc in most cases, so cycles are an obvious problem for us. Cycles created inadvertently, such as this, are the biggest problem. We would probably backport the fix anyway, but would prefer not to carry patches and instead pick up fixes via 3.3.x releases. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17927 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17927] Argument copied into cell still referenced by frame
Guido van Rossum added the comment: Martin Morrison, can you check if my fix (cellfree4.diff) applies cleanly to 3.3? Or if nearly so, could you upload a fixed version here? (Or create a new issue and assign it to me if you can't upload to this issue.) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17927 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17927] Argument copied into cell still referenced by frame
Nick Coghlan added the comment: Ah, I misread the second patch, I think due to the copy the cell into in the comment. I believe I would have grasped it immediately if it said something like reference the cell from. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17927 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17927] Argument copied into cell still referenced by frame
Guido van Rossum added the comment: Ok, here's a version with a unittest. I've also improved the comment that set Nick off the track. -- keywords: +patch Added file: http://bugs.python.org/file30185/cellfree3.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17927 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17927] Argument copied into cell still referenced by frame
Benjamin Peterson added the comment: Consider the following testcase class X: def meth(self): print(self) super() def f(): k = X() def g(): return k return g c = f().__closure__[0] X.meth(c) With patch $ ./python unboxing.py cell at 0x7fddacab1eb8: X object at 0x7fddac7876d8 Without patch $ ./python unboxing.py cell at 0x7f2d0a218eb8: X object at 0x7f2d09eee6d8 Traceback (most recent call last): File x.py, line 12, in module X.meth(c) File x.py, line 4, in meth super() TypeError: super(type, obj): obj must be an instance or subtype of type Maybe you don't care. OTOH, perhaps it could be fixed by checking if the first argument is in fact a closure in super(). In the best world, super() would be syntax instead of a call, and we would just push the __class__ the closure and first argument in the interpreter loop. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17927 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17927] Argument copied into cell still referenced by frame
Guido van Rossum added the comment: I see. I really don't care, it's extremely rare to see a closure object. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17927 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17927] Argument copied into cell still referenced by frame
Guido van Rossum added the comment: cellfree4.diff. Addressed review: - Moved test to test_scope.py - Added @cpython_only - Fixed comment indent (and removed tabs :-) - Fixed missing NULL test I decided to keep the super() call; it's likely that it's tested elsewhere but I don't want to verify that there really is a test that puts self into a cell *and* uses super(). I also decided not to bother rewriting the test using weakrefs. Let me know if you really want me to do that. -- Added file: http://bugs.python.org/file30188/cellfree4.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17927 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17927] Argument copied into cell still referenced by frame
Benjamin Peterson added the comment: I think it looks good now. I'll probably just rewrite the test once you commit it. -- keywords: -needs review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17927 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17927] Argument copied into cell still referenced by frame
Nick Coghlan added the comment: Looks good to me, too. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17927 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17927] Argument copied into cell still referenced by frame
Changes by Phil Connell pconn...@gmail.com: -- nosy: +pconnell ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17927 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17927] Argument copied into cell still referenced by frame
Changes by Phil Connell pconn...@gmail.com: -- nosy: +isoschiz ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17927 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17927] Argument copied into cell still referenced by frame
Changes by Guido van Rossum gu...@python.org: -- stage: needs patch - patch review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17927 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17927] Argument copied into cell still referenced by frame
Guido van Rossum added the comment: Here's a new version that copies the cell into the arg slot instead of just clearing it, with matching code in super() that looks in the cell. I'd appreciate a review from another senior core dev. -- keywords: +needs review -patch Added file: http://bugs.python.org/file30176/cellfree2.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17927 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17927] Argument copied into cell still referenced by frame
Changes by Antoine Pitrou pit...@free.fr: -- nosy: +benjamin.peterson, brett.cannon, georg.brandl, ncoghlan ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17927 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17927] Argument copied into cell still referenced by frame
Changes by Mark Dickinson dicki...@gmail.com: -- nosy: +mark.dickinson ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17927 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17927] Argument copied into cell still referenced by frame
Guido van Rossum added the comment: Ok, if I don't hear from anyone soon I'm going to commit. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17927 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17927] Argument copied into cell still referenced by frame
Amaury Forgeot d'Arc added the comment: with a unit test maybe? -- nosy: +amaury.forgeotdarc ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17927 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17927] Argument copied into cell still referenced by frame
Antoine Pitrou added the comment: +1 for a unit test! -- nosy: +pitrou ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17927 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17927] Argument copied into cell still referenced by frame
Nick Coghlan added the comment: Guido, did you try combining your first patch (clearing the local var when populating the cell) with your second patch (by only checking for a cell when the local var is cleared rather than when it is populated)? It seems slightly more logical to me to have a cell fallback for the local ref is NULL case than it does to special case local ref is not NULL. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17927 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17927] Argument copied into cell still referenced by frame
Guido van Rossum added the comment: I thought about that but I like this version better because the super() code does not have to know the details of how to find the cell. On Wednesday, May 8, 2013, Nick Coghlan wrote: Nick Coghlan added the comment: Guido, did you try combining your first patch (clearing the local var when populating the cell) with your second patch (by only checking for a cell when the local var is cleared rather than when it is populated)? It seems slightly more logical to me to have a cell fallback for the local ref is NULL case than it does to special case local ref is not NULL. -- ___ Python tracker rep...@bugs.python.org javascript:; http://bugs.python.org/issue17927 ___ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17927 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17927] Argument copied into cell still referenced by frame
New submission from Guido van Rossum: This came out of some investigations into Tulip reference cycles. I've only confirmed this with 3.3 and 3.4, but I suspect it's a problem in earlier versions too. The scenario is as follows. Consider a object with a method that ends up catching an exception and storing the exception on the object. We know that the __traceback__ attribute of the exception references the stack frame where the exception was caught, so there is a cycle: self - exception - traceback - frame - self. To break this cycle without disturbing the __traceback__ on the exception, the method sets self = None before it returns. (The point of breaking the cycle is that at some later point when the object is deleted the traceback can be printed by the __del__ method.) This works beautifully... Except if the function happens to contain a nested function or a lambda that references 'self'. *Even if the function is never created* (e.g. if 0: lambda: self). Then setting self = None does not break the cycle. It's not a real leak, because gc.collect() will collect the cycle. But it's still annoying that I can't break the cycle (I don't want to break it at any other point because it would reduce the usefulness of the exception stored on the object). After two days of investigations and thinking about it I found the cause: the presence of the lambda cause a cell to be created into which self is copied, but the original self argument is still referenced by the frame. Setting self = None clears the cell but doesn't affect the original self argument. (FWIW, this indicates it's not specifically about self, it's about any argument that gets copied into a cell.) I thought I had a one-line fix (see cellfree.diff attached) but it breaks argument-less super(), which looks at the original first argument. I think I can fix super() (it must introspect the code object to find out into which cell self has been copied, if it finds it NULL), but I have to think about that more. If anyone wants to jump in and suggest an approach to that I'd appreciate it. -- assignee: gvanrossum components: Interpreter Core files: cellfree.diff keywords: patch messages: 188672 nosy: gvanrossum priority: normal severity: normal stage: needs patch status: open title: Argument copied into cell still referenced by frame type: resource usage versions: Python 3.3 Added file: http://bugs.python.org/file30166/cellfree.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17927 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17927] Argument copied into cell still referenced by frame
Changes by Yuval Greenfield ubershme...@gmail.com: -- nosy: +ubershmekel ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17927 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
