[issue23720] __del__() order is broken since 3.4.0
Changes by Terry J. Reedy tjre...@udel.edu: -- nosy: -terry.reedy ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23720 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23720] __del__() order is broken since 3.4.0
Terry J. Reedy added the comment: While this is not a bug, Antoine said he might look at improving the situation, so I leave it to him to close it or not. -- nosy: +terry.reedy stage: - needs patch type: behavior - performance versions: -Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23720 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23720] __del__() order is broken since 3.4.0
Changes by Raymond Hettinger raymond.hettin...@gmail.com: -- assignee: - pitrou priority: normal - low resolution: - not a bug ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23720 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23720] __del__() order is broken since 3.4.0
Antoine Pitrou added the comment: Alexey, you're right that in this case (bug2.py) the cyclic GC is a bit less friendly than it was. It's not obvious there's a way to change that without introduce a lot of complexity. I'll try to take a look some day, although others may help too :-) That said, my advice in msg238680 still holds. When you're writing a Python wrapper around a C or C++ library with well-defined ownership relationships, I think you should enforce those in the Python wrapper as well (that's my experience with llvmlite, anyway). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23720 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23720] __del__() order is broken since 3.4.0
Changes by Antoine Pitrou pit...@free.fr: -- nosy: +tim.peters ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23720 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23720] __del__() order is broken since 3.4.0
Serhiy Storchaka added the comment: There is a cycle involving the class object, but I don’t think there is a cycle involving the instance objects this time. It is. a = A() a.__class__.__del__.__globals__['a'] __main__.A object at 0xb702230c And all objects referenced from user class or module level instance of user class are in a cycle. This problem can cause issues such as issue17852. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23720 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23720] __del__() order is broken since 3.4.0
Alexey Kazantsev added the comment: Ok, even assuming that all module globals are in circular reference starting with python 3.4, here is another example without using the globals: Brief description: v holds reference to d a.v = v b.d = d Now when we form a circular reference a - b, the destructor order becomes wrong for v and d. -- resolution: rejected - status: closed - open Added file: http://bugs.python.org/file38621/bug2.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23720 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23720] __del__() order is broken since 3.4.0
Serhiy Storchaka added the comment: Yes, in bug2.py we have different cycle. a ↔ b ↓ ↓ v → d a and b are in a cycle, and therefore v and d are in cycle. I think that in such case v always should be destroyed before d, independently of a cycle that refers them. And this is the same situation, as for io classes. A TextIOWrapper object refers a BufferedWriter object, a BufferedWriter object refers a FileIO object. and some cycle refers a TextIOWrapper object. As a result a FileIO object can be closed before a TextIOWrapper object or a BufferedWriter object flush its buffer. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23720 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23720] __del__() order is broken since 3.4.0
Martin Panter added the comment: But in the case of bug2.py, “a” is a variable inside main(), not a global variable. BTW I take back my second paragraph questioning the whole order thing; I clearly didn’t think that one through. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23720 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23720] __del__() order is broken since 3.4.0
Serhiy Storchaka added the comment: There is a cycle for every class with a method. class A: ... def __del__(self): pass ... A.__del__.__globals__['A'] class '__main__.A' -- nosy: +serhiy.storchaka ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23720 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23720] __del__() order is broken since 3.4.0
Martin Panter added the comment: There is a cycle involving the class object, but I don’t think there is a cycle involving the instance objects this time. However, I wonder if __del__() is meant to be called in any particular order anyway. What’s to stop the garbage collector itself from creating a temporary reference to the Device instance, destroying the Vector instance, which invokes Vector.__del__(), and finally destroying the temporary reference to the Device instance? -- nosy: +vadmium ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23720 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23720] __del__() order is broken since 3.4.0
Raymond Hettinger added the comment: Can this be closed as not-a-bug. -- nosy: +rhettinger ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23720 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23720] __del__() order is broken since 3.4.0
Antoine Pitrou added the comment: Amaury is right. In your case you could keep track of the Vectors in the Device object and invalidate them when the Device is destroyed (using e.g. a WeakSet). Or Vector could delegate its destruction to Device, e.g.: class Device(object): destroyed = False def __del__(self): self.destroyed = True def _dealloc_vector(self, v): if not self.destroyed: ... class Vector(object): def __init__(self, device): self.device = device def __del__(self): self.device._dealloc_vector(self) -- resolution: - rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23720 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23720] __del__() order is broken since 3.4.0
New submission from Alexey Kazantsev: Pythons prior to 3.4.0 print Vector! Device! while =3.4.0 print Device! Vector! If we replace Main with Vector on line 21, the behavior becomes random: in 50% of all cases it prints the wrong sequence, in other 50% the right. Our team treats this as a bug for several reasons: 1) Objects should be destroyed in breadth first reference tree traversal order, starting from the root. There are no cycles. It is nonsense to have freed children in parent's destructor. 2) Our applications suffer very much from this bug. Real Vector holds GPGPU memory and real Device holds the context, and CUDA/OpenCL require the context to be freed the last. With CUDA, the invalid destructor call order leads to segmentation faults. This may have something to deal with the implementation of PEP 442 (though in our case there no reference cycles at all). -- files: bug.py messages: 238661 nosy: Alexey Kazantsev priority: normal severity: normal status: open title: __del__() order is broken since 3.4.0 type: behavior versions: Python 3.4, Python 3.5 Added file: http://bugs.python.org/file38599/bug.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23720 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23720] __del__() order is broken since 3.4.0
Changes by Alexey Kazantsev ajk@gmail.com: -- components: +Interpreter Core ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23720 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23720] __del__() order is broken since 3.4.0
Changes by Amaury Forgeot d'Arc amaur...@gmail.com: -- nosy: +pitrou ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23720 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23720] __del__() order is broken since 3.4.0
Amaury Forgeot d'Arc added the comment: Actually there *is* a cycle: assert a.vector is a.vector.device.__class__.__del__.__globals__['a'].vector A workaround is to not store objects with __del__ in module globals... Or have only one (the Main instance in your case) -- nosy: +amaury.forgeotdarc ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23720 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23720] __del__() order is broken since 3.4.0
Vadim Markovtsev added the comment: +1 -- nosy: +Vadim Markovtsev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23720 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
