[issue25612] nested try..excepts don't work correctly for generators

STINNER Victor added the comment: It seems like this issue introduced a regression in Python 3.7.0: bpo-33996 "Crash in gen_send_ex(): _PyErr_GetTopmostException() returns freed memory". -- ___ Python tracker

[issue25612] nested try..excepts don't work correctly for generators

Serhiy Storchaka added the comment: This change was in top5 breaking changes for 3.7. It broke many projects that use Cython until they upgraded Cython to the version that supports 3.7. -- ___ Python tracker

[issue25612] nested try..excepts don't work correctly for generators

Ned Deily added the comment: P.S. I added a very minimal What's New in 3.7 entry for this change since it has affected some downline projects. I just copied the NEWS entry. Feel free to expand it and/or move it to a better location in the file. --

[issue25612] nested try..excepts don't work correctly for generators

Ned Deily added the comment: New changeset ef5f4ba96e05f61ad3baca502012085f31e99342 by Ned Deily (Miss Islington (bot)) in branch '3.7': bpo-25612: Add minimal What's New in 3.7 entry (GH-7656) (GH-7658) https://github.com/python/cpython/commit/ef5f4ba96e05f61ad3baca502012085f31e99342

[issue25612] nested try..excepts don't work correctly for generators

Change by miss-islington : -- pull_requests: +7275 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue25612] nested try..excepts don't work correctly for generators

Ned Deily added the comment: New changeset 04290cb9945eca1a97f6924495256c15f29fab41 by Ned Deily in branch 'master': bpo-25612: Add minimal What's New in 3.7 entry (GH-7656) https://github.com/python/cpython/commit/04290cb9945eca1a97f6924495256c15f29fab41 --

[issue25612] nested try..excepts don't work correctly for generators

Change by Ned Deily : -- pull_requests: +7273 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue25612] nested try..excepts don't work correctly for generators

Ned Deily added the comment: New changeset 508d7693bc09affd99fdaa4a321cc3da0638c8a0 by Ned Deily (Miss Islington (bot)) in branch '3.7': bpo-33612: Remove PyThreadState_Clear() assertion (GH-7069) (GH-7074)

[issue25612] nested try..excepts don't work correctly for generators

STINNER Victor added the comment: New changeset b6dccf54fd3bac9c87348d96f9d6b571608c15bc by Victor Stinner in branch 'master': bpo-33612: Remove PyThreadState_Clear() assertion (#7069) https://github.com/python/cpython/commit/b6dccf54fd3bac9c87348d96f9d6b571608c15bc

[issue25612] nested try..excepts don't work correctly for generators

Change by miss-islington : -- pull_requests: +6705 ___ Python tracker ___

[issue25612] nested try..excepts don't work correctly for generators

Change by STINNER Victor : -- pull_requests: +6698 ___ Python tracker ___ ___

[issue25612] nested try..excepts don't work correctly for generators

Change by Chris Jerdonek : -- nosy: +chris.jerdonek ___ Python tracker ___ ___

[issue25612] nested try..excepts don't work correctly for generators

Serhiy Storchaka added the comment: Thank you for the clarification Mark. I agree that the current behavior is well justified. Cython was fixed in https://github.com/cython/cython/commit/2d3392463b77eb550bd54a69bd28cc161947acb5. -- status: open -> closed

[issue25612] nested try..excepts don't work correctly for generators

Mark Shannon added the comment: Looking at the docs: https://docs.python.org/3.6/library/sys.html#sys.exc_info states: If the current stack frame is not handling an exception, the information is taken from the calling stack frame, or its caller, and so on until a stack

[issue25612] nested try..excepts don't work correctly for generators

Serhiy Storchaka added the comment: The problem I mentioned in msg304117 has been resolved in backward direction: "raise" outside of an except block don't raise a RuntimeError. -- status: closed -> open ___ Python

[issue25612] nested try..excepts don't work correctly for generators

Serhiy Storchaka added the comment: Note that removing exc_type, exc_value and exc_traceback from PyThreadState breaks Cython. -- nosy: +scoder ___ Python tracker

[issue25612] nested try..excepts don't work correctly for generators

Antoine Pitrou added the comment: Thank you Mark for the fix! -- versions: -Python 2.7, Python 3.6 ___ Python tracker ___

[issue25612] nested try..excepts don't work correctly for generators

Antoine Pitrou added the comment: New changeset ae3087c6382011c47db82fea4d05f8bbf514265d by Antoine Pitrou (Mark Shannon) in branch 'master': Move exc state to generator. Fixes bpo-25612 (#1773) https://github.com/python/cpython/commit/ae3087c6382011c47db82fea4d05f8bbf514265d

[issue25612] nested try..excepts don't work correctly for generators

Mark Shannon added the comment: Thanks Serhiy for spotting that. 'raise' should raise the same exception as sys.exc_info() returns. I'll update the PR. -- nosy: +Mark.Shannon ___ Python tracker

[issue25612] nested try..excepts don't work correctly for generators

Serhiy Storchaka added the comment: I tried the following code: def g(): yield 1 raise yield 2 i = g() try: 1/0 except: next(i) next(i) Currently it raises: Traceback (most recent call last): File "", line 5, in File "", line 2, in

[issue25612] nested try..excepts don't work correctly for generators

Changes by Roundup Robot : -- pull_requests: +1854 ___ Python tracker ___

[issue25612] nested try..excepts don't work correctly for generators

Changes by Guido van Rossum : -- nosy: -gvanrossum ___ Python tracker ___ ___

[issue25612] nested try..excepts don't work correctly for generators

Nathaniel Smith added the comment: (Issue 29587 is a duplicate of this one, but it has some more information on where the !throwflag check came from and why it might be possible to remove it now.) -- ___ Python tracker

[issue25612] nested try..excepts don't work correctly for generators

Nathaniel Smith added the comment: I read the patch but I don't understand the logic behind it :-). Why should the value of tstate->exc_type affect whether we save/restore exc_info? Won't this mean that things are still broken for code like: - def f(): try: raise KeyError

[issue25612] nested try..excepts don't work correctly for generators

Changes by Andrew Svetlov : -- nosy: +asvetlov ___ Python tracker ___ ___

[issue25612] nested try..excepts don't work correctly for generators

Yury Selivanov added the comment: I closed issue #25683 as "not a bug". So it's just this issue that we need to fix. Anyone wants to review the patch? :) Since the code involved here is quite complex, I don't want to commit this patch without a thorough review. --

[issue25612] nested try..excepts don't work correctly for generators

Martin Panter added the comment: Nick, your scenario seems to behave no differently with and without the patch: Traceback (most recent call last): File "", line 2, in __main__.MainError -- ___ Python tracker

[issue25612] nested try..excepts don't work correctly for generators

Guido van Rossum added the comment: I'm sorry, I'm no help here -- I don't know how __context__ is supposed to work. :-( On Fri, Nov 20, 2015 at 8:14 AM, Yury Selivanov wrote: > > Yury Selivanov added the comment: > > Martin, you might be right here. Guido, do you

[issue25612] nested try..excepts don't work correctly for generators

Yury Selivanov added the comment: Anyways, here's a separate issue for the __context__ problem: http://bugs.python.org/issue25683 -- ___ Python tracker

[issue25612] nested try..excepts don't work correctly for generators

Yury Selivanov added the comment: Guido, Martin, I've just posted to python-dev about this ticket. Let's see what others think. -- ___ Python tracker

[issue25612] nested try..excepts don't work correctly for generators

Yury Selivanov added the comment: Martin, you might be right here. Guido, do you think it's OK that SubError doesn't have MainError in its __context__? http://bugs.python.org/issue25612#msg254576 -- ___ Python tracker

[issue25612] nested try..excepts don't work correctly for generators

Nick Coghlan added the comment: Interrogating the thread state like that makes me wonder how this patch behaves in the following scenario: class MainError(Exception): pass class SubError(Exception): pass def yield_coro(): yield coro = yield_coro() coro.send(None)

[issue25612] nested try..excepts don't work correctly for generators

Yury Selivanov added the comment: > Thinking about the __context__ thing some more, I guess it might make sense > for __context__ to be overwritten by the generator context. The same way it > gets overwritten when you execute “raise” inside an exception handler. Not sure I understand what

[issue25612] nested try..excepts don't work correctly for generators

Larry Hastings added the comment: I don't plan to hold up 3.5.1 for this. -- priority: release blocker -> normal ___ Python tracker ___

[issue25612] nested try..excepts don't work correctly for generators

Martin Panter added the comment: I was making an analogy between how the “raise” statement works, and how the throw() method could work. In this example, there are three exception objects (MainError, SubError, and ValueError). I was suggesting that it is okay for the context to be set to the

[issue25612] nested try..excepts don't work correctly for generators

Yury Selivanov added the comment: Can someone work with me on fixing this issue? I think it's important to ship 3.5.1 with this resolved (and 3.4.4 FWIW). -- ___ Python tracker

[issue25612] nested try..excepts don't work correctly for generators

Guido van Rossum added the comment: You might have to ping python-dev. But in terms of priorities I think it's not a blocker -- it's been broken for quite some time now, and it's a fairly odd corner of the language. -- ___ Python tracker

[issue25612] nested try..excepts don't work correctly for generators

STINNER Victor added the comment: > Can someone work with me on fixing this issue? I think it's important to > ship 3.5.1 with this resolved (and 3.4.4 FWIW). It don't consider this issue as a blocker for Python 3.5.1. This release already contains a *lot* of bugfixes! It's important to get

[issue25612] nested try..excepts don't work correctly for generators

Martin Panter added the comment: Thinking about the __context__ thing some more, I guess it might make sense for __context__ to be overwritten by the generator context. The same way it gets overwritten when you execute “raise” inside an exception handler. --

[issue25612] nested try..excepts don't work correctly for generators

Martin Panter added the comment: Regarding the second bug, did you consider that the exception thrown to the generator can already have __context__ set? try: try: raise ValueError("Context outside of generator") except ValueError as ex: raise SubError() from ex except SubError as ex:

[issue25612] nested try..excepts don't work correctly for generators

Changes by A. Jesse Jiryu Davis : -- nosy: +emptysquare ___ Python tracker ___ ___

[issue25612] nested try..excepts don't work correctly for generators

New submission from Yury Selivanov: Nested try..except statements with yields can loose reference to the current exception. The following code: class MainError(Exception): pass class SubError(Exception): pass def main(): try: raise MainError()

[issue25612] nested try..excepts don't work correctly for generators

Changes by Yury Selivanov : -- nosy: +serhiy.storchaka ___ Python tracker ___ ___

[issue25612] nested try..excepts don't work correctly for generators

STINNER Victor added the comment: Is this issue related to the issue #23353? -- ___ Python tracker ___ ___

[issue25612] nested try..excepts don't work correctly for generators

Changes by Yury Selivanov : -- versions: +Python 2.7 ___ Python tracker ___ ___

[issue25612] nested try..excepts don't work correctly for generators

Changes by STINNER Victor : -- nosy: +pitrou ___ Python tracker ___ ___

[issue25612] nested try..excepts don't work correctly for generators

Yury Selivanov added the comment: Here's a patch the fixes the first problem (but __context__ bug is still open). I'm not sure that the patch is correct :( But at least I've added new unittests (one still failing) -- keywords: +patch Added file:

[issue25612] nested try..excepts don't work correctly for generators

Yury Selivanov added the comment: This was originally discovered here: https://github.com/python/asyncio/issues/287 -- nosy: +brett.cannon ___ Python tracker

[issue25612] nested try..excepts don't work correctly for generators

STINNER Victor added the comment: I reverted the change of the issue #23353 but it doesn't fix this example, so it looks like these issues are not related. Cool. -- ___ Python tracker

[issue25612] nested try..excepts don't work correctly for generators

Yury Selivanov added the comment: Another bug: class MainError(Exception): pass class SubError(Exception): pass def main(): try: raise MainError() except MainError: yield coro = main() coro.send(None)