[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Chris Jerdonek]

Chris Jerdonek  added the comment:


New changeset d7184d3dbd249444ec3961641dc08a9ad3c1ac34 by Chris Jerdonek in 
branch 'master':
bpo-29587: Add another test for the gen.throw() fix. (GH-19859)
https://github.com/python/cpython/commit/d7184d3dbd249444ec3961641dc08a9ad3c1ac34


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Chris Jerdonek]

Chris Jerdonek  added the comment:


New changeset 75cd8e48c62c97fdb9d9a94fd2335be06084471d by Chris Jerdonek in 
branch 'master':
bpo-29587: Make gen.throw() chain exceptions with yield from (GH-19858)
https://github.com/python/cpython/commit/75cd8e48c62c97fdb9d9a94fd2335be06084471d


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Chris Jerdonek]

Chris Jerdonek  added the comment:

Would someone be able to approve / take a look at this small PR addressing 
another aspect of this issue?
https://github.com/python/cpython/pull/19858
I'm hoping it can be part of 3.9, and the deadline is just a week away.

It builds on the already merged PR to address an "Example 3" of this issue (the 
"yield from" case as opposed to the "yield" case, which the merged PR 
addressed):

Example 3:

def f():
yield

def g():
try:
raise KeyError('a')
except Exception:
yield from f()

gen = g()
gen.send(None)
gen.throw(ValueError)

--
Output without PR:

Traceback (most recent call last):
  File "/.../test-example3.py", line 12, in 
gen.throw(ValueError)
  File "/.../test-example3.py", line 8, in g
yield from f()
  File "/.../test-example3.py", line 2, in f
yield
ValueError

--
Output with PR:

Traceback (most recent call last):
  File "/.../test-example3.py", line 6, in g
raise KeyError('a')
KeyError: 'a'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/.../test-example3.py", line 12, in 
gen.throw(ValueError)
  File "/.../test-example3.py", line 8, in g
yield from f()
  File "/.../test-example3.py", line 2, in f
yield
ValueError

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Chris Jerdonek]

Change by Chris Jerdonek :


--
pull_requests: +19266
pull_request: https://github.com/python/cpython/pull/19858

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[STINNER Victor]

STINNER Victor  added the comment:


New changeset b0be6b3b94fbdf31b796adc19dc86a04a52b03e1 by Victor Stinner in 
branch 'master':
bpo-29587: _PyErr_ChainExceptions() checks exception (GH-19902)
https://github.com/python/cpython/commit/b0be6b3b94fbdf31b796adc19dc86a04a52b03e1


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[STINNER Victor]

STINNER Victor  added the comment:

tl; dr I wrote PR 19902 to remove the "XXX" comment.

--

Commit 21893fbb74e8fde2931fbed9b511e2a41362b1ab adds the following code:

/* XXX It seems like we shouldn't have to check not equal to Py_None
   here because exc_type should only ever be a class.  But not including
   this check was causing crashes on certain tests e.g. on Fedora. */
if (gen->gi_exc_state.exc_type && gen->gi_exc_state.exc_type != Py_None) { 
... }

I don't think that you should mention "Fedora" as a platform impacted by this 
issue: all platforms should be affected. It's just unclear why the issue was 
first seen on Fedora.

gen_send_ex() copies tstate->exc_info to gen->gi_exc_state.

tstate->exc_info->exc_type is set at least in the following 3 places in CPython 
code base:

* ceval.c: POP_EXCEPT opcode: exc_info->exc_type = POP();
* ceval.c: UNWIND_EXCEPT_HANDLER() macro: exc_info->exc_type = POP();
* errors.c: PyErr_SetExcInfo()

I saw in practice POP_EXCEPT and UNWIND_EXCEPT_HANDLER() setting 
exc_info->exc_type to Py_None (I didn't test PyErr_SetExcInfo()).

_PyErr_GetTopmostException() also handles exc_info->exc_type == Py_None case:

_PyErr_StackItem *
_PyErr_GetTopmostException(PyThreadState *tstate)
{
_PyErr_StackItem *exc_info = tstate->exc_info;
while ((exc_info->exc_type == NULL || exc_info->exc_type == Py_None) &&
   exc_info->previous_item != NULL)
{
exc_info = exc_info->previous_item;
}
return exc_info;
}

--

So exc_type=None is not a bug, but it's done on purpose.

If you don't want to get None in genobject.c, we should modify all places which 
set tstate->exc_info->exc_type. Problem: the structure currently exposed in the 
public C API (bpo-40429), and I wouldn't be surprised if Cython or greenlet 
modify tstate->exc_info directly.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +19216
pull_request: https://github.com/python/cpython/pull/19902

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Chris Jerdonek]

Chris Jerdonek  added the comment:

Since none of you are subscribed to the other issue except for Nathaniel, I 
thought I'd let you know I also posted a PR to fix another issue he filed 
similar to this one (but this time about the stack trace being incorrect for 
`gen.throw()` in certain circumstances): https://bugs.python.org/issue29590

That issue is the second of the two issues he cited back in 2017 about 
`gen.throw()` being broken:
https://vorpus.org/~njs/misc/trio-language-summit-2017.pdf

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Chris Jerdonek]

Chris Jerdonek  added the comment:


New changeset 21893fbb74e8fde2931fbed9b511e2a41362b1ab by Chris Jerdonek in 
branch 'master':
bpo-29587: allow chaining NULL exceptions in _gen_throw() (GH-19877)
https://github.com/python/cpython/commit/21893fbb74e8fde2931fbed9b511e2a41362b1ab


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Chris Jerdonek]

Chris Jerdonek  added the comment:

Okay, I was able to remove the NULL value check and get things working with a 
NULL exception value. I just posted a second follow-on PR that does this (which 
Guido approved - thanks, Guido!):
https://github.com/python/cpython/pull/19877

I wanted to tell you what I found since it raises some questions.

To remove that check I had to add the following new check prior to calling 
`_PyErr_ChainExceptions()` with the exception info, as a replacement:
`gen->gi_exc_state.exc_type != Py_None`
Without doing this, code like the following would crash e.g. on Fedora 32 (this 
is the crash that was happening in the first version of my PR, reduced down):

def g():
try:
raise KeyError
except KeyError:
pass

try:
yield
except Exception:
# Crash happens here e.g. on Fedora 32 but not Mac.
raise RuntimeError

gen = g()
gen.send(None)
gen.throw(ValueError)

This raises two questions for me:

First, I thought exc_type could only ever be NULL or an exception class. So I'm 
wondering if this points to a problem elsewhere in the code base.

Second, I don't know why it would crash on Fedora but not Mac. (On Mac, you 
instead see the following exception chained beneath the ValueError:
> TypeError: print_exception(): Exception expected for value, NoneType found )

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Chris Jerdonek]

Change by Chris Jerdonek :


--
pull_requests: +19189
pull_request: https://github.com/python/cpython/pull/19877

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Chris Jerdonek]

Chris Jerdonek  added the comment:

FYI, I was able to finish addressing the other exception-chaining cases (the 
ones covered by the other issue) with a simple change to the fix for this PR.

I moved the call to _PyErr_ChainExceptions() a bit deeper: into gen_send_ex() 
before the call to _PyEval_EvalFrame(), and behind an `if (exc)` check.

So I think the tracebacks should improve a lot.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Chris Jerdonek]

Chris Jerdonek  added the comment:

FYI, I just created a PR to add one more test to the prior fix, to test a 
slightly different aspect.

Whereas the test in the first PR checks that exc.__context__ is set after 
gen.throw() is finished, the new test checks that exc.__context__ is available 
also from within the generator (while the generator is running). I'm adding 
this test partly because this behavior is different from the "awaiting on a 
task" case, which I'm working on next.

Both of these tests were failing prior to the fix, which I confirmed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Chris Jerdonek]

Change by Chris Jerdonek :


--
pull_requests: +19175
pull_request: https://github.com/python/cpython/pull/19859

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Guido van Rossum]

Guido van Rossum  added the comment:

Wow, thanks all! FWIW I agree that ideally things should work with a NULL
value...
-- 
--Guido (mobile)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Chris Jerdonek]

Chris Jerdonek  added the comment:

Okay, thanks everyone. I think I'll take a look at issue 40466 next.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Chris Jerdonek]

Chris Jerdonek  added the comment:


New changeset 02047265eb83a43ba18cc7fee81756f1a1a1f968 by Chris Jerdonek in 
branch 'master':
bpo-29587: Update gen.throw() to chain exceptions (#19823)
https://github.com/python/cpython/commit/02047265eb83a43ba18cc7fee81756f1a1a1f968


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Chris Jerdonek]

Chris Jerdonek  added the comment:

By the way, I just posted a new issue about exception chaining getting 
suppressed, but this time in an asyncio setting with ensure_future():
https://bugs.python.org/issue40466

I first noticed this issue two or three years ago and raised it on the 
async-sig list. It was suggested there that this was a special case of the 
current issue. Having written code to fix the current issue, though, the 
ensure_future() issue still exists. So I filed a separate issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Chris Jerdonek]

Chris Jerdonek  added the comment:

Okay, I was able to get the tests passing on the other platforms.

I'm still not sure why Mac and Fedora behaved differently with my original PR. 
I was able to come up with a simple script that isolated the behavior 
difference (posted in the PR comments). It's very strange. Maybe it signals an 
issue elsewhere in the Python code base.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Guido van Rossum]

Guido van Rossum  added the comment:

Could it be running out of memory due to excessive chaining of tracebacks?

(And yes, it's 3.9 only.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Yury Selivanov]

Yury Selivanov  added the comment:

IMO this is a 3.9 fix.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Ned Deily]

Ned Deily  added the comment:

Whatever the resolution of this is, it seems to me that this sort of behavior 
change should not be introduced at this stage of 3.7.x's life.  Whether it 
should go into 3.8.x should be Łukasz's call once the final change is in master 
and has stabilized.

--
nosy: +ned.deily
versions:  -Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Chris Jerdonek]

Change by Chris Jerdonek :


--
pull_requests: +19143
stage: resolved -> patch review
pull_request: https://github.com/python/cpython/pull/19823

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Chris Jerdonek]

Chris Jerdonek  added the comment:

> Oh, no! Buildbot failures. Help?!

Yes, I'm sorry. There's something not so simple going on that I haven't yet 
reproduced locally. Will reply on the tracker.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[STINNER Victor]

STINNER Victor  added the comment:

> Sorry, I had to revert this change since it broke like 41+ buildbots: (...)

If someone wants to investigate, you can find examples of failed buildbot 
builds at:
https://github.com/python/cpython/pull/19811

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Guido van Rossum]

Change by Guido van Rossum :


--
resolution: fixed -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[STINNER Victor]

STINNER Victor  added the comment:

> bpo-29587: Enable implicit exception chaining with gen.throw() (GH-19811)
> https://github.com/python/cpython/commit/2514a632fb7d37be24c2059d0e286d35600f9795

Sorry, I had to revert this change since it broke like 41+ buildbots:
https://pythondev.readthedocs.io/ci.html#revert-on-fail

Almost all CIs passed on the PR (except of the Ubuntu job of Azure Pipelines). 
That's unusual. No idea why the bug occurs only on some platforms and why it 
wasn't seen before.

The revert is an opportunity to get more time to investigate the issue, without 
having the pressure to have to fix the CI "ASAP".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 3c7f9db85095952821f9d106dd874f75662ce7ec by Victor Stinner in 
branch 'master':
Revert "bpo-29587: Enable implicit exception chaining with gen.throw() 
(GH-19811)" (#19821)
https://github.com/python/cpython/commit/3c7f9db85095952821f9d106dd874f75662ce7ec


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[STINNER Victor]

Change by STINNER Victor :


--
nosy: +vstinner
nosy_count: 6.0 -> 7.0
pull_requests: +19141
pull_request: https://github.com/python/cpython/pull/19821

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Guido van Rossum]

Guido van Rossum  added the comment:

Oh, no! Buildbot failures. Help?!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Guido van Rossum]

Guido van Rossum  added the comment:

Example 1 is now fixed too by Chris Jerdonek's PR 19811. Thanks Chris!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Guido van Rossum]

Guido van Rossum  added the comment:


New changeset 2514a632fb7d37be24c2059d0e286d35600f9795 by Chris Jerdonek in 
branch 'master':
bpo-29587: Enable implicit exception chaining with gen.throw() (GH-19811)
https://github.com/python/cpython/commit/2514a632fb7d37be24c2059d0e286d35600f9795


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Chris Jerdonek]

Chris Jerdonek  added the comment:

I made a naive attempt at addressing this issue here:
https://github.com/python/cpython/pull/19811

The code has diverged significantly since Nathaniel first linked to a specific 
line above. However, what I came up with is simple, and seems to work. But I 
could very well be missing some things.

It would be great if this could be fixed in some way.

--
versions: +Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Chris Jerdonek]

Change by Chris Jerdonek :


--
keywords: +patch
pull_requests: +19131
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19811

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Yury Selivanov]

Yury Selivanov  added the comment:

Guido,

The second example ("No active exception to reraise") was an actual long open 
bug.  It was recently fixed by Mark Shannon in [1].

I think we should be able to fix the first case (missing __context__) although 
it's not critical.  I'll look into it after the context PEP and few other open 
asyncio issues.

[1] https://github.com/python/cpython/pull/1773

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Guido van Rossum]

Guido van Rossum  added the comment:

Yury do you agree this is a bug? Do you have any ideas on how to fix it?

--
nosy: +gvanrossum, yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Hynek Schlawack]

Change by Hynek Schlawack :


--
nosy: +hynek

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Chris Jerdonek]

Change by Chris Jerdonek :


--
nosy: +chris.jerdonek

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Martin Panter]

Martin Panter added the comment:

The second example seems like the original complaint in Issue 25612. That 
spawned a separate bug related to the first situation, which was later closed: 
Issue 25683.

--
nosy: +martin.panter
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[Nathaniel Smith]

New submission from Nathaniel Smith:

Example 1:

---
def f():
try:
raise KeyError
except Exception:
yield

gen = f()
gen.send(None)
gen.throw(ValueError)
-

Output:
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 5, in f
ValueError

Expected output:

Something involving the string "During handling of the above exception, another 
exception occurred", and a traceback for the original KeyError.


Example 2:

---
import sys

def f():
try:
raise KeyError
except Exception:
print(sys.exc_info())
try:
yield
except Exception:
pass
print(sys.exc_info())
raise

gen = f()
gen.send(None)
gen.throw(ValueError)
---

Output:

(, KeyError(), )
(None, None, None)
Traceback (most recent call last):
  File "/tmp/foo.py", line 17, in 
gen.throw(ValueError)
  File "/tmp/foo.py", line 13, in f
raise
RuntimeError: No active exception to reraise

Expected output: certainly not that :-)


This seems to happen because normally, generators save the current exc_info 
when yielding, and then restore it when re-entered. But, if we re-enter through 
'throw' (throwflag is true), this is disabled:

https://github.com/python/cpython/blob/b2ee40ed9c9041dcff9c898aa19aacf9ec60308a/Python/ceval.c#L1027

This check seems to have been added in ae5f2f4a39e6a3f4c45e9dc95bd4e1fe5dfb60f2 
as a fix for:

https://bugs.python.org/issue7173

which had to do with a nasty situation involving a generator object that  was 
part of a reference cycle: the gc sometimes would free the objects stored in 
the generator's saved exc_info, and then try to clean up the generator by 
throwing in a GeneratorExit.

AFAICT this situation shouldn't be possible anymore thanks to PEP 442, which 
makes it so that finalizers are run before any part of the cycle is freed. And 
in any case it certainly doesn't justify breaking code like the examples above.

(Note: the examples use generators for simplicity, but of course the way I 
noticed this was that I had some async/await code where exceptions were 
mysteriously disappearing instead of showing up in __context__ and couldn't 
figure out why. It's likely that more people will run into this in the future 
as async/await becomes more widely used. As a workaround for now I'll probably 
modify my coroutine runner so that it never uses 'throw'.)

--
messages: 287987
nosy: njs
priority: normal
severity: normal
status: open
title: Generator/coroutine 'throw' discards exc_info state, which is bad
versions: Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com