[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

Serhiy Storchaka added the comment: I think that's all with this issue. -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___ ___

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

Roundup Robot added the comment: New changeset bce18f5c0bc4 by Serhiy Storchaka in branch '3.5': Issue #23996: Added _PyGen_SetStopIterationValue for safe raising https://hg.python.org/cpython/rev/bce18f5c0bc4 New changeset a2c9f06ada28 by Serhiy Storchaka in branch '3.6': Issue #23996: Added _P

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

Changes by Guido van Rossum : -- nosy: -gvanrossum ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mai

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

Serhiy Storchaka added the comment: Added comments. -- Added file: http://bugs.python.org/file45355/gen_set_stopiteration_value_2.patch ___ Python tracker ___ ___

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

Serhiy Storchaka added the comment: Yet one special case -- if asynchronous iterator in aiter_wrapper is an instance of StopIteration. Proposed patch adds the function _PyGen_SetStopIterationValue() that raises StopIteration with correctly wrapped value (exception is normalized only if needed

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

Yury Selivanov added the comment: > No, this cannot be tested from the Python level. Stefan, could you please upload a C program that showcases the bug you're trying to fix? -- ___ Python tracker

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

Changes by Serhiy Storchaka : Added file: http://bugs.python.org/file45347/test_stopiteration_tuple_value.patch ___ Python tracker ___ ___ Py

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

Yury Selivanov added the comment: Serhiy, I think you forgot to attach the patch. aiter_wrapper shouldn't ever receive tuples, so it should be fine with PyErr_SetObject. -- ___ Python tracker

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

Serhiy Storchaka added the comment: Here is a test that passed with current code but will fail with the patch. I don't know whether it make much sense. If yes, then perhaps aiter_wrapper_iternext needs the same workaround as other invocations of PyErr_SetObject(PyExc_StopIteration, ...).

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

Yury Selivanov added the comment: > Looks like I forgot about this. My final fix still hasn't been applied, so > the code in Py3.4+ is incorrect now. Left a question in code review -- ___ Python tracker _

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

Changes by Serhiy Storchaka : -- type: crash -> behavior versions: +Python 3.7 ___ Python tracker ___ ___ Python-bugs-list mailing lis

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

Stefan Behnel added the comment: Looks like I forgot about this. My final fix still hasn't been applied, so the code in Py3.4+ is incorrect now. No, this cannot be tested from the Python level. -- ___ Python tracker

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

Serhiy Storchaka added the comment: Is it possible to test from Python level? -- ___ Python tracker ___ ___ Python-bugs-list mailing l

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

Stefan Behnel added the comment: Regarding tests, it looks like iteration isn't currently tested at the C level at all. At least, the xx test modules don't have any types that use it. I can write one up next week, or add it to one of the existing types (Xxo_Type?). Unlikely that I'll make the dea

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

Serhiy Storchaka added the comment: Could you provide tests covering all branches (normalized exception, unnormalized exception, absent value, non-tuple value, empty tuple value, non-empty tuple value...) Stefan? -- nosy: +serhiy.storchaka stage: patch review -> test needed __

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

Yury Selivanov added the comment: > Any of the last two patches ("*_value") will fix it, with my preference on > the last one. Stefan, the last patch looks good to me. Do you think we can have a unittest for this? -- ___ Python tracker

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

Guido van Rossum added the comment: Please try to make sure this is fixed before 3.5 rc 1. -- ___ Python tracker ___ ___ Python-bugs-l

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

Changes by Antoine Pitrou : -- nosy: +giampaolo.rodola, gvanrossum, haypo ___ Python tracker ___ ___ Python-bugs-list mailing list Uns

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

Stefan Behnel added the comment: The fix wasn't applied yet, so the current code in 3.4 and later branches is still incorrect. Any of the last two patches ("*_value") will fix it, with my preference on the last one. -- versions: +Python 3.6 ___ Pyth

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

Changes by Yury Selivanov : -- nosy: +yselivanov resolution: fixed -> stage: resolved -> patch review ___ Python tracker ___ ___ Pyth

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

Stefan Behnel added the comment: No. It's more that it feels wrong to spend actual time on the second most common case that can occur instead of just handling it in no time at all. The third case that it's really required to instantiate the StopIteration exception (if user code didn't do so al

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

Antoine Pitrou added the comment: Have you tried benchmarking the "slow" solution? -- ___ Python tracker ___ ___ Python-bugs-list mail

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

Changes by Stefan Behnel : Added file: http://bugs.python.org/file39692/fix_stopiteration_value.patch ___ Python tracker ___ ___ Python-bugs-l

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

Stefan Behnel added the comment: Here are two patches that fix this case, one with special casing, one without. Please choose and apply one. -- Added file: http://bugs.python.org/file39691/fix_stopiteration_value_slow.patch ___ Python tracker

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

Stefan Behnel added the comment: I noticed that my patch isn't entirely correct. If the exception value is a tuple, both PyErr_SetObject() and PyErr_NormalizeException() use it directly as *argument tuple* for the exception instantiation call, i.e. they essentially unpack it into separate argu

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

Antoine Pitrou added the comment: Thanks for the patch! -- resolution: -> fixed stage: -> resolved status: open -> closed versions: -Python 3.3 ___ Python tracker ___

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

Roundup Robot added the comment: New changeset 15c80f63ea1c by Antoine Pitrou in branch '3.4': Issue #23996: Avoid a crash when a delegated generator raises an unnormalized StopIteration exception. Patch by Stefan Behnel. https://hg.python.org/cpython/rev/15c80f63ea1c New changeset 9d0c6c66b0a

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

Changes by Stefan Behnel : -- nosy: +pitrou ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

Stefan Behnel added the comment: And in fact, fixing it in ceval.c would not be enough, since gen_throw() also calls the function. So this is really the right place to fix it. -- ___ Python tracker ___

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

Changes by Stefan Behnel : -- nosy: +ncoghlan ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.pyth

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

Stefan Behnel added the comment: And another patch update that should avoid any potential performance regressions due to the additional type check. -- Added file: http://bugs.python.org/file39119/fix_stopiteration_crash.patch ___ Python tracker

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

Stefan Behnel added the comment: Here's a better patch that avoids exception normalisation in all "normal" cases. -- Added file: http://bugs.python.org/file39116/fix_stopiteration_crash.patch ___ Python tracker ___

[issue23996] _PyGen_FetchStopIterationValue() crashes on unnormalised exceptions

New submission from Stefan Behnel: The yield-from implementation calls _PyGen_FetchStopIterationValue() to get the exception value. If the StopIteration exception is not normalised, e.g. because it was set by PyErr_SetObject() in a C extension, then _PyGen_FetchStopIterationValue() will cast t