Steve Stagg <stest...@gmail.com> added the comment:
Ok, so I now understand a bit more, and think it's not a bug! But explaining it involves some fairly deep thinking about generators. I'll try to explain my reasoning. Let's take a simple example: --- def foo(): try: yield except: print("ERROR") for x in foo(): print(1) --- It's convenient to think of it as python adding an implicit throw StopIteration() at the end of the generator function, I'll also rewrite the code to be roughly equivalent: --- 1. def foo(): 2. try: 3. yield None 4. except: 5. print("ERROR") 6. #throw StopIteration(): 7. 8. foo_gen = foo() 9. while True: 10. try: 11. x = next(foo_gen) 12. except StopIteration: 13. break 14. print(1) 15. del foo_gen --- Now, if we step through how python runs the code starting at line 8.: 8. Create foo() <- this just creates a generator object 9. Enter while loop (not interesting for now) 10. try: 11. call next() on our generator to get the next value 2. try: <- we're running the generator until we get a yield now 3. yield None <- yield None to parent code 11. x = None <- assign yielded value to x 12. except StopIteration <- no exception raised so this doesn't get triggered 14. print(1) <- Print 1 to the output 9. Reached end of while loop, return to the start 10. try: 11. Re-enter the generator to get the next value, starting from where we left it.. 4. except: <- there was no exception raised, so this doesn't get triggered 6. (implicit) throw StopIteration because generator finished 12. `except StopIteration` <- this is triggered because generator threw StopIteration 13. break <- break out of while loop 15. remove foo_gen variable, and clean up generator. <- Deleting `foo_gen` causes `.close()` to be called on the generator which causes a GeneratorExit exception to be raised in the generator, BUT generator has already finished, so the GeneratorExit does nothing. -- This is basically how the for-loop in your example works, and you can see that there's no generator exception, BUT if we change the print(1) to print(i) and try again: 8. Create foo() <- this just creates a generator object 9. Enter while loop (not interesting for now) 10. try: 11. call next() on our generator to get the next value 2. try: <- we're running the generator until we get a yield now 3. yield None <- yield None to parent code 11. x = None <- assign yielded value to x 12. except StopIteration <- no exception raised so this doesn't get triggered *** CHANGED BIT *** 14. print(i) <- i doesn't exist, so throw NameError 14. The exception made us exit the current stack frame, so start cleaning up/deleting local variables <- Deleting `foo_gen` causes `.close()` to be called on the generator which causes GeneratorExit() to be raised within the generator, but the generator is currently paused on line 3. so raise exception as-if we're currently running line 3: 4. except: <- this broad except catches the GeneratorExit exception because it appears to have happened on line 3. 5. print("ERROR") <- We only get here if the above steps happened. --- So, if you don't let a generator naturally finish itself, but stop consuming the generator before it's raised its final StopIteration, then when the variable goes out-of-scope, a GeneratorExit will be raised at the point of the last yield that it ran. If you then catch that GeneratorExit, and enter a new un-consumed loop (as in your `yield from foo()` line), then that line will also create the same situation again in a loop.. I understand that this used to "work" in previous python versions, but actually, having dug into things a lot more, I think the current behaviour is correct, and previous behaviors were not correct. The "bug" here is in the example code that is catching GeneratorExit and then creating a new generator in the except:, rather than anything in Python ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue42762> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com