The state of a generator is not much more that a single Python stack frame plus an integer indicating where in the bytecode the resume point is. But copying/pickling a stack frame is complicated -- it's not just all the locals but also the try/except stack and the expression evaluation stack. Have a look here: https://github.com/python/cpython/blob/master/Include/frameobject.h. I'm not sure that I want to sign up for making all that stuff copyable (pickling will be an even harder challenge). But perhaps you (and/or another fearless hacker) are interested in trying?
Or were you just trying to see if the core dev team has spare cycles to implement this for you? --Guido On Tue, Jun 19, 2018 at 3:56 PM Micheál Keane <ffaristoc...@gmail.com> wrote: > > Add a function to generator objects to copy the entire state of it: > > Proposed example code: > > game1 = complicated_game_type_thing() > > # Progress the game to the first decision point > choices = game1.send(None) > > # Choose something > response = get_a_response(choices) > > # Copy the game generator > game2 = game1.copy() > > # send the same response to each game > x = game1.send(response) > y = game2.send(response) > > # verify the new set of choices is the same > assert x == y > > > History: > > I found this stackoverflow Q&A > <https://stackoverflow.com/questions/7180212/why-cant-generators-be-pickled> > which > among other things linked to an in-depth explanation of why generators > could not be pickled > <http://peadrop.com/blog/2009/12/29/why-you-cannot-pickle-generators/> and > this enhancement request for 2.6 <https://bugs.python.org/issue1092962> on > the bugtracker. All the reasons given there are perfectly valid.... but > they were also given nearly 10 years ago. It may be time to revisit the > issue. > > I couldn't turn up any previous threads here related to this so I'm > throwing it out for discussion. > > > Use case: > > My work involves Monte Carlo Tree Searches of games, eventually in > combination with tensorflow. MCTS involves repeatedly copying the state of > a simulation to explore the potential outcomes of various choices in depth. > > If you're doing a game like Chess or Go, a game state is dead simple to > summarize - you have a list of board positions with which pieces they have > and whose turn it is. > > If you're doing complex games that don't have an easily summarized state > at any given moment, you start running into problems. Think something > along the lines of Magic the Gathering with complex turn sequences between > players and effect resolutions being done in certain orders that are > dependent on choices made by players, etc. > > Generators are an ideal way to run these types of simulations but the > inability to copy the state of a generator makes it impossible to do this > in MCTS. > > As Python is being increasingly used for data science, this use case will > be increasingly common. Being able to copy generators will save a lot of > work. > > Keep in mind, I don't necessarily propose that generators should be fully > picklable; there are obviously a number of concerns and problems there. > Just being able to duplicate the generator's state within the interpreter > would be enough for my use case. > > > Workarounds: > > The obvious choice is to refactor the simulation as an iterator that > stores each state as something that's easily copied/pickled. It's probably > possible but it'll require a lot of thought and code for each type of > simulation. > > There's a Python2 package from 2009 called generator_tools > <https://pypi.org/project/generator_tools/> that purports to do this. I > haven't tried it yet to see if it still works in 2.x and it appears beyond > my skill level to port to 3.x. > > PyPy & Stackless Python apparently support this within certain limits? > > > Thoughts? > > > Washington, DC USA > ffaristoc...@gmail.com > _______________________________________________ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- --Guido van Rossum (python.org/~guido)
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/