I wanted to sound out a couple things. First, I couldn't find any real discussion about it after 2011 so I had no idea if the reasons it was ruled unfeasible with Python 2 still held nearly 10 years later with Python 3. I was mainly wondering if all the recent asynchronous work had changed things significantly. Apparently not?
Secondly, one SO comment had included the suggestion that it be posted to this list - my searching couldn't find it ever having been done so here it is. Finally, another comment made the point that there wasn't a strong use case given for it. With the data science libraries that have sprung up around Python in the intervening years, I believe there now is one. Washington, DC USA ffaristoc...@gmail.com On Wed, Jun 20, 2018 at 12:25 AM, Guido van Rossum <gu...@python.org> wrote: > The state of a generator is not much more that a single Python stack frame > plus an integer indicating where in the bytecode the resume point is. But > copying/pickling a stack frame is complicated -- it's not just all the > locals but also the try/except stack and the expression evaluation stack. > Have a look here: https://github.com/python/cpython/blob/master/Include/ > frameobject.h. I'm not sure that I want to sign up for making all that > stuff copyable (pickling will be an even harder challenge). But perhaps you > (and/or another fearless hacker) are interested in trying? > > Or were you just trying to see if the core dev team has spare cycles to > implement this for you? > > --Guido > > On Tue, Jun 19, 2018 at 3:56 PM Micheál Keane <ffaristoc...@gmail.com> > wrote: > >> >> Add a function to generator objects to copy the entire state of it: >> >> Proposed example code: >> >> game1 = complicated_game_type_thing() >> >> # Progress the game to the first decision point >> choices = game1.send(None) >> >> # Choose something >> response = get_a_response(choices) >> >> # Copy the game generator >> game2 = game1.copy() >> >> # send the same response to each game >> x = game1.send(response) >> y = game2.send(response) >> >> # verify the new set of choices is the same >> assert x == y >> >> >> History: >> >> I found this stackoverflow Q&A >> <https://stackoverflow.com/questions/7180212/why-cant-generators-be-pickled> >> which >> among other things linked to an in-depth explanation of why generators >> could not be pickled >> <http://peadrop.com/blog/2009/12/29/why-you-cannot-pickle-generators/> and >> this enhancement request for 2.6 <https://bugs.python.org/issue1092962> on >> the bugtracker. All the reasons given there are perfectly valid.... but >> they were also given nearly 10 years ago. It may be time to revisit the >> issue. >> >> I couldn't turn up any previous threads here related to this so I'm >> throwing it out for discussion. >> >> >> Use case: >> >> My work involves Monte Carlo Tree Searches of games, eventually in >> combination with tensorflow. MCTS involves repeatedly copying the state of >> a simulation to explore the potential outcomes of various choices in depth. >> >> If you're doing a game like Chess or Go, a game state is dead simple to >> summarize - you have a list of board positions with which pieces they have >> and whose turn it is. >> >> If you're doing complex games that don't have an easily summarized state >> at any given moment, you start running into problems. Think something >> along the lines of Magic the Gathering with complex turn sequences between >> players and effect resolutions being done in certain orders that are >> dependent on choices made by players, etc. >> >> Generators are an ideal way to run these types of simulations but the >> inability to copy the state of a generator makes it impossible to do this >> in MCTS. >> >> As Python is being increasingly used for data science, this use case will >> be increasingly common. Being able to copy generators will save a lot of >> work. >> >> Keep in mind, I don't necessarily propose that generators should be fully >> picklable; there are obviously a number of concerns and problems there. >> Just being able to duplicate the generator's state within the interpreter >> would be enough for my use case. >> >> >> Workarounds: >> >> The obvious choice is to refactor the simulation as an iterator that >> stores each state as something that's easily copied/pickled. It's probably >> possible but it'll require a lot of thought and code for each type of >> simulation. >> >> There's a Python2 package from 2009 called generator_tools >> <https://pypi.org/project/generator_tools/> that purports to do this. I >> haven't tried it yet to see if it still works in 2.x and it appears beyond >> my skill level to port to 3.x. >> >> PyPy & Stackless Python apparently support this within certain limits? >> >> >> Thoughts? >> >> >> Washington, DC USA >> ffaristoc...@gmail.com >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas@python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > > > -- > --Guido van Rossum (python.org/~guido) >
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/