At 07:08 AM 7/24/2010 -0700, Guido van Rossum wrote:
- After seeing Raymond's talk about monocle (search for it on PyPI) I
am getting excited again about PEP 380 (yield from, return values from
generators). Having read the PEP on the plane back home I didn't see
anything wrong with it, so it could just be accepted in its current
form.

I would like to reiterate (no pun intended) the suggestion of a special syntactic form for the return, such as "yield return x", or "return with x" or something similar, to distinguish it from a normal generator return.

I think that when people are getting used to the idea of generators, it's important for them to get the idea that the function's "return value" isn't really a value, it's an iterator object. Allowing a return value, but then having that value silently disappear, seems like it would delay that learning, so, a special form might help to make it clear that the generator in question is intended for use with a corresponding "yield from", and help avoid confusion on this.

(I could of course be wrong, and would defer to anyone who sees a better way to explain/teach around this issue. In any event, I'm +1 on the PEP otherwise.)

By the way, the PEP's "optimized" implementation could probably be done just by making generator functions containing yield-from statements return an object of a different type than the standard geniter. Here's a Python implementation sketch, using a helper class and a decorator -- translation to a C version is likely straightforward, as it'll basically be this plus a light sprinkling of syntactic sugar.

So, in the pure-Python prototype (without syntax sugaring), usage would look like this:

    @From.container
    def some_generator(...):
        ...
        yield From(other_generator(...))  # equivalent to 'yield from'
        ...

    def other_generator(...):
        ...
        raise StopIteration(value)  # equivalent to 'return value'


We mark some_generator() with @From.container to indicate that it uses 'yield from' internally (which would happen automatically in the C/syntax sugar version). We don't mark other_generator(), though, because it doesn't contain a 'yield from'.

Now, the implementation code (a slightly altered/watered-down version of a trampoline I've used before in 2.x, hopefully altered correctly for Python 3.x syntax/semantics):

class From:
    @classmethod
    def container(cls, func):
        def decorated(*args, **kw):
return cls(func(*args, **kw)) # wrap generator in a From() instance
        return decorated

    def __new__(cls, geniter):
        if isinstance(geniter, cls):
            # It's already a 'From' instance, just return it
            return geniter
        self = object.__new__(cls, geniter)
        self.stack = [geniter]
        return self

    def __iter__(self):
        return self

    def __next__(self):
        return self._step()

    def send(self, value):
        return self._step(value)

    def throw(self, *exc_info):
        return self._step(None, exc_info)

    def _step(self, value=None, exc_info=()):
        if not self.stack:
            raise RuntimeError("Can't resume completed generator")
        try:
            while self.stack:
                try:
                    it = self.stack[-1]
                    if exc_info:
                        try:
                            rv = it.throw(*exc_info)
                        finally:
                            exc_info = ()
                    elif value is not None:
                        rv = it.send(value)
                    else:
                        rv = it.next()
                except:
                    value = None
                    exc_info = sys.exc_info()
                    if exc_info[0] is StopIteration:
                        # pass return value up the stack
                        value, = exc_info[1].args or (None,)
                        exc_info = ()   # but not the error
                    self.stack.pop()
                else:
                    if isinstance(rv, From):
                        stack.extend(rv.stack)  # Call subgenerator
                        value, exc_info, rv = None, (), None
                    else:
                        return rv   # it's a value to yield/return
            else:
                # Stack's empty, so exit w/current return value or error
                if exc_info:
                    raise exc_info[1]
                else:
                    return value
        finally:
            exc_info = ()   # don't let this create garbage

    def close(self):
        if self.stack:
            try:
                # There's probably a cleaner way to do this in Py 3, I just
                # don't know what it is off the top of my head...
                raise GeneratorExit
            except GeneratorExit as e:
                try:
                    self.throw(*sys.exc_info())
                except (StopIteration, GeneratorExit):
                    pass
                else:
                    raise RuntimeError("Generator(s) failed to close()")
                    # XXX probably needs some more code here to clean up stack

    def __del__(self):
        try:
            self.close()
        except:
            pass

As you can (hopefully) see, the main code path simply ends up delegating next/send/close etc. directly to the iterator on the top of the stack, so there isn't any multi-layer passthru going on, as in the "non-optimized" version of the spec in the PEP.

_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Reply via email to