2010/12/5 Stefan Behnel <stefan...@behnel.de>:
> Hi,
>
> I just noticed that Vitja's closure refactoring brought us closer to
> supporting generator expressions (before supporting generators and
> coroutines). Here's what's missing:
>
> 1) transform a GeneratorExpressionNode into a DefNode or a subclass
> (obviously excluding inlined generator expressions)
>
> 2) generate the resume code at the start of the function body
>
> 3) store temps in closures. For simplicity, this can be done with a tuple
> (which supports NULL values to a certain extent). That way, we do not need
> to declare each temp separately as a closure field. The remaining tricky
> bit is then to figure out the maximum number of temps that are alive at a
> YieldExprNode (ok, that's actually easy) and to pass that number on to the
> closure class that needs to allocate a sufficiently large tuple.
>
> That doesn't sound too hard.
>
> The generators CEP has the details for each step.
>
> http://wiki.cython.org/enhancements/generators
>
> Going for generator expressions before implementing generators has the
> advantage that it gives us most of the infrastructure without requiring the
> additional steps of passing initial arguments into the generator and
> implementing the complete coroutine protocol (the yield nodes cannot return
> values, for example).
>
> Anyone with a little time to spare?
>
> Stefan
> _______________________________________________
> Cython-dev mailing list
> Cython-dev@codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>
Hi!
I've started woring on generators on friday.
Now it can generate some code, but it doesn't work yet, what is done:
- switch() for resuming and argument parsing
- yield stuff
- closure temp allocation
I'm going to put all the generator stuff into closure
Generator function:
Creates closure with all temps, args, variables, and all the internal stuff:
Also closure have all the required generator methods __iter__(),
__nextiter__() and so on.
def my_generator(self, args, kwargs):
closure = my_generator_closure_new()
closure.is_running = 0
closure.resume_label = 0
closure.args = args
closure.kwargs = kwargs
closure.outer_scope = self
closure.generator_body = my_generator_body
return closure
generator body:
def my_generator_body(closure, value, is_exception):
# switch case
switch (closure.resume_lable):
case 0:
goto parse_arguments;
case 1:
goto resume_from_yield1;
parse_arguments:
# function body here
About temps:
now temps are allocated in closure using declare_var() and it works fine.
I'm not sure about manage_ref flag I guess that temps with
manage_ref=False shouldn't be allocated on closure (am I right?)
I'm not sure that tuple should be better for temps.
TODO:
- create C function that returns closure
- generator utility code
I have problem with closures and generators.
Now I want to implement closure structure this way:
struct abstract_generator {
PY_OBJECT_HEAD
/* generator internal vars */
PyObject *(*body)(struct closure *, PyObject *, int);
int resume_label;
}
struct closure {
/* abstract_generator */
PY_OBJECT_HEAD
/* generator internal vars */
PyObject *(*body)(struct closure *, PyObject *, int);
int resume_label;
,,,,
/* Closure stuff */
} ;
and generator utility function will work like this:
PyObject *abstract_generator_next(PyObject *self, PyObject *args)
{
struct abstract_generator *generator = (struct abstract_generator *) self;
PyObject *retval;
// check running
retval = generator->body(self, None, 0);
// unlock running
return retval;
}
But this way should be much better, don't know if that is possible btw.
struct closure {
struct abstract_generator generator;
/* closure stuff here */
...
} ;
I add MarkGenerator visitor, that sets is_generator flag on DefNode
and checks for returns/yields.
Then CreateClosureClasses will create correct closure class for generator.
--
vitja.
_______________________________________________
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev
