On Jul 6, 2009, at 9:46 AM, Dag Sverre Seljebotn wrote: > Stefan Behnel wrote: >> Hi, >> >> should we let Cython know certain stdlib modules and consider >> their import >> as something that won't change meaning at runtime? >> >> This is mainly driven by loop optimisations. Imagine >> >> from itertools import izip >> >> always referred to the same thing. Then Cython could translate >> >> for a,b in izip(seq1, seq2): >> ... >> >> into (the equivalent of) >> >> while True: >> try: >> temp_a = seq1.next() >> temp_b = seq2.next() >> except StopIteration: >> break >> a,b = temp_a, temp_b >> ... >> >> instead of letting izip create a new iterator and tons of tuples >> along the >> way, plus the obvious optimisations for constant sequences, lists, >> dicts, >> etc. (Note that doing this for Py2's builtin zip() would not be >> correct as >> the sequences are allowed to change during iteration in that case). >> >> http://wiki.cython.org/enhancements/forin >> >> Or think of the math module and its constants and functions. We >> could use >> specialised C functions instead if we know the types that we are >> applying >> them to. That way, users wouldn't have to care about two sets of >> functions >> in Python's math module and C's math.h. >> >> Comments? > > This has been on my mind too for a while, and I'm undecided, or rather > ambient: I both love and hate the idea :-)
Same here :) > I guess it depends on the motivation. I'm sceptical of hit-or-miss > optimization in general -- that is, I don't think we can aim for > "magically optimizing" very much user code through single optimization > points like this. They are more for users who already know that their > code will be both interpreted in Python and compiled with Cython -- of > course, for such users an optimized izip could still be useful. > > I think my opinion is that I'd like to see this split this into a > couple > of orthogonal and generic features: > > a) Auto-cimport (if a directive is set, which defaults to on?). > I.e. if > you do a Python import, and a pxd file can be found, a cimport is done > automatically. For Sage, we need a way to turn this off. (A lot of stuff is in Cython, but doesn't need to be cimported, which could slow down the compilation time a lot). > b) Overloading combined with fall-through to Python space. I started a > CEP on this [1], Robert discussed it with me, but it never got > finished. > > One way is to allow e.g. (in "math.pxd") > > double sin(double x): return libc.math.sin(x) > object sin(object x) > > Somehow it must be added here though that sin should be looked up at > module initialization using getattr rather than through > __pyx_capi/Cython call convention. Danilo's working on function overloading right now, which would make this lots easier. (It's still unclear how to declare the non- optimized version). > c) For izip (and, in time, zip in __builtin__ too) I'm more in > favour of > a slightly magic optimization -- it's easier to be an expert on > hacking > pxd files than the Cython compiler. Namely: > > If a generator function contains exactly one "yield", and is declared > "inline", and used in a for-loop, then the entire generator will be > inlined into the function (putting the for-loop body at the > position of > the yield). Given the benefit and positive reaction we've had with the range optimization, I'm all for this as well. If the semantics could ever differ, we should allow it to be disabled, but the proposal above looks just fine. > [1] http://wiki.cython.org/enhancements/overlaypythonmodules, but I > wouldn't bother with it On that, I'm not so sure about the typemixin idea, and handling subclasses, but we did decide that "cdef list" means exactly a list (and not a subclass thereof) so we may be able to perform optimizations of this sort. - Robert _______________________________________________ Cython-dev mailing list [email protected] http://codespeak.net/mailman/listinfo/cython-dev
