On Aug 25, 2009, at 6:16 PM, Kurt Smith wrote: > On Tue, Aug 25, 2009 at 4:21 PM, Fernando > Perez<[email protected]> wrote: >> Hi folks, >> >> On Tue, Aug 25, 2009 at 12:08 PM, Kurt Smith<[email protected]> >> wrote: >>> Weave-like support in Cython -- Fernando Perez (not at the BoF) >>> suggested that Cython could assume the same functionality of >>> scipy.weave (see: http://www.scipy.org/Weave), since scipy.weave is >>> poorly documented and could use more maintenance than it is >>> currently >>> getting. I'm personally very interested in this -- it seems that >>> there is much overlap between scipy.weave and the combination of >>> pyximport & the pure-python mode of Cython. It seems that with a >>> bit >>> of work from an interested user and some guidance from yours >>> truly, we >>> could improve pyximport & Cython's pure-python mode to >>> incorporate the >>> good stuff from scipy.weave, but better ;-) I intend to write-up a >>> CEP with my thoughts sometime in the near future. >>> >> >> Kurt, thanks for resurrecting this comment. Indeed, weave is an >> extremely useful piece of functionality but one that has been >> somewhat >> neglected for years. There's a lot of good in there though, and the >> need for it is significant. It's *really* cool to do things like: >> >> def prod(m, v): >> """Matrix-vector multiply via weave/blitz++""" >> nrows, ncolumns = m.shape >> assert v.ndim==1 and ncolumns==v.shape[0],"Shape mismatch in prod" >> >> res = np.zeros(nrows, float) >> code = r""" >> for (int i=0; i<nrows; i++) >> { >> for (int j=0; j<ncolumns; j++) >> { >> res(i) += m(i,j)*v(j); >> } >> } >> """ >> err = inline(code,['nrows', 'ncolumns', 'res', 'm', 'v'], >> verbose=2, >> type_converters=converters.blitz) >> return res > > Here's what I'm envisioning for the above usecase -- I admit to not > using the pure-python mode that much, so there may be errors (and some > of what I'm using isn't implemented yet). But it shows how much is > already there in Cython's pure-python mode. > > @cython.compile # this is new & where most of the work is required! > @cython.locals(m=cython.float[:,::1], v=cython.float[::1]) > def prod(m, v): > cython.declare( > i=cython.int, > j=cython.int, > nrows=cython.int, > ncolumns=cython.int) > nrows = m.shape[0]; ncolumns = m.shape[1] > cython.declare(res=cython.float[::1]) > res = np.zeros(nrows, np.float) > for i in range(nrows): > for j in range(ncolumns): > res[i] += m[i,j] * v[j] > return res > > Once the new array type / memoryview slice is merged into cython-devel > and we get it integrated into pure-python mode, this will be close to > working. If you just want things fast in the function body (what > weave gives you currently) then the above is all you need.
You beat me too it--this is almost exactly what I was going to write. (Just a note, it does understand the less verbose cython.declare (i=int, j=int, ...)). > The hard part is getting things to compile & run automatically for > just this function (the cython.compile decorator). This overlaps with > pyximport a good amount, and it will take some thought as to how it > will all work together. Perhaps the core devs won't like so much > going on behind the scenes -- cython is principally a utility for > generating a C extension module, not compiling & loading the extension > module. Personally, I think this is an awsome way to use Cython. Also, from the notebook one can create a Cython cell that does the same (compiles, loads, and spits the function right into your local namespace right from your browsers) which is really handy. Of course being able to make full modules with persistent globals and C functions that call each other is more powerful than just this. Even more, if cython is not available (or disabled for easier debugging, or no gcc was present) this could become a no-op, and would work just fine (though not as fast). > Weave compiles individual snippets into their own extension module, > whereas Cython has always compiled entire modules into extension > modules, and requires a separate compilation step (whether through the > commandline or through pyximport). The cython.compile decorator > would be different, if it were to be just like weave -- it would > compile a single function into its own extension module, without an > explicit compilation step. Much more is automated. > > This seems to be the main thing that weave offers -- fast code only > where you want it, without the explicit compilation step. Pyximport > overlaps here, but pyximport still compiles an entire module, not just > a part. > > So those are some of my thoughts. They'll be in a CEP sometime > soon :-) I have to say the one thing I don't like about weave is its use of strings to contain opaque code. OK, there's two things--the context switch between C and Fortran and Python. Nonetheless, I can see how it's really handy and a clever hack. I really want to avoid Cython allowing snippets of C and Fortran scattered in valid Cython sources if at all possible. (I would not be opposed to a cython.eval or cython.exec that would invoke the complier on a string). - Robert _______________________________________________ Cython-dev mailing list [email protected] http://codespeak.net/mailman/listinfo/cython-dev
