On Wed, 20 Aug 2008, Stefan Behnel wrote: > Hi, > > Robert Bradshaw wrote: >> I think we should rather support inline functions in .pxd files. Non- >> inline functions can be cimported. Includes should be much less used >> then the are now, but they have their place (when you actually want >> to textually include code in multiple places.) >> [...] >> I would *much* rather see a patch that lets distutils run a >> preprocesor (your choice of the C preprocessor or something more >> modern) rather than building more preprocessing into the language >> itself. > > Does it have to be distutils running the preprocessor? The use-cases I've seen > so far appeared to be based on an import or include, so I think it might be > enough to enable the preprocessor only for .pxd files (and maybe .pxi files, > depends on the targeted expressiveness of .pxd files). That way, any templates > would automatically end up in a separate file (where reuse is possible) and > could get configured from within Cython source code. IMHO, that sounds better > than inventing a new calling convention from within distutils. And in the > worst case, you can always write an almost empty .pyx file with a single > cimport, and /maybe/ an additional function call. :) > > We might also consider making the preprocessor interface pluggable, so that > people could register their own preprocessor for a DSL. That way, we'd keep > Cython itself free from any futuristic niche language extensions and would > just require that whatever people output from their preprocessors is valid > .pxd code. You could then ship your preprocessor in a separate package > directory of your project sources, or distribute it through PyPI and require > it in your setup.py. > > A plugin interface could work like this: > > - say this in Cython code: > cimport "abc_file.pxdx" a=1 b=... > where the generic .pxdx extension enables the preprocessing and the > keyword arguments are passed to the preprocessor plugin. I'm not sure > about the exact eval semantics here, being able to pass expressions > instead of evaluating them at compile time might be nice. Maybe the plugin > could get an eval callback into Cython and always do the evaluation > itself if it needs to. > > - have the source file start with a > #pre: a.b.xyz_plugin a=2 x="abc" ... > line that links it to the a.b.xyz_plugin and defines default arguments for > the plugin which get overridden by the arguments in the source code > > - let Cython import and invoke the a.b.xyz_plugin with the name of the file > and all keyword arguments expanded > > - let the a.b.xyz_plugin return a string that contains a parsable .pxd file, > which Cython will use to resolve the cimport in the .pyx file. > > - maybe add an additional error callback into the plugin that Cython could > invoke with a source position to let the plugin provide the 'real' source > lines that contained the error. > > Having other people experiment with their own preprocessors would make it much > easier to learn some best practices and cool features, and to fold them back > into some built-in preprocessor in the long term, while leaving all niche > features out of the core. > > Stefan
This is an interesting proposal, but I think it is much more heavyweight than I would like. It requires defining a preprocessor API, writing plugins in Python, etc. What about a directive like # cython: preprocess = [gcc -E | m4 | ...] This would pipe the file (be it a .pyx, .pxd, .pxi, .py, whatever) through the preprocessor before parsing, and only the output need be valid Cython code. - Robert _______________________________________________ Cython-dev mailing list [email protected] http://codespeak.net/mailman/listinfo/cython-dev
