> I wonder if bypassing all of Cython except the parts we need for beta-3/4 is > a possibility. Then, when we have it working well enough, work on bringing > it back into Cython.
Things are changing rapidly on the Pyrex/Cython front. Let me summarize: Cython was founded with the primary goal of remaining compatible with Pyrex. We, likewise, wanted to remain compatible with Cython. Greg (the author of Pyrex) recently changed the syntax for "for" in Pyrex 0.9.7 and has said that he'll remove support for the old syntax in the next micro release (0.9.8). This means that code that compiled with 0.9.6.4 will not compile with 0.9.8 and produces numerous obnoxious depreciation warnings with 0.9.7 - this is an ongoing pattern and part of why we need to migrate away from Pyrex. Cython guys are obviously upset, because Greg didn't consult -anyone- for even opinions before doing so, a behavior many of us may recognize the author of Soya demonstrating frequently, and they are realizing that their goal of remaining Pyrex-compatible puts them at Greg's mercy as far as language changes. Several recent discussions in the Cython crowd about doing a fairly major rewrite and changing the Cython language in an incompatible way from Pyrex (to more closely mimic Python syntax). I agree with their language goals, but the core problem is that any work we do based on Cython.Compiler may need to be redone several times over the Summer, the PyMill repository is already severely out of sync with Cython's repository due to code being moved around on both ends. So in summary, I'm not worried about Pyrex or Cython compatibility at this point. I think we should do what we need to do for PySoy, what makes sense for *us*, and figure out what to do as the dust settles more around this Fall. > So basically it would generate the headers in a packagable and well-defined > place, and then create the C code implementation files that include their > own generated headers, and also any that were explicitly "cinclude"-ed? Mostly right There's three phases, the first two are part of distutils "build_ext" command: 1) Parse .pym sources and generate one .h file for each extension and potentially many .c files within the build/ directory directly (not to the system) 2) Compile .c files which include the headers generated in step 1 as needed (along with any headers installed on the system) and link them into one extension (for each instance of Extension) 3) Install the extension(s) and headers during the distutils install command Note that the headers cannot be generated directly to some system-wide path, this would break many conventions and make many people upset with us. > Maybe breaking this down into smaller tasks will help, at least > conceptually: I agree, but given the above, the steps need to be a bit more first - we need to define the language. this was not done with Pyrex, and we see what kind of mess it is internally second - build the main mill package from scratch, using the wisdom of the Cython guys (reverse engineering in some cases), even the people working on Cython agree it needs to be rewritten since the barrier to entry for new developers is obscenely high third - test small extensions, as you suggested, or possibly unit tests fourth - migrate PySoy's codebase, in whatever ways needed, to the language spec'ed above (changes should be MINOR!) fifth - debug PyMill using PySoy as a guide, and then optimisations on PyMill can continue while PySoy dev continues unhindered I'll note that we have a trac site for PyMill now at http://pysoy.org:8000/ _______________________________________________ PySoy-Dev mailing list [email protected] http://www.pysoy.org/mailman/listinfo/pysoy-dev
