Feel like posting any links to source code? Thanks, Jared
On Wed, Jul 21, 2010 at 11:57 AM, Kay Hayen <[email protected]> wrote: > > Hello Cython-World, > > some time ago, I found Cython and wanted to use it. I even came up with > a couple of patches, which I believe didn't get merged, mostly because > it soon became apparent that we have not the same goals completely. > > My interest and goal was to use Cython for my pure Python code. I > believe that Python is the language and that using language extensions > that turn it to another language is not what I want. > > Obviously, this is not exactly what Cython is about. So I thought of > having my own "toy compiler" which translates pure Python into C or > another language which is then compiled. > > Time is short for me, I have a day job and everything, but this is to > inform you that I believe I have succeeded with it. > > That is, I can take pure Python 2.5 syntax and compile it to C++ and it > does pass most of the CPython tests. It is capable of generator > expressions and list comprehensions, selection or/and, etc. in all its > beauty. It does lambda expressions, decorators for classes and > functions. It does exception handling, args with "**" and "*", > conditional expressions, and most interestingly, it is capable of > generator functions (yield). > > Well, just everything. You can imagine, I learned a lot about Python in > the process. > > My initial approach was to start from CPython bytecode. (Stefan, you > likely recall that discussion.) Turned out that bytecode got me very > far, also in understanding how CPython works, but for some cases, I just > wasn't happy. I was just almost everything. I wanted everything though. > > Then I used the parser module of Python itself, and that went much > better. Of course the semantics was often new to me, and there is always > somebody who uses a syntax in just one test. > > But I managed to pull through that. BTW: Did you know [ x,y for x in > range(4) for y in range(5) ] and could you predict its value? How often > is each iterated being created? Deeply confusing stuff. I also > absolutely enjoy these strangenesses. > > Currently, I have for the first release 4 goals. > > a) Pass the CPython tests even more completely. Some decorators e.g. use > the function decorated also as a dictionary to remember things. > > My "compiled function" type doesn't allow that yet. > > I do handle eval, exec, locals, globals, execfile, etc. correctly, but > there is stuff I will never have, e.g. a "func_code" object, or an > execution frame. I find it very convenient to have the CPython tests, I > also have some of my own, but being able use "unittest" was a real break > through, you can surely imagine. > > b) I never compiled the compiler with the compiler. I surely want to do > that. > > c) Whole program support. I currently compile module by module from > ".py" to ".so" or ".exe". I can recursively compile into os.path, but > currently don't have the code to make it actually run. > > d) Detect dead code. Really, the compiler is a result of solo extreme > programming. I at least want the compiler to tell me about unused code > and test coverage. > > That would be milestone 1 and address primarily correctness. How fast it > is, I didn't care much yet. In some cases faster than CPython and > Cython, but that is not the point yet. Currently libpython does almost > all the hard work, and what I mostly do, is to eliminate the bytecode > generation and interpretation, and replace it with C++. Depending on the > program, that is close to nothing. > > The idea though, is clearly to get to the point, where I have it > optimize the C type cases. > > I expect that a release achieving these 3 goals will happen at the end > of August. I hope to be able to build on your experiences as well, and I > also hope to convince you to accept my idea of a "hints" module with > decorators that should at least optionally replace the cdef syntax. > > My core idea there, is that if I say "@hints.returns( int )" or > something like this, it would decorate the function into something that > asserts that this is really true, where for the compiler or Cython, the > use of hints.returns() could be more in place. > > Probably also interesting will be an exchange of optimization methods, > although I have not ventured there much already. I did e.g. compile > pystone only to find how much faster CPython was. Then I stole the > specialization from them. And I guess, I can learn a lot more from > Cython too. I am sure, I have a couple of tricks now, that I will gladly > share after my release. > > You clearly invested a lot of effort to see what's fast and what is not, > if only because to me the C-API looks alike and you have tested those > already. > > Regarding newer CPython, I already use Python 2.6 syntax, but not the > "with" statement, and I didn't look at 2.7 at all yet. I will probably > only make the jump to there after the release. I do not care much about > Python 3.x at this point. > > Best regards, > Kay Hayen > _______________________________________________ > Cython-dev mailing list > [email protected] > http://codespeak.net/mailman/listinfo/cython-dev >
_______________________________________________ Cython-dev mailing list [email protected] http://codespeak.net/mailman/listinfo/cython-dev
