Re: First release of Shed Skin, a Python-to-C++ compiler.
This is great news. Congratulations! By the way, I read in your blog that you would be releasing a windows intaller soon. Have you, or anyone else, managed to do it? Cheers, Luis -- http://mail.python.org/mailman/listinfo/python-list
Re: First release of Shed Skin, a Python-to-C++ compiler.
Mark Dufour wrote: In general it's considered quite pythonic to catch exceptions :-) It's a particularly useful way of implementing duck typing for example. I'm not sure if I've got *any* code that doesn't use exceptions somewhere Hehe. Okay. It will probably always be the case that you have to lose some Python features if you want the code to run really fast. I suppose PyPy's restricted Python subset doesn't support duck typing either. Luckily not all code is performance critical, or you could just try and optimize some performance critical part. But anyway, I'm starting to understand that Shed Skin should probably support exceptions wherever possible :-) Ok - the point I was trying to make was that exceptions were pretty integral to Python. I accept that losing the more dynamic features of Python for 'compilation' is a possibly worthwhile tradeoff. How easy is it going to be to call your c++ code from Python (and vice versa) ? Fuzzyman http://www.voidspace.org.uk/python/index.shtml -- http://mail.python.org/mailman/listinfo/python-list
First release of Shed Skin, a Python-to-C++ compiler.
Hehe. Okay. It will probably always be the case that you have to lose some Python features if you want the code to run really fast. I suppose PyPy's restricted Python subset doesn't support duck typing either. Luckily not all code is performance critical, or you could just try and optimize some performance critical part. But anyway, I'm starting to understand that Shed Skin should probably support exceptions wherever possible :-) Ok - the point I was trying to make was that exceptions were pretty integral to Python. I accept that losing the more dynamic features of Python for 'compilation' is a possibly worthwhile tradeoff. Shed Skin's main use will be in algorithmic-like code, which is also in most need of optimization. Typically such code is not very dynamic, or uses many exceptions, if any. It may be the case that most programs written in Python are not algorithmic-like at this point, which would be logical, since Python is traditionally not one of the fastest languages.. In that case applying Shed Skin to existing programs may not be very useful in most cases. But in any case, exceptions are not really dynamic (I think..) so, yeah, they should be supported.. How easy is it going to be to call your c++ code from Python (and vice versa) ? I haven't really thought deeply about this, but I guess it shouldn't be too hard to use existing C++/Python bridges there. However, I have no experience at all in this area.. (hint hint) thanks! mark. -- http://mail.python.org/mailman/listinfo/python-list
Re: First release of Shed Skin, a Python-to-C++ compiler.
Mark Dufour wrote: You're right, I don't feel safe about that. It's a bad example. I just prefer error codes, because the code usually becomes cleaner (at least to me). I would really like it if I could do something like this: f = file(name) if not f: print 'error opening file %s: %s', name, str(f.error) sys.exit() I think you meant: print 'error opening file %s: %s' % (name, str(f.error)) You forgot to check for an error when: o when you wrote f.error [attribute error might not exist e.g. f is None] o you called str(f.error) [might contain unicode characters that can't be converted to a string using the default encoding] o when you used the % operator [format string might be wrong] And, of course, pretty much every expression can result in a memory error. Exception handling is a boon because almost EVERY expression that you write can result in a error and checking each one is tedious in the extreme. So people forget and then their programs exhibit very odd behavior that is very difficult to debug. If you look at the Python C source, you'll notice that probably 50% of the code is devoted to error handling (that was a guess). Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: First release of Shed Skin, a Python-to-C++ compiler.
You forgot to check for an error when: o when you wrote f.error [attribute error might not exist e.g. f is None] o you called str(f.error) [might contain unicode characters that can't be converted to a string using the default encoding] o when you used the % operator [format string might be wrong] And, of course, pretty much every expression can result in a memory error. I don't mind the program bailing out in such a case and telling me where it went wrong, so I can fix the problem. I just don't really see the need for catching exceptions yourself. Especially not in well-tested code, potentially to be compiled. Exception handling is a boon because almost EVERY expression that you write can result in a error and checking each one is tedious in the extreme. So people forget and then their programs exhibit very odd behavior that is very difficult to debug. What do you mean odd behaviour? If they don't catch exceptions, the program will bail out, showing what went wrong, yeah? If you look at the Python C source, you'll notice that probably 50% of the code is devoted to error handling (that was a guess). That's a lot of error handling.. thanks! mark. -- http://mail.python.org/mailman/listinfo/python-list
Re: First release of Shed Skin, a Python-to-C++ compiler.
Mark Dufour wrote: [Brian Quinlan wrote:] You forgot to check for an error when: o when you wrote f.error [attribute error might not exist e.g. f is None] o you called str(f.error) [might contain unicode characters that can't be converted to a string using the default encoding] o when you used the % operator [format string might be wrong] And, of course, pretty much every expression can result in a memory error. I don't mind the program bailing out in such a case and telling me where it went wrong, so I can fix the problem. I just don't really see the need for catching exceptions yourself. Especially not in well-tested code, potentially to be compiled. grep the standard library. Tons of use-cases. It really is fundamental to the way most of us code Python. Exception handling is a boon because almost EVERY expression that you write can result in a error and checking each one is tedious in the extreme. So people forget and then their programs exhibit very odd behavior that is very difficult to debug. What do you mean odd behaviour? If they don't catch exceptions, the program will bail out, showing what went wrong, yeah? He was referring to the error code method of error handling. People forget to check the error code (or else wrongly assume that nothing wrong can happen). Uncaught exceptions *do* make the program bail out at the point of bogosity (more or less). If you look at the Python C source, you'll notice that probably 50% of the code is devoted to error handling (that was a guess). That's a lot of error handling.. It's code that needs to be robust but can't avail itself of exceptions. -- Robert Kern [EMAIL PROTECTED] In the fields of hell where the grass grows high Are the graves of dreams allowed to die. -- Richard Harter -- http://mail.python.org/mailman/listinfo/python-list
Re: First release of Shed Skin, a Python-to-C++ compiler.
Mark Dufour wrote: You forgot to check for an error when: o when you wrote f.error [attribute error might not exist e.g. f is None] o you called str(f.error) [might contain unicode characters that can't be converted to a string using the default encoding] o when you used the % operator [format string might be wrong] And, of course, pretty much every expression can result in a memory error. I don't mind the program bailing out in such a case and telling me where it went wrong, so I can fix the problem. I just don't really see the need for catching exceptions yourself. Especially not in well-tested code, potentially to be compiled. Exception handling is a boon because almost EVERY expression that you write can result in a error and checking each one is tedious in the extreme. So people forget and then their programs exhibit very odd behavior that is very difficult to debug. What do you mean odd behaviour? If they don't catch exceptions, the program will bail out, showing what went wrong, yeah? If you look at the Python C source, you'll notice that probably 50% of the code is devoted to error handling (that was a guess). That's a lot of error handling.. Mark: You have achieved so much with the first release of Shed Skin that it's strange to see you apparently trying to argue that exceptions aren't necessary when in fact they are such a fundamental part of Python's philosophy. Their real value is in removing the necessity to perform explicit error checking after each operation in a program. For example, when I write try: func1(...) func2(...) except SomeException: handle the exception the authors of func1()) and func2() don't have to worry about explicitly returning control if some condition they can't handle occurs. They just write raise SomeException knowing that this will immediately cause all calling contexts to be unstacked until the context is reached that is prepared to handle the exception. Without such an ability the logic of the functions tends to look something like if (errorCondition1): return SomeErrorFlag1 elif (errorCondition2): return SomeErrorFlag2 else: perform normal computation return Value and the calling code then has to check for the flags before handling the return value. So basically exceptions allow simplification of program structure with more effective and simpler error handling. If you read through (for example) the code you'll find in the standard library and you'll get a better feel for the usefulness of exceptions. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ -- http://mail.python.org/mailman/listinfo/python-list
First release of Shed Skin, a Python-to-C++ compiler.
You have achieved so much with the first release of Shed Skin that it's strange to see you apparently trying to argue that exceptions aren't necessary when in fact they are such a fundamental part of Python's philosophy. To be honest, I am a relative newcomer to Python, and Shed Skin is the first large program I have written in it. My background is mostly writing algorithms in C, and I don't think I've ever (voluntarily) caught an exception in my life, or that doing so could have made my code any cleaner. So for me personally, and for the types of programs I usually write, I'm still not sure if they will ever be very useful. I guess I have just naturally ignored them.. :-) Thank you for stressing the importance of exceptions in Python in general. I'm always happy to learn more about the language and its philosophy.. thanks! mark. -- http://mail.python.org/mailman/listinfo/python-list
First release of Shed Skin, a Python-to-C++ compiler.
Obviously, neither the 0 nor the message following should have been displayed. It's a pity that this assumption was made, but given the short time the project's been going I can understand it, hopefully Mark will continue towards greater python compliance :) The latter is certainly my goal. I just haven't looked into supporting exceptions yet, because I personally never use them. I feel they should only occur in very bad situations, or they become goto-like constructs that intuitively feel very ugly. In the 5500 lines of the compiler itself, I have not needed to use a single exception. For example, I prefer to check whether a file exists myself, rather than executing code that can suddenly jump somewhere else. There's probably some use for exceptions, but I don't (want to?) see it. Seeing how often exceptions are used by other people though, it's probably one of the first things I should look into.. :-) thanks! mark. -- http://mail.python.org/mailman/listinfo/python-list
Re: First release of Shed Skin, a Python-to-C++ compiler.
Mark Dufour wrote: Obviously, neither the 0 nor the message following should have been displayed. It's a pity that this assumption was made, but given the short time the project's been going I can understand it, hopefully Mark will continue towards greater python compliance :) The latter is certainly my goal. I just haven't looked into supporting exceptions yet, because I personally never use them. I feel they should only occur in very bad situations, or they become goto-like constructs that intuitively feel very ugly. In the 5500 lines of the compiler itself, I have not needed to use a single exception. For example, I prefer to check whether a file exists myself, rather than executing code that can suddenly jump somewhere else. There's probably some use for exceptions, but I don't (want to?) see it. Seeing how often exceptions are used by other people though, it's probably one of the first things I should look into.. :-) In general it's considered quite pythonic to catch exceptions :-) It's a particularly useful way of implementing duck typing for example. I'm not sure if I've got *any* code that doesn't use exceptions somewhere ;-) All the best, Fuzzyman http://www.voidspace.org.uk/python/index.shtml thanks! mark. -- http://mail.python.org/mailman/listinfo/python-list
Re: First release of Shed Skin, a Python-to-C++ compiler.
Mark Dufour wrote: The latter is certainly my goal. I just haven't looked into supporting exceptions yet, because I personally never use them. I feel they should only occur in very bad situations, or they become goto-like constructs that intuitively feel very ugly. In the 5500 lines of the compiler itself, I have not needed to use a single exception. For example, I prefer to check whether a file exists myself, rather than executing code that can suddenly jump somewhere else. There's probably some use for exceptions, but I don't (want to?) see it. I don't understand your example here. When you check that a file exists, you feel safe that openning it will succeed? What if: o you don't have permission to open the file o the file is deleted in the time between you checking for it's existance and opening it (possible race condition) o the system doesn't have sufficient resources to open the file e.g. too many open file handles o the file is already open with exclusive read/write permission Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: First release of Shed Skin, a Python-to-C++ compiler.
On 9/12/05, Brian Quinlan [EMAIL PROTECTED] wrote: Mark Dufour wrote: The latter is certainly my goal. I just haven't looked into supporting exceptions yet, because I personally never use them. I feel they should only occur in very bad situations, or they become goto-like constructs that intuitively feel very ugly. In the 5500 lines of the compiler itself, I have not needed to use a single exception. For example, I prefer to check whether a file exists myself, rather than executing code that can suddenly jump somewhere else. There's probably some use for exceptions, but I don't (want to?) see it. I don't understand your example here. When you check that a file exists, you feel safe that openning it will succeed? What if: o you don't have permission to open the file o the file is deleted in the time between you checking for it's existance and opening it (possible race condition) o the system doesn't have sufficient resources to open the file e.g. too many open file handles o the file is already open with exclusive read/write permission You're right, I don't feel safe about that. It's a bad example. I just prefer error codes, because the code usually becomes cleaner (at least to me). I would really like it if I could do something like this: f = file(name) if not f: print 'error opening file %s: %s', name, str(f.error) sys.exit() Error codes may be more work in some cases, but I really like the (subjectively) cleaner code. Again, exceptions are probably useful in some programs, but I wouldn't know.. :) thanks! mark. -- http://mail.python.org/mailman/listinfo/python-list
First release of Shed Skin, a Python-to-C++ compiler.
In general it's considered quite pythonic to catch exceptions :-) It's a particularly useful way of implementing duck typing for example. I'm not sure if I've got *any* code that doesn't use exceptions somewhere Hehe. Okay. It will probably always be the case that you have to lose some Python features if you want the code to run really fast. I suppose PyPy's restricted Python subset doesn't support duck typing either. Luckily not all code is performance critical, or you could just try and optimize some performance critical part. But anyway, I'm starting to understand that Shed Skin should probably support exceptions wherever possible :-) The main goal of Shed Skin is to be able to specify C++-like code at a higher level, not to be able to optimize arbitrary Python programs.. :-) For the kinds of things I write (algorithmic-like code), I really don't need the full flexibility of Python. It's just great to be able to leave out type declarations, and to use the beautiful Python syntax. ;-) All the best, thanks! mark. Fuzzyman http://www.voidspace.org.uk/python/index.shtml thanks! mark. -- http://mail.python.org/mailman/listinfo/python-list
Re: First release of Shed Skin, a Python-to-C++ compiler.
A.B., Khalid wrote: Mark Dufour wrote: After nine months of hard work, I am proud to introduce my baby to the world: an experimental Python-to-C++ compiler. Good work. I have good news and bad news. First the good news: ShedSkin (SS) more or less works on Windows. After patching gc6.5 for MinGW, building it, and testing it on WinXP with some succuess, and after patching my local copy of SS, I can get the test.py to compile from Python to C++, and it seems that I can get almost all the unit tests in unit.py to pass. I am reluctant to attempt an arduous installation on Windows, but if Mr. Dufour or someone else could create a web site that would let you paste in Python code and see a C++ translation, I think this would expand the user base. Alternatively, a Windows executable would be nice. -- http://mail.python.org/mailman/listinfo/python-list
First release of Shed Skin, a Python-to-C++ compiler.
I am reluctant to attempt an arduous installation on Windows, but if Mr. Dufour or someone else could create a web site that would let you paste in Python code and see a C++ translation, I think this would expand the user base. Alternatively, a Windows executable would be nice. The web site is a great idea, thanks. Maybe I will get to that, but I have so much other things on my TODO list.. The source code of the compiler can be downloaded, so anyone is free to tinker with it. The problem of making a Windows executable is that the compiler itself makes Windows executables, and therein lies the problem. It would be nice if somebody would come up with a complete package, because I have no access to Windows nor would I know how to approach that.. thanks! mark. -- http://mail.python.org/mailman/listinfo/python-list
Re: First release of Shed Skin, a Python-to-C++ compiler.
Hi! adDoc's networker Phil wrote: experimental Python-to-C++ compiler. why that instead of Pypy? . pypy compiles to llvm (low-level virtual machine) bytecode which is obviously not as fast as the native code coming from c++ compilers; PyPy can currently compile Python code to C code and to LLVM bytecode. Note that even for LLVM bytecode the argument is void since LLVM (despite its name, which might lead one to think that it is Java-like) compiles its bytecode to native assembler. but the primary mission of pypy is just having a python system that is written in something like python rather than c or c++ it's really just plain python (it completely runs on top of CPython after all) together with some restrictions -- which seem similar to the restictions that shedskin imposes btw. . there is no reason why the pypy project can't have a .NET architecture instead of the java-like arrangement I assume it has now Sorry, I can't really follow you here. In what way does PyPy have a Java-like arrangement? . without such a pypy.NET system, shedskin is offering a service that pypy can't yet provide: a ( python - c++ )-conversion allows me to smoothly integrate python contributions with my already-staggering c++ library . I'm not suggesting that pypy should be another Mono rewritten in python, because the essential mission of the .NET architecture is being able to compile any language of the user`s choice, to some intermediate language designed to be far more efficiently compiled to any machine language of the user`s choice than any human-readable language such as c++ . perhaps llvm bytecode can serve as such an intermediate language? then llvm could be the new c++ (our defacto IL (intermediate language)) and shedskin (python - IL=c++) could then be replaced by the combination of pypy (python - IL=llvm) and some incentive for all target platforms to develope a highly optimized ( llvm - native code)-compiler -- assuming also, that there is available a highly optimized ( c++ - llvm bytecode )-compiler . there is. look at the LLVM page for details: www.llvm.org Cheers, Carl Friedrich Bolz -- http://mail.python.org/mailman/listinfo/python-list
First release of Shed Skin, a Python-to-C++ compiler.
After nine months of hard work, I am proud to introduce my baby to the world: an experimental Python-to-C++ compiler. Wow, looks really cool. But why that instead of Pypy? I agree with anyone that a JIT compiler that supports the full Python semantics (which I thought to be the goal of PyPy?) is probably the best long term solution. It will really be a lot of work, however, and in general probably result in slower code than what my compiler produces, when it works (because of run-time overheads, lack of global optimizations.) Considering that Shed Skin does what it does in only about 7500 lines (5500 without C++ implementations of builtins), and is extreme in both the requirements it puts on the compiler and in the speed of the resulting code, for me personally it is just a very appealing alternative to investigate (I like extremes :-)) It should work for many programs of the type I usually write (algorithms, compilers..), so it's also a case of scratching my own itch. thanks! mark. -- http://mail.python.org/mailman/listinfo/python-list
Re: First release of Shed Skin, a Python-to-C++ compiler.
Carl Friedrich Bolz wrote: . there is no reason why the pypy project can't have a .NET architecture instead of the java-like arrangement I assume it has now Sorry, I can't really follow you here. In what way does PyPy have a Java-like arrangement? I imagine that this remark was made in reference to the just-in-time compilation techniques that PyPy may end up using, although I was under the impression that most CLR implementations also use such techniques (and it is possible to compile Java to native code as gcj proves). But on the subject of LLVM: although it seems like a very interesting and versatile piece of software, it also seems to be fairly difficult to build; my last attempt made the old-style gcc bootstrapping process seem like double-clicking on setup.exe. Does this not worry the PyPy team, or did I overlook some easier approach? (Noting that a Debian package exists for LLVM 1.4 but not 1.5.) Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: First release of Shed Skin, a Python-to-C++ compiler.
Mark Dufour wrote:
With this initial release, I hope to attract other people to help me
locate remaining problems,
Well, you did say you want help with locating problems. One problem with
this is it doesn't build...
If I try and build (following your instructions), I get presented with a
whole slew of build errors - knock on errors from the first few:
In file included from builtin_.cpp:1:
builtin_.hpp:4:29: gc/gc_allocator.h: No such file or directory
builtin_.hpp:5:23: gc/gc_cpp.h: No such file or directory
In file included from builtin_.cpp:1:
builtin_.hpp:89: error: syntax error before `{' token
builtin_.hpp:93: error: virtual outside class declaration
Which C++ libraries are you dependent on? (Stating this would be really
useful, along with specific versions and if possible where you got them :)
For reference, I'm building this on SuSE 9.3, under which I also have
boehm-gc-3.3.5-5 installed. I suspect you're using the same gc library
(having downloaded libgc from sourceforge and finding the includes don't
match the above include names) but a different version. For reference this
version/distribution of boehm-gc has the following file structure:
/usr/include/gc.h
/usr/include/gc_backptr.h
/usr/include/gc_config_macros.h
/usr/include/gc_cpp.h
/usr/include/gc_local_alloc.h
/usr/include/gc_pthread_redirects.h
/usr/lib/libgc.a
/usr/lib/libgc.la
/usr/lib/libgc.so
/usr/lib/libgc.so.1
/usr/lib/libgc.so.1.0.1
It's specifically the gc_cpp.h file that makes me suspect it's the same gc.
Regards,
Michael.
--
http://mail.python.org/mailman/listinfo/python-list
Re: First release of Shed Skin, a Python-to-C++ compiler.
Hi Paul! Paul Boddie wrote: Carl Friedrich Bolz wrote: . there is no reason why the pypy project can't have a .NET architecture instead of the java-like arrangement I assume it has now Sorry, I can't really follow you here. In what way does PyPy have a Java-like arrangement? I imagine that this remark was made in reference to the just-in-time compilation techniques that PyPy may end up using, although I was under the impression that most CLR implementations also use such techniques (and it is possible to compile Java to native code as gcj proves). Well, PyPy is still quite far from having a JIT build in. Plus the JIT-techniques will probably differ quite a bit from Java _and_ the CLR :-). But on the subject of LLVM: although it seems like a very interesting and versatile piece of software, it also seems to be fairly difficult to build; my last attempt made the old-style gcc bootstrapping process seem like double-clicking on setup.exe. Does this not worry the PyPy team, or did I overlook some easier approach? (Noting that a Debian package exists for LLVM 1.4 but not 1.5.) We are not that worried about this since a) building LLVM is not _that_ bad (you don't need to build the C-frontend, which is the really messy part) and b) the LLVM-backend is one of the more experimental backends we have anyway (in fact, we have discovered some bugs in LLVM with PyPy already). Since the C backend is quite stable we are not dependent solely on LLVM so this is not too big a problem. Note that this doesn't mean that the LLVM backend is not important: it's the only other backend (apart from the C one) that can succesfully translate the whole PyPy interpreter. Cheers, Carl Friedrich -- http://mail.python.org/mailman/listinfo/python-list
Re: First release of Shed Skin, a Python-to-C++ compiler.
Michael Sparks wrote: Well, you did say you want help with locating problems. One problem with this is it doesn't build... I found that I needed both the libgc and libgc-dev packages for my Kubuntu distribution - installing them fixed the include issues that you observed - and it does appear to be the Boehm-Demers-Weiser GC library, yes. The only other issue I observed was the importing of the profile and pstats modules which don't exist on my system, but those imports seemed to be redundant and could be commented out anyway. Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: First release of Shed Skin, a Python-to-C++ compiler.
Mark Dufour wrote: After nine months of hard work, I am proud to introduce my baby to the world: an experimental Python-to-C++ compiler. Good work. I have good news and bad news. First the good news: ShedSkin (SS) more or less works on Windows. After patching gc6.5 for MinGW, building it, and testing it on WinXP with some succuess, and after patching my local copy of SS, I can get the test.py to compile from Python to C++, and it seems that I can get almost all the unit tests in unit.py to pass. Here is what I used: 1. shedskin-0.0.1 2. pyMinGW patched and MinGW compiled Python 2.4.1 from CVS: Python 2.4.1+ (#65, Aug 31 2005, 22:34:14) [GCC 3.4.4 (mingw special)] on win32 Type help, copyright, credits or license for more information. 3. MinGW 3.4.4: g++ -v Reading specs from e:/UTILIT~1/PROGRA~1/MINGW/BIN/../lib/gcc/mingw32/3.4.4/specs Configured with: ../gcc/configure --with-gcc --with-gnu-ld --with-gnu-as --host=mingw32 --target=mingw32 --prefix=/mingw --enable-threads --disable-nls --enable-languages=c,c++,f77,ada,objc,java --disable-win32-registry --disable-shared --enable-sjlj-exceptions --enable-libgcj --disable-java-awt --without-x --enable-java-gc=boehm --disable-libgcj-debug --enable-interpreter --enable-hash-synchronization --enable-libstdcxx-debug Thread model: win32 gcc version 3.4.4 (mingw special) 4. Also using: - mingw-runtime 3.8 - w32api-3.3 - binutils-2.16.91-20050827-1 - gc6.5 (Bohem GC) locally patched Now the bad news. Four tests in Unit.py fail, brief output is as follows[1]. [SKIP 19532 lines] *** tests failed: 4 [(60, '__class__ and __name__ attributes'), (85, 'ifa: mixing strings and lists of strings in the same list'), (122, 'neural network simulator XXX later: recursive customization, plus some small fixes'), (124, 'small factorization program by Rohit Krishna Kumar')] Moreover, and since the GC system you used only works in recent versions of Windows, it follows that this solution will not work in all versions. I tested it on Win98 and both GC tests and SS's unit.py tests crash; although SS can still seem to compile the tests to C++. At any rate, if anyone is interested in the patches they can be downloaded from [2]. Regards, Khalid [1] The entire output of unit.py can also be found at [2] [2] http://jove.prohosting.com/iwave/ipython/Patches.html -- http://mail.python.org/mailman/listinfo/python-list
Re: First release of Shed Skin, a Python-to-C++ compiler.
Hi Mark! Mark Dufour wrote: After nine months of hard work, I am proud to introduce my baby to the world: an experimental Python-to-C++ compiler. Wow, looks really cool. But why that instead of Pypy? I agree with anyone that a JIT compiler that supports the full Python semantics (which I thought to be the goal of PyPy?) is probably the best long term solution. It will really be a lot of work, however, and in general probably result in slower code than what my compiler produces, when it works (because of run-time overheads, lack of global optimizations.) Although it is one of the goals of the PyPy project is producing a JIT for Python at some point in the future, it also contains a _static_ Python compiler right now: The whole source code of PyPy's interpreter is written in RPython, a restricted subset of the Python language. This subset is choosen in a way to make type inference possible and translate the code to a more low level language (at the moment C and LLVM are supported). This part of PyPy, the translator (http://codespeak.net/pypy/dist/pypy/doc/translation.html) has quite some similarities to Shed Skin. The kind of code that Shed Skin supports and RPython look quite similar (the biggest difference I see right now is that RPython supports exceptions). This was probably what Paul was referring to. The translator is needed to translate the whole PyPy interpreter into a low level language to get speed back, so Shed Skin and the translator exist for the same reason. Considering that Shed Skin does what it does in only about 7500 lines (5500 without C++ implementations of builtins), and is extreme in both the requirements it puts on the compiler and in the speed of the resulting code, for me personally it is just a very appealing alternative to investigate (I like extremes :-)) It should work for many programs of the type I usually write (algorithms, compilers..), so it's also a case of scratching my own itch. Shed Skin looks indeed quite impressive. If you are interested in discussing static translation of subsets of Python a bit or are even interested in getting to know PyPy a bit better feel free to contact us (http://codespeak.net/pypy/dist/pypy/doc/contact.html). Cheers, Carl Friedrich Bolz -- http://mail.python.org/mailman/listinfo/python-list
Re: First release of Shed Skin, a Python-to-C++ compiler.
Paul Boddie wrote: Michael Sparks wrote: Well, you did say you want help with locating problems. One problem with this is it doesn't build... I found that I needed both the libgc and libgc-dev packages for my Kubuntu distribution - installing them fixed the include issues that you observed - and it does appear to be the Boehm-Demers-Weiser GC library, yes. The only other issue I observed was the importing of the profile and pstats modules which don't exist on my system, but those imports seemed to be redundant and could be commented out anyway. Mark's also let me know this. Part of the problem is the version in SuSE 9.3 of the GC used is ancient - it should be version 6.5 onwards. Also for people compiling from source you (at minimum) should be using the configure line along the lines of: ./configure --enable-cplusplus If you don't, you get build problems because one of the needed libraries isn't built by default. I started off with the obvious hello world type program : print GAME OVER Which compiled cleanly and worked as expected. I then read Mark's short paper linked from his blog Efficient Implementation of Modern Imperative Languages; Application to Python, and got concerned by the comments: We have decided not to investigate two types of features: [...snip...]; and those features that may be turned off without affecting correct programs, e.g. array bounds checking, and exceptions That set some alarm bells ringing, largely because LBYL being deprecated by many people in favour of exceptions based code. (And more to the point, widely used as a result) As a result, I tried a trivial, but obvious program that should have clear behaviour: x = [] print GAME OVER x.append(5) print x[0] try: print x[1] print This should never be seen... except IndexError: print It's OK, we caught it... This compiles, but unfortunately has the following behaviour: GAME OVER 5 0 This should never be seen... It's OK, we caught it... Obviously, neither the 0 nor the message following should have been displayed. It's a pity that this assumption was made, but given the short time the project's been going I can understand it, hopefully Mark will continue towards greater python compliance :) Michael. -- http://mail.python.org/mailman/listinfo/python-list
Re: First release of Shed Skin, a Python-to-C++ compiler.
Carl Friedrich Bolz wrote: a) building LLVM is not _that_ bad (you don't need to build the C-frontend, which is the really messy part) That piece of wisdom must have passed me by last time, when I probably heeded the scary warning from the configure script and made the mistake of getting the C front end. This time, the build process was virtually effortless, and I'll now have to investigate LLVM further. Thanks for the tip! Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: First release of Shed Skin, a Python-to-C++ compiler.
Mark Dufour [EMAIL PROTECTED] writes: After nine months of hard work, I am proud to introduce my baby to the world: an experimental Python-to-C++ compiler. Wow, looks really cool. But why that instead of Pypy? -- http://mail.python.org/mailman/listinfo/python-list
Re: First release of Shed Skin, a Python-to-C++ compiler.
experimental Python-to-C++ compiler.why that instead of Pypy? . pypy compiles to llvm (low-level virtual machine) bytecode which is obviously not as fast as the native code coming from c++ compilers; but the primary mission of pypy is just having a python system that is written in something like python rather than c or c++ . there is no reason why the pypy project can't have a .NET architecture instead of the java-like arrangement I assume it has now . without such a pypy.NET system, shedskin is offering a service that pypy can't yet provide: a ( python - c++ )-conversion allows me to smoothly integrate python contributions with my already-staggering c++ library . I'm not suggesting that pypy should be another Mono rewritten in python, because the essential mission of the .NET architecture is being able to compile any language of the user`s choice, to some intermediate language designed to be far more efficiently compiled to any machine language of the user`s choice than any human-readable language such as c++ . perhaps llvm bytecode can serve as such an intermediate language? then llvm could be the new c++ (our defacto IL (intermediate language)) and shedskin (python - IL=c++) could then be replaced by the combination of pypy (python - IL=llvm) and some incentive for all target platforms to develope a highly optimized ( llvm - native code)-compiler -- assuming also, that there is available a highly optimized ( c++ - llvm bytecode )-compiler . -- American Dream Documentshttp://www.geocities.com/amerdreamdocs/home/(real opportunity starts with real documentation) -- http://mail.python.org/mailman/listinfo/python-list
First release of Shed Skin, a Python-to-C++ compiler.
After nine months of hard work, I am proud to introduce my baby to the
world: an experimental Python-to-C++ compiler. It can convert many
Python programs into optimized C++ code, without any user intervention
such as adding type declarations. It uses rather advanced static type
inference techniques to deduce type information by itself. In
addition, it determines whether deduced types may be parameterized,
and if so, it generates corresponding C++ generics. Based on deduced
type information, it also attempts to convert heap allocation into
stack and static preallocation (falling back to libgc in case this
fails.)
The compiler was motivated by the belief that in many cases it should
be possible to automatically deduce C++ versions of Python programs,
enabling users to enjoy both the productivity of Python and the
efficiency of C++. It works best for Python programs written in a
relatively static C++-style, in essence enabling users to specify C++
programs at a higher level.
At the moment the compiler correctly handles 124 unit tests, six of
which are serious programs of between 100 and 200 lines:
-an othello player
-two satisfiability solvers
-a japanese puzzle solver
-a sudoku solver
-a neural network simulator
Unfortunately I am just a single person, and much work remains to be
done. At the moment, there are several limitations to the type of
Python programs that the compiler accepts. Even so, there is enough of
Python left to be able to remain highly productive in many cases.
However, for most larger programs, there are probably some minor
problems that need to be fixed first, and some external dependencies
to be implemented/bridged in C++.
With this initial release, I hope to attract other people to help me
locate remaining problems, help implement external dependencies, and
in the end hopefully even to contribute to the compiler itself. I
would be very happy to receive small programs that the compiler does
or should be able to handle. If you are a C++ template wizard, and you
would be interested in working on the C++ implementation of builtin
types, I would also love to get in contact with you. Actually, I'd
like to talk to anyone even slightly interested in the compiler, as
this would be highly motivating to me.
The source code is available at the following site. Please check the
README for simple installation/usage instructions. Let me know if you
would like to create ebuild/debian packages.
Sourceforge site: http://shedskin.sourceforge.net
Shed Skin blog: http://shed-skin.blogspot.com
Should you reply to this mail, please also reply to me directly. Thanks!
Credits
Parts of the compiler have been sponsored by Google, via its Summer of
Code program. I am very grateful to them for keeping me motivated
during a difficult period. I am also grateful to the Python Software
Foundation for chosing my project for the Summer of Code. Finally, I
would like to thank my university advisor Koen Langendoen for guiding
this project.
Details
The following describes in a bit more detail various aspects of the
compiler. Before seriously using the compiler, please make sure to
understand especially its limitations.
Main Features
-very precise, efficient static type inference (iterative object
contour splitting, where each iteration performs the cartesian product
algorithm)
-stack and static pre-allocation (libgc is used as a fall-back)
-support for list comprehensions, tuple assignments, anonymous funcs
-generation of arbitrarily complex class and function templates
(even member templates, or generic, nested list comprehensions)
-binary tuples are internally analyzed
-some understanding of inheritance (e.g. list(dict/list) becomes
listpyiterA)
-hierarchical project support: generation of corresponding C++
hierarchy, including (nested) Makefiles; C++ namespaces
-annotation of source code with deduced types
-builtin classes, functions (enumerate, sum, min, max, range, zip..)
-polymorphic inline caches or virtual vars/calls (not well tested)
-always unbox scalars (compiler bails out with error if scalars are
mixed with pointer types)
-full source code available under the MIT license
Main Limitations/TODO's
-Windows support (I don't have Windows, sorry)
-reflection (getattr, hasattr), dynamic inheritance, eval, ..
-mixing scalars with pointer types (e.g. int and None in a single variable)
-mixing unrelated types in single container instance variable other
than tuple-2
-holding different types of objects in tuples with length 2;
builtin 'zip' can only take 2 arguments.
-exceptions, generators, nested functions, operator overloading
-recursive types (e.g. a = []; a.append(a))
-expect some problems when mixing floats and ints together
-varargs (*x) are not very well supported; keyword args are not supported yet
-arbitrary-size arithmetic
-possible non-termination ('recursive customization', have not
encountered it yet)
-profiling will be required
