Re: PyPy 1.0: JIT compilers for free and more
In article [EMAIL PROTECTED], Jarek Zgoda [EMAIL PROTECTED] wrote: Carl Friedrich Bolz napisa³(a): Welcome to the PyPy 1.0 release - a milestone integrating the results of four years of research, engineering, management and sprinting efforts, concluding the 28 months phase of EU co-funding! So it took 4 yars of work and over 2 yaers of consumption of EU funds, yeah, great. Could anybody explain, what this gives to Python and its users? Would we, eventually, get GIL-free VM, that is capable to consume all available power of multicore processors? Or, maybe, we'll get something special, I am unable to dream of? Or is it purely academic project to create Python VM in Python? No, PyPy is not a purely academic project. It has been a practically oriented project from the very start. For the regular user of Python, the main benefit of PyPy is execution speed. With the Just-in-time optimizer, we have examples of functions doing integer math that are 63 times faster than CPython. Further work will broaden the scope of the optimizations to other language constructs. Bulding the framework for the JIT took a long time. Now that it is in place, extending the range of optimizations is a much smaller task. For the Python language developers, PyPy offers an easy environment for testing new ideas. Changes and alternative solutions can more quickly be implemented in Python than in C. For people who want to broaden the scope of Python, there is the option of using some of the models that the PyPy project has implemented, or roll their own. Constraints satisfaction, lazy evaluation, cooperative massive multitasking, security oriented environments, transparent distributed computing and persistence are examples of things the project have implemented. There are some very neat APIs so that you can easily add your own behaviour to the Python interpreter. For those who want to do Computer Science, PyPy offers a wonderful research platform. For instance, you can experiment with alternate or even multiple implementations of builtin types. You can build frontends for your favourite programming language and backends for your favourite platform. Since Javascript is a backend that we already handle fairly well, you can write your FORTRAN parser and have a FORTRAN to JS compiler. The neat thing is that once you build the parser, you should automatically have your code working on all the different backends. (This is a goal and not quite reality at this point in time. Bugs and shortcomings will still be in the way.) There is a tradeoff in computer language design between execution speed flexibility and ease of use. I personally think that PyPy is about to break out of the tradeoff and bring it to another level. I know that I am biased, having been involved with the project from the start, but I think that it is the most interesting work in CS in many years. Jacob Hallén -- -- http://mail.python.org/mailman/listinfo/python-list
Re: PyPy 1.0: JIT compilers for free and more
Christian Tismer wrote: ... something special, I am unable to dream of? Or is it purely academic project to create Python VM in Python? It will eventually give you a GIL-free VM, and it already gives you a lot more than you have dreamt of. There is one feature missing that is probably hard to add. Handling the 'posters who are not willing to read before they post' syndrome. ... I come from an academic background (sadly not in computer science) so I understand the need to promote the abstract advances that are being made here. However, I am a bit confused about the applicability of pypy. All the talk of backends, annotation object spaces etc etc may make some sense to really interested parties, but I would like to know what I can do in real terms with the new interpreters? The demo javascript stuff seems a bit fake since others have already done some work in that direction already using ordinary python. RPython may be larger or smaller than those subsets, but that's not clear. I would be interested to know if it's possible to point a tool at an existing python-2.4 module and wrap it into a much faster extension that can be used by my CPython stuff. I know there is preliminary work in that direction. I am hugely encouraged by this C:\Python\devel\pypy-1.0.0\python24\python \python\lib\test\pystone.py Pystone(1.1) time for 5 passes = 1.49586 This machine benchmarks at 33425.6 pystones/second C:\Python\devel\pypy-1.0.0.\pypy-c.exe \python\lib\test\pystone.py Pystone(1.1) time for 5 passes = 2.16123e-005 This machine benchmarks at 2.3135e+009 pystones/second :) not -- Robin Becker -- http://mail.python.org/mailman/listinfo/python-list
Re: PyPy 1.0: JIT compilers for free and more
Robin Becker [EMAIL PROTECTED] wrote: I am hugely encouraged by this C:\Python\devel\pypy-1.0.0\python24\python \python\lib\test \pystone.py Pystone(1.1) time for 5 passes = 1.49586 This machine benchmarks at 33425.6 pystones/second C:\Python\devel\pypy-1.0.0.\pypy-c.exe \python\lib\test\pystone.py Pystone(1.1) time for 5 passes = 2.16123e-005 This machine benchmarks at 2.3135e+009 pystones/second :) not It looks like time.clock() is completely borked. C:\work\pypy-1.0.0\python25\python Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)] on win 32 Type help, copyright, credits or license for more information. from time import clock, sleep clock(); sleep(3); clock() 1.6203176660720846e-005 3.000427987355935 clock(); sleep(3); clock() 8.2580051375244619 11.258311321690327 clock(); sleep(3); clock() 16.283118664523005 19.283642753478446 ^Z C:\work\pypy-1.0.0pypy-c Python 2.4.1 (pypy 1.0.0 build 41438) on win32 Type help, copyright, credits or license for more information. from time import clock, sleep clock(); sleep(3); clock() 1.00337512 1.050338639583 clock(); sleep(3); clock() 1.103837992871 1.154196831568 Or perhaps it is simply telling you the sort of speeds it hopes to reach some day (I wouldn't say no to 9.16021e+009 pystones/second if that was what it actually did). -- http://mail.python.org/mailman/listinfo/python-list
Re: PyPy 1.0: JIT compilers for free and more
On 28 mar, 23:36, Jarek Zgoda [EMAIL PROTECTED] wrote: Carl Friedrich Bolz napisa³(a): Welcome to the PyPy 1.0 release - a milestone integrating the results of four years of research, engineering, management and sprinting efforts, concluding the 28 months phase of EU co-funding! So it took 4 yars of work and over 2 yaers of consumption of EU funds, yeah, great. Could anybody explain, what this gives to Python and its users? Would we, eventually, get GIL-free VM, that is capable to consume all available power of multicore processors? http://codespeak.net/pypy/dist/pypy/doc/stackless.html Or, maybe, we'll get something special, I am unable to dream of? Or is it purely academic project to create Python VM in Python? RTFM. -- http://mail.python.org/mailman/listinfo/python-list
Re: PyPy 1.0: JIT compilers for free and more
Duncan Booth wrote: Robin Becker [EMAIL PROTECTED] wrote: I am hugely encouraged by this C:\Python\devel\pypy-1.0.0\python24\python \python\lib\test \pystone.py Pystone(1.1) time for 5 passes = 1.49586 This machine benchmarks at 33425.6 pystones/second C:\Python\devel\pypy-1.0.0.\pypy-c.exe \python\lib\test\pystone.py Pystone(1.1) time for 5 passes = 2.16123e-005 This machine benchmarks at 2.3135e+009 pystones/second :) not It looks like time.clock() is completely borked. time.clock was indeed borked under windows (of by a factor of 1000). fixed in svn now. Cheers, Carl Friedrich -- http://mail.python.org/mailman/listinfo/python-list
Re: PyPy 1.0: JIT compilers for free and more
On Mar 28, 5:36 pm, Jarek Zgoda [EMAIL PROTECTED] wrote: Carl Friedrich Bolz napisa³(a): Welcome to the PyPy 1.0 release - a milestone integrating the results of four years of research, engineering, management and sprinting efforts, concluding the 28 months phase of EU co-funding! So it took 4 yars of work and over 2 yaers of consumption of EU funds, yeah, great. Could anybody explain, what this gives to Python and its users? Would we, eventually, get GIL-free VM, that is capable to consume all available power of multicore processors? That is the eventual hope, along with a faster Python interpreter (with a true functional JIT), and stackless features, and more easily developed syntax, and retargetable front- and back- ends (allowing, e.g. easier development of Python on the .NET CLI, JVM, etc, or other languages like Javascript, Lisp, or whatever on the PyPy VM), and fast extensions written in RPython instead of C, and many other features. -- http://mail.python.org/mailman/listinfo/python-list
Re: PyPy 1.0: JIT compilers for free and more
Carl Friedrich Bolz a écrit : == PyPy 1.0: JIT compilers for free and more == Welcome to the PyPy 1.0 release - a milestone integrating the results of four years of research, engineering, management and sprinting efforts You guys rock ! -- http://mail.python.org/mailman/listinfo/python-list
Re: PyPy 1.0: JIT compilers for free and more
Hi Luis! Luis M. González wrote: Carl Friedrich Bolz wrote: == PyPy 1.0: JIT compilers for free and more == [snip] Congratulations! Thanks :-) I just have a couple of questions: Attempting to execute pypy-c.exe (precompiled binary for Windows), raise an error message indicating that it cannot find gc_pypy.dll. Have you missed something or I'm doing something wrong? Brain error on our side: the gc_pypy.dll is the dll of the Boehm garbage collector, which you would need to compile yourself (which makes precompiled binaries a bit useless :-) ). We updated the zip file, would you mind checking whether it works better now? Regarding the jit integration, do you have any rough idea of when will it speed up arbitrary code, other than integer aritmethic? Relatively soon. The hard bits of that are done, the JIT compiler generator is in a pretty good shape, we need to apply it to more parts of the PyPy Python interpreter. There are some non-coding related reasons why we are not doing it _right_ now: we still need to write some EU-reports (about the JIT for example) then we will need a while to recover from the EU project. Cheers, Carl Friedrich Bolz -- http://mail.python.org/mailman/listinfo/python-list
Re: PyPy 1.0: JIT compilers for free and more
On Mar 27, 11:48 pm, Carl Friedrich Bolz [EMAIL PROTECTED] wrote: == PyPy 1.0: JIT compilers for free and more == Welcome to the PyPy 1.0 release - a milestone integrating the results of four years of research, engineering, management and sprinting efforts, concluding the 28 months phase of EU co-funding! Although still not mature enough for general use, PyPy 1.0 materializes for the first time the full extent of our original vision: - A flexible Python interpreter, written in RPython: - Mostly unaware of threading, memory and lower-level target platform aspects. - Showcasing advanced interpreter features and prototypes. - Passing core CPython regression tests, translatable to C, LLVM and .NET. - An advanced framework to translate such interpreters and programs: - That performs whole type-inference on RPython programs. - Can weave in threading, memory and target platform aspects. - Has low level (C, LLVM) and high level (CLI, Java, JavaScript) backends. - A **Just-In-Time Compiler generator** able to **automatically** enhance the low level versions of our Python interpreter, leading to run-time machine code that runs algorithmic examples at speeds typical of JITs! Previous releases, particularly the 0.99.0 release from February, already highlighted features of our Python implementation and the abilities of our translation approach but the **new JIT generator** clearly marks a major research result and gives weight to our vision that one can generate efficient interpreter implementations, starting from a description in a high level language. We have prepared several entry points to help you get started: * The main entry point for JIT documentation and status: http://codespeak.net/pypy/dist/pypy/doc/jit.html * The main documentation and getting-started PyPy entry point: http://codespeak.net/pypy/dist/pypy/doc/index.html * Our online play1 demos showcasing various Python interpreters, features (and a new way to program AJAX applications): http://play1.codespeak.net/ * Our detailed and in-depth Reports about various aspects of the project: http://codespeak.net/pypy/dist/pypy/doc/index-report.html In the next few months we are going to discuss the goals and form of the next stage of development - now more than ever depending on your feedback and contributions - and we hope you appreciate PyPy 1.0 as an interesting basis for greater things to come, as much as we do ourselves! have fun, the PyPy release team, Samuele Pedroni, Armin Rigo, Holger Krekel, Michael Hudson, Carl Friedrich Bolz, Antonio Cuni, Anders Chrigstroem, Guido Wesdorp Maciej Fijalkowski, Alexandre Fayolle and many others: http://codespeak.net/pypy/dist/pypy/doc/contributor.html What is PyPy? Technically, PyPy is both a Python interpreter implementation and an advanced compiler, or more precisely a framework for implementing dynamic languages and generating virtual machines for them. The framework allows for alternative frontends and for alternative backends, currently C, LLVM and .NET. For our main target C, we can can mix in different garbage collectors and threading models, including micro-threads aka Stackless. The inherent complexity that arises from this ambitious approach is mostly kept away from the Python interpreter implementation, our main frontend. PyPy is now also a Just-In-Time compiler generator. The translation framework contains the now-integrated JIT generation technology. This depends only on a few hints added to the interpreter source and should be able to cope with the changes to the interpreter and be generally applicable to other interpreters written using the framework. Socially, PyPy is a collaborative effort of many individuals working together in a distributed and sprint-driven way since 2003. PyPy would not have gotten as far as it has without the coding, feedback and general support from numerous people. Formally, many of the current developers were involved in executing an EU contract with the goal of exploring and researching new approaches to language and compiler development and software engineering. This contract's duration is about to end this month (March 2007) and we are working and preparing the according final review which is scheduled for May 2007. For the future, we are in the process of setting up structures to help maintain conceptual integrity of the project and to discuss and deal with funding opportunities related to further PyPy sprinting and developments. See here for results of the discussion so far: http://codespeak.net/pipermail/pypy-dev/2007q1/003577.html 1.0.0 Feature highlights == Here is a summary list of key features included in PyPy 1.0: - The
Re: PyPy 1.0: JIT compilers for free and more
On 28 Mar, 14:12, Kay Schluehr [EMAIL PROTECTED] wrote: A somewhat unrelated question. With Py3K Python gets optional type annotations. No, I believe the consensus is that Python 3000 gets optional annotations which aren't specifically for type information... nudge nudge, wink wink! That last part is important, because that's also how many other people have interpreted this particular feature. I seem to recall that some of the PyPy people weren't interested in static analysis [1], so I'm skeptical about whether type annotations are their kind of thing, either. That said, I imagine that the handling of metadata (including type details) is an element in the way RPython works, but I also imagine that things get more complicated as such metadata diverges from plain type information and into arbitrary observations about other aspects of program behaviour. Still, I'd like to see the PyPy people stick their necks out and answer your questions. ;-) Paul [1] http://codespeak.net/pipermail/pypy-dev/2007q1/003607.html -- http://mail.python.org/mailman/listinfo/python-list
Re: PyPy 1.0: JIT compilers for free and more
On 28 Mar, 14:12, Kay Schluehr [EMAIL PROTECTED] wrote: A somewhat unrelated question. With Py3K Python gets optional type annotations. No, I believe the consensus is that Python 3000 gets optional annotations which aren't specifically for type information... nudge nudge, wink wink! That last part is important, because that's also how many other people have interpreted this particular feature. I seem to recall that some of the PyPy people weren't interested in static analysis [1], so I'm skeptical about whether type annotations are their kind of thing, either. That said, I imagine that the handling of metadata (including type details) is an element in the way RPython works, but I also imagine that things get more complicated as such metadata diverges from plain type information and into arbitrary observations about other aspects of program behaviour. Still, I'd like to see the PyPy people stick their necks out and answer your questions. ;-) Paul [1] http://codespeak.net/pipermail/pypy-dev/2007q1/003607.html -- http://mail.python.org/mailman/listinfo/python-list
Re: PyPy 1.0: JIT compilers for free and more
Kay Schluehr [EMAIL PROTECTED] writes: A somewhat unrelated question. With Py3K Python gets optional type annotations. Are you already architecting an annotation handler that can process these annotations? This feature is somewhat competitive to all the complicated type inference and jitting you have been worked out so I don't know how it fits well into the current PyPy architecture? What's the nature of the conflict between annotations and inference? Haskell for example has both. -- http://mail.python.org/mailman/listinfo/python-list
Re: PyPy 1.0: JIT compilers for free and more
On 28.03.2007, at 10:38, Carl Friedrich Bolz wrote: Brain error on our side: the gc_pypy.dll is the dll of the Boehm garbage collector, which you would need to compile yourself (which makes precompiled binaries a bit useless :-) ). We updated the zip file, would you mind checking whether it works better now? Why can't we provide a pre-compiled binary? Is this a license issue? cheers - chris -- http://mail.python.org/mailman/listinfo/python-list
Re: PyPy 1.0: JIT compilers for free and more
On Mar 28, 2:54 pm, Paul Boddie [EMAIL PROTECTED] wrote: On 28 Mar, 14:12, Kay Schluehr [EMAIL PROTECTED] wrote: A somewhat unrelated question. With Py3K Python gets optional type annotations. No, I believe the consensus is that Python 3000 gets optional annotations which aren't specifically for type information... nudge nudge, wink wink! That last part is important, because that's also how many other people have interpreted this particular feature. You are right. The feature is more situated on a syntactical / interface level. The semantical level has to be specified by annotation handlers. Among them there will also be those that deal with type annotations. I have little doubts that one of them will make it into the stdlib sooner or later and might also influence compilation. I seem to recall that some of the PyPy people weren't interested in static analysis [1], so I'm skeptical about whether type annotations are their kind of thing, either. That's why I asked the question with regard to Py3K style annotations and not static analysis in Py2X which I consider intractible. The JIT measures types at runtime and optimizes code using this information but doing things in this style global optimizations are hardly tractible as well. My main interest is in an intermediate level, given type measurements as a byproduct of UTs and code coverage. This methodology is process-oriented and non interferent. That said, I imagine that the handling of metadata (including type details) is an element in the way RPython works, but I also imagine that things get more complicated as such metadata diverges from plain type information and into arbitrary observations about other aspects of program behaviour. RPython is heuristically defined as a subset of Python static enough to be translatable to C. So it is actually static analysis that is done here, not on a local scale but on a simpler sublanguage. It is not clear to me whether for a sufficiently annotated Py3K program the inequation RPython != Python still holds? Still, I'd like to see the PyPy people stick their necks out and answer your questions. ;-) Paul [1]http://codespeak.net/pipermail/pypy-dev/2007q1/003607.html -- http://mail.python.org/mailman/listinfo/python-list
Re: PyPy 1.0: JIT compilers for free and more
Hi Christian! Christian Tismer wrote: On 28.03.2007, at 10:38, Carl Friedrich Bolz wrote: Brain error on our side: the gc_pypy.dll is the dll of the Boehm garbage collector, which you would need to compile yourself (which makes precompiled binaries a bit useless :-) ). We updated the zip file, would you mind checking whether it works better now? Why can't we provide a pre-compiled binary? Is this a license issue? We can. That's exactly what we did. Cheers, Carl Friedrich Bolz -- http://mail.python.org/mailman/listinfo/python-list
Re: PyPy 1.0: JIT compilers for free and more
On Mar 28, 4:34 pm, Paul Rubin http://[EMAIL PROTECTED] wrote: Kay Schluehr [EMAIL PROTECTED] writes: A somewhat unrelated question. With Py3K Python gets optional type annotations. Are you already architecting an annotation handler that can process these annotations? This feature is somewhat competitive to all the complicated type inference and jitting you have been worked out so I don't know how it fits well into the current PyPy architecture? What's the nature of the conflict between annotations and inference? Haskell for example has both. There is no conflict in Haskell because Haskell has a type system by default. But Python has none and so one had to define RPython, a subset of Python *viewed* as static enough to be translatable to C ( RPython is of course as polymorphic as any other Python code and can be fed with arbitrary types. Only the context makes it special ). RPython is an approximation to Python on which static type inference is feasible. RPython is yet defined heuristically and this heuristics might be changed when RPython can be defined by the power of a type system defined *formally*. So it might be just a matter of reason to define a typesystem for interpreter/jit level Python code. -- http://mail.python.org/mailman/listinfo/python-list
Re: PyPy 1.0: JIT compilers for free and more
Kay Schluehr: RPython is heuristically defined as a subset of Python static enough to be translatable to C. So it is actually static analysis that is done here, not on a local scale but on a simpler sublanguage. It is not clear to me whether for a sufficiently annotated Py3K program the inequation RPython != Python still holds? You may also take a very good look at ShedSkin, it's already able to compile a decent part of Python, and that part is slowly growing still. I'd like to see a comparison of SSPython and RPython (width-wise and running speed wise too) :-) Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: PyPy 1.0: JIT compilers for free and more
Hi! Suppose I have a py-written module. Is it possible somehow run PyPy on the whole module? I didn't find it in documentation. And if yes (or if just run in every module func) what will be after computer restart? Should I restart PyPy on the module once again? And are there any chances/intends for PyPy to be included into Python core? Thx, D. On Mar 28, 12:48 am, Carl Friedrich Bolz [EMAIL PROTECTED] wrote: ==PyPy1.0: JIT compilers for free and more == Welcome to thePyPy1.0 release - a milestone integrating the results of four years of research, engineering, management and sprinting efforts, concluding the 28 months phase of EU co-funding! Although still not mature enough for general use,PyPy1.0 materializes for the first time the full extent of our original vision: - A flexible Python interpreter, written in RPython: - Mostly unaware of threading, memory and lower-level target platform aspects. - Showcasing advanced interpreter features and prototypes. - Passing core CPython regression tests, translatable to C, LLVM and .NET. - An advanced framework to translate such interpreters and programs: - That performs whole type-inference on RPython programs. - Can weave in threading, memory and target platform aspects. - Has low level (C, LLVM) and high level (CLI, Java, JavaScript) backends. - A **Just-In-Time Compiler generator** able to **automatically** enhance the low level versions of our Python interpreter, leading to run-time machine code that runs algorithmic examples at speeds typical of JITs! Previous releases, particularly the 0.99.0 release from February, already highlighted features of our Python implementation and the abilities of our translation approach but the **new JIT generator** clearly marks a major research result and gives weight to our vision that one can generate efficient interpreter implementations, starting from a description in a high level language. We have prepared several entry points to help you get started: * The main entry point for JIT documentation and status: http://codespeak.net/pypy/dist/pypy/doc/jit.html * The main documentation and getting-startedPyPyentry point: http://codespeak.net/pypy/dist/pypy/doc/index.html * Our online play1 demos showcasing various Python interpreters, features (and a new way to program AJAX applications): http://play1.codespeak.net/ * Our detailed and in-depth Reports about various aspects of the project: http://codespeak.net/pypy/dist/pypy/doc/index-report.html In the next few months we are going to discuss the goals and form of the next stage of development - now more than ever depending on your feedback and contributions - and we hope you appreciatePyPy1.0 as an interesting basis for greater things to come, as much as we do ourselves! have fun, thePyPyrelease team, Samuele Pedroni, Armin Rigo, Holger Krekel, Michael Hudson, Carl Friedrich Bolz, Antonio Cuni, Anders Chrigstroem, Guido Wesdorp Maciej Fijalkowski, Alexandre Fayolle and many others: http://codespeak.net/pypy/dist/pypy/doc/contributor.html What isPyPy? Technically,PyPyis both a Python interpreter implementation and an advanced compiler, or more precisely a framework for implementing dynamic languages and generating virtual machines for them. The framework allows for alternative frontends and for alternative backends, currently C, LLVM and .NET. For our main target C, we can can mix in different garbage collectors and threading models, including micro-threads aka Stackless. The inherent complexity that arises from this ambitious approach is mostly kept away from the Python interpreter implementation, our main frontend. PyPyis now also a Just-In-Time compiler generator. The translation framework contains the now-integrated JIT generation technology. This depends only on a few hints added to the interpreter source and should be able to cope with the changes to the interpreter and be generally applicable to other interpreters written using the framework. Socially,PyPyis a collaborative effort of many individuals working together in a distributed and sprint-driven way since 2003. PyPywould not have gotten as far as it has without the coding, feedback and general support from numerous people. Formally, many of the current developers were involved in executing an EU contract with the goal of exploring and researching new approaches to language and compiler development and software engineering. This contract's duration is about to end this month (March 2007) and we are working and preparing the according final review which is scheduled for May 2007. For the future, we are in the process of setting up structures to help maintain conceptual integrity of the project and to discuss
Re: PyPy 1.0: JIT compilers for free and more
Hi! dmitrey wrote: Hi! Suppose I have a py-written module. Is it possible somehow run PyPy on the whole module? I didn't find it in documentation. And if yes (or if just run in every module func) what will be after computer restart? Should I restart PyPy on the module once again? And are there any chances/intends for PyPy to be included into Python core? PyPy contains a full Python interpreter (which can include a JIT compiler) and thus replaces the Python core. PyPy can never really be integrated into CPython. Cheers, Carl Friedrich Bolz -- http://mail.python.org/mailman/listinfo/python-list
Re: PyPy 1.0: JIT compilers for free and more
Carl Friedrich Bolz napisał(a): Welcome to the PyPy 1.0 release - a milestone integrating the results of four years of research, engineering, management and sprinting efforts, concluding the 28 months phase of EU co-funding! So it took 4 yars of work and over 2 yaers of consumption of EU funds, yeah, great. Could anybody explain, what this gives to Python and its users? Would we, eventually, get GIL-free VM, that is capable to consume all available power of multicore processors? Or, maybe, we'll get something special, I am unable to dream of? Or is it purely academic project to create Python VM in Python? -- Jarek Zgoda http://jpa.berlios.de/ -- http://mail.python.org/mailman/listinfo/python-list
Re: PyPy 1.0: JIT compilers for free and more
Kay Schluehr wrote: Nice to read that things are going on. I've still a PyPy 0.7 version on my notebook. I guess I will upgrade :) A somewhat unrelated question. With Py3K Python gets optional type annotations. Are you already architecting an annotation handler that can process these annotations? This feature is somewhat competitive to all the complicated type inference and jitting you have been worked out so I don't know how it fits well into the current PyPy architecture? I don't see at all why type annotations are competitive to the type inference that PyPy's translation toolchain is doing. The type inference of PyPy is an artefact of the way we are implementing our interpreter (namely in RPython). You also wouldn't say that the static typing of C is competitive to type annotations because CPython is written in C, right? The JIT (which is completely independent from our type inference engine) will hopefully deal well with the eventual addition of type annotations (it's not clear to me how soon we will start supporting Py3k features, we are not even fully supporting Python 2.5 yet). Since the JIT is automatically generated out of the Python interpreter it should deal with any sort of language changes rather well. Whether that is true in practice remains to be seen, but this is one of the reason why PyPy was started in the first place. Also, I fail to see how type annotations can have a huge speed-advantage versus what our JIT and Psyco are doing. Imagine you have a function that is nicely algorithmic, takes only integers as arguments and only does some calculations with them. If you have a Psyco-like scheme, the JIT will note that you are using it mostly with ints, generate relatively efficient assembly for the whole function (at least after it has been called several times). If you call that functions, the arguments are checked whether they are integers and then the fast assembly code is used without any further type checks. How can you improve on this with type annotations? If the type annotations say that your arguments have to be ints, you _still_ have to check whether this is true. So it is not faster than what Psyco is doing and has the disadvantage that it only works with ints -- good bye, duck typing. Psyco on the other hand will be perfectly happy with you using a custom class that looks mostly like an int and generate assembly optimized for this situation. Cheers, Carl Friedrich Bolz -- http://mail.python.org/mailman/listinfo/python-list
Re: PyPy 1.0: JIT compilers for free and more
On 28.03.2007, at 23:36, Jarek Zgoda wrote: Carl Friedrich Bolz napisał(a): Welcome to the PyPy 1.0 release - a milestone integrating the results of four years of research, engineering, management and sprinting efforts, concluding the 28 months phase of EU co-funding! So it took 4 yars of work and over 2 yaers of consumption of EU funds, yeah, great. Could anybody explain, what this gives to Python and its users? Would we, eventually, get GIL-free VM, that is capable to consume all available power of multicore processors? Or, maybe, we'll get something special, I am unable to dream of? Or is it purely academic project to create Python VM in Python? It will eventually give you a GIL-free VM, and it already gives you a lot more than you have dreamt of. There is one feature missing that is probably hard to add. Handling the 'posters who are not willing to read before they post' syndrome. cheers - chris -- http://mail.python.org/mailman/listinfo/python-list
Re: PyPy 1.0: JIT compilers for free and more
Carl Welcome to the PyPy 1.0 release... ... Carl - A **Just-In-Time Compiler generator** able to **automatically** Carl enhance the low level versions of our Python interpreter, Carl leading to run-time machine code that runs algorithmic examples Carl at speeds typical of JITs! Woo hoo! Congratulations to all the PyPynistas! Skip -- http://mail.python.org/mailman/listinfo/python-list
Re: PyPy 1.0: JIT compilers for free and more
Carl Friedrich Bolz wrote: == PyPy 1.0: JIT compilers for free and more == Welcome to the PyPy 1.0 release - a milestone integrating the results of four years of research, engineering, management and sprinting efforts, concluding the 28 months phase of EU co-funding! Although still not mature enough for general use, PyPy 1.0 materializes for the first time the full extent of our original vision: - A flexible Python interpreter, written in RPython: - Mostly unaware of threading, memory and lower-level target platform aspects. - Showcasing advanced interpreter features and prototypes. - Passing core CPython regression tests, translatable to C, LLVM and .NET. - An advanced framework to translate such interpreters and programs: - That performs whole type-inference on RPython programs. - Can weave in threading, memory and target platform aspects. - Has low level (C, LLVM) and high level (CLI, Java, JavaScript) backends. - A **Just-In-Time Compiler generator** able to **automatically** enhance the low level versions of our Python interpreter, leading to run-time machine code that runs algorithmic examples at speeds typical of JITs! Previous releases, particularly the 0.99.0 release from February, already highlighted features of our Python implementation and the abilities of our translation approach but the **new JIT generator** clearly marks a major research result and gives weight to our vision that one can generate efficient interpreter implementations, starting from a description in a high level language. We have prepared several entry points to help you get started: * The main entry point for JIT documentation and status: http://codespeak.net/pypy/dist/pypy/doc/jit.html * The main documentation and getting-started PyPy entry point: http://codespeak.net/pypy/dist/pypy/doc/index.html * Our online play1 demos showcasing various Python interpreters, features (and a new way to program AJAX applications): http://play1.codespeak.net/ * Our detailed and in-depth Reports about various aspects of the project: http://codespeak.net/pypy/dist/pypy/doc/index-report.html In the next few months we are going to discuss the goals and form of the next stage of development - now more than ever depending on your feedback and contributions - and we hope you appreciate PyPy 1.0 as an interesting basis for greater things to come, as much as we do ourselves! have fun, the PyPy release team, Samuele Pedroni, Armin Rigo, Holger Krekel, Michael Hudson, Carl Friedrich Bolz, Antonio Cuni, Anders Chrigstroem, Guido Wesdorp Maciej Fijalkowski, Alexandre Fayolle and many others: http://codespeak.net/pypy/dist/pypy/doc/contributor.html What is PyPy? Technically, PyPy is both a Python interpreter implementation and an advanced compiler, or more precisely a framework for implementing dynamic languages and generating virtual machines for them. The framework allows for alternative frontends and for alternative backends, currently C, LLVM and .NET. For our main target C, we can can mix in different garbage collectors and threading models, including micro-threads aka Stackless. The inherent complexity that arises from this ambitious approach is mostly kept away from the Python interpreter implementation, our main frontend. PyPy is now also a Just-In-Time compiler generator. The translation framework contains the now-integrated JIT generation technology. This depends only on a few hints added to the interpreter source and should be able to cope with the changes to the interpreter and be generally applicable to other interpreters written using the framework. Socially, PyPy is a collaborative effort of many individuals working together in a distributed and sprint-driven way since 2003. PyPy would not have gotten as far as it has without the coding, feedback and general support from numerous people. Formally, many of the current developers were involved in executing an EU contract with the goal of exploring and researching new approaches to language and compiler development and software engineering. This contract's duration is about to end this month (March 2007) and we are working and preparing the according final review which is scheduled for May 2007. For the future, we are in the process of setting up structures to help maintain conceptual integrity of the project and to discuss and deal with funding opportunities related to further PyPy sprinting and developments. See here for results of the discussion so far: http://codespeak.net/pipermail/pypy-dev/2007q1/003577.html 1.0.0 Feature highlights == Here is a summary list of key features included in PyPy 1.0: - The Just-In-Time compiler generator,
Re: PyPy 1.0: JIT compilers for free and more
Carl Friedrich Bolz [EMAIL PROTECTED] writes: == PyPy 1.0: JIT compilers for free and more == Welcome to the PyPy 1.0 release - a milestone integrating the results of four years of research, engineering, management and sprinting efforts, concluding the 28 months phase of EU co-funding! Fabulous! Congratulations to all who contributed to getting PyPy to this significant milestone. Here's to all the cool new things people will be doing with PyPy! -- \ If nothing changes, everything will remain the same. -- | `\ Barne's Law | _o__) | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
