Re: [pypy-dev] Does PyPy supports Java ME?
On Thu, Oct 08, 2009 at 15:30 +0200, Jan Wedel wrote: > > Do you have a pointer to documentation or a text that says so? > > Yes : > > http://codespeak.net/pypy/extradoc/eu-report/D11.1_PyPy_for_Embedded_Devices-2007-03-26.pdf > > > It should be possible with future versions of PyPy to use the > framework in other ways in the embedded context > > for instance by shipping a restricted version of the interpreter, > dedicated to the interpretation of a given > > set of programs. > > I think I read something more precise but I couldn't find it anymore. That use case study mainly worked with using RPython and says that its efficiency is not far from using C. I guess this could be extended to also apply for Java-ME so that an RPython program would run as fast as native Java. A restricted version of the existing Python interpreter is quite a lot of work and it's unclear how successful it could be. Python might be an easy-to-read and to-learn language but it's internals are quite complex, unfortunately. best, holger > -Ursprüngliche Nachricht- > Von: holger krekel [mailto:hol...@merlinux.eu] > Gesendet: Donnerstag, 8. Oktober 2009 15:17 > An: Jan Wedel > Cc: pypy-dev@codespeak.net > Betreff: Re: [pypy-dev] Does PyPy supports Java ME? > > Hi Jan, > > On Tue, Oct 06, 2009 at 15:15 +0200, Jan Wedel wrote: > > I was currently working on a port of PyMite (8-Bit MCU port of > CPython) > > to Java ME. I already got the interpreter working so that basic Python > > > scripts could run as code images on a mobile java IMP device. Actually > > > we need Python interpreters on different embedded platforms supporting > > > either C or Java ME (IMP). I discovered a lot of problems including > the > > missing class implementation of PyMite. > > > > Then, by recommendation, I found PyPy. It looks as if it is exactly > what > > I was looking for and I read that it is specifically designed to > support > > embedded platforms. > > Do you have a pointer to documentation or a text that says so? I > wouldn't > see PyPy this way. However, we do have a pypy-c version that works on > Maemo and is more efficient for larger amounts of user objects. We also > have ideas how to reduce RSS size after interpreter startup, > mostly tied to pypy-c so far, though. > > In principle it's also possible to translate RPython to C or JVM > and avoid interpretation alltogether. RPython is >50 times faster > than CPython but less convenient to work with than regular Python > so only makes sense to consider for special purposes. And is > probably best learned in collaboration with developers who know > how to work with it and are willing or paid to help. > > cheers, > holger > > -- > Metaprogramming, Python, Testing: http://tetamap.wordpress.com > Python, PyPy, pytest contracting: http://merlinux.eu > > > ___ > pypy-dev@codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > -- Metaprogramming, Python, Testing: http://tetamap.wordpress.com Python, PyPy, pytest contracting: http://merlinux.eu ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Does PyPy supports Java ME?
> Do you have a pointer to documentation or a text that says so? Yes : http://codespeak.net/pypy/extradoc/eu-report/D11.1_PyPy_for_Embedded_Devices-2007-03-26.pdf > It should be possible with future versions of PyPy to use the framework in other ways in the embedded context > for instance by shipping a restricted version of the interpreter, dedicated to the interpretation of a given > set of programs. I think I read something more precise but I couldn't find it anymore. -Ursprüngliche Nachricht- Von: holger krekel [mailto:hol...@merlinux.eu] Gesendet: Donnerstag, 8. Oktober 2009 15:17 An: Jan Wedel Cc: pypy-dev@codespeak.net Betreff: Re: [pypy-dev] Does PyPy supports Java ME? Hi Jan, On Tue, Oct 06, 2009 at 15:15 +0200, Jan Wedel wrote: > I was currently working on a port of PyMite (8-Bit MCU port of CPython) > to Java ME. I already got the interpreter working so that basic Python > scripts could run as code images on a mobile java IMP device. Actually > we need Python interpreters on different embedded platforms supporting > either C or Java ME (IMP). I discovered a lot of problems including the > missing class implementation of PyMite. > > Then, by recommendation, I found PyPy. It looks as if it is exactly what > I was looking for and I read that it is specifically designed to support > embedded platforms. Do you have a pointer to documentation or a text that says so? I wouldn't see PyPy this way. However, we do have a pypy-c version that works on Maemo and is more efficient for larger amounts of user objects. We also have ideas how to reduce RSS size after interpreter startup, mostly tied to pypy-c so far, though. In principle it's also possible to translate RPython to C or JVM and avoid interpretation alltogether. RPython is >50 times faster than CPython but less convenient to work with than regular Python so only makes sense to consider for special purposes. And is probably best learned in collaboration with developers who know how to work with it and are willing or paid to help. cheers, holger -- Metaprogramming, Python, Testing: http://tetamap.wordpress.com Python, PyPy, pytest contracting: http://merlinux.eu ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Does PyPy supports Java ME?
Hi Jan, On Tue, Oct 06, 2009 at 15:15 +0200, Jan Wedel wrote: > I was currently working on a port of PyMite (8-Bit MCU port of CPython) > to Java ME. I already got the interpreter working so that basic Python > scripts could run as code images on a mobile java IMP device. Actually > we need Python interpreters on different embedded platforms supporting > either C or Java ME (IMP). I discovered a lot of problems including the > missing class implementation of PyMite. > > Then, by recommendation, I found PyPy. It looks as if it is exactly what > I was looking for and I read that it is specifically designed to support > embedded platforms. Do you have a pointer to documentation or a text that says so? I wouldn't see PyPy this way. However, we do have a pypy-c version that works on Maemo and is more efficient for larger amounts of user objects. We also have ideas how to reduce RSS size after interpreter startup, mostly tied to pypy-c so far, though. In principle it's also possible to translate RPython to C or JVM and avoid interpretation alltogether. RPython is >50 times faster than CPython but less convenient to work with than regular Python so only makes sense to consider for special purposes. And is probably best learned in collaboration with developers who know how to work with it and are willing or paid to help. cheers, holger -- Metaprogramming, Python, Testing: http://tetamap.wordpress.com Python, PyPy, pytest contracting: http://merlinux.eu ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Does PyPy supports Java ME?
> so far pypy-jvm doesn't use any of those, so it should not be a problem. > However, in the future reflection will be surely needed to access java > libraries (but you don't seem to be interested in it) and class loading for > the jit. Hmm, I'm not sure if one the goals for the future is still to use PyPy for embedded target (as it was stated in one of these EU papers). But if the answer is yes, you should really check embedded system constraits. I think the idea of PyPy is really good because you can develop platform independend application in Python and moreover auto-generate interpreters for different software platforms while keeping the interpreter code "synchronized" for all platforms. This is a really good approach but you should consider the limitations of embedded devices. > I don't know for sure how to measure memory usage of java programs, but if I > just launch pypy-jvm and measure the amount of memory used on linux, I get > something like ~400 MB: this also includes the overhead of the jvm itself, but > it's still a lot. I think for my task, PyPy is still too heavyweight although 400MB includes paged memory and all loaded Java libraries so its REALLY not possible to find out the real memory usage of a Java application (I tried this on Linux before). So, I guess I will stick to the Java port of PyMite/Python-on-a-chip. But thank you for your help and good luck! Jan -Ursprüngliche Nachricht- Von: Antonio Cuni [mailto:anto.c...@gmail.com] Gesendet: Donnerstag, 8. Oktober 2009 14:37 An: Jan Wedel; PyPy Dev Betreff: Re: AW: Re: [pypy-dev] Does PyPy supports Java ME? Jan Wedel wrote: > Hi Anto! > > Thanks for your answer. Hi Jan, (I'm cc-ing pypy-dev again) > Although it's now made by Sun, it shows the key points: > - No class loading > - No JNI > - No reflection > > Which means you cannot load dynamic code during run-time. On top, there > are a lot of libraries missing (XML parsing, Regex, logging...) so far pypy-jvm doesn't use any of those, so it should not be a problem. However, in the future reflection will be surely needed to access java libraries (but you don't seem to be interested in it) and class loading for the jit. > THAT might be a problem since we are working on memory constrained > systems with a maximum of a few MB of flash and approx. 200kB RAM. Are > there any estimations about the RAM usage of PyPy using classes and > other memory-consuming features? I think that 200kB are definitely not enough for pypy-jvm. I don't know for sure how to measure memory usage of java programs, but if I just launch pypy-jvm and measure the amount of memory used on linux, I get something like ~400 MB: this also includes the overhead of the jvm itself, but it's still a lot. > And is this 8-10MB including only the core interpreter? Or are there any > ways to strip it down by excluding unnecessary libs? it includes the core interpreter and the builtin modules; some of them can be stripped down, but I don't know how much you can save. > I didn't make myself clear. I meant python libs/modules. E.g. sys, > socket, sre etc... Because if these modules are supported I won't need a > lot of Java Libraries. As I explained, I already ported PyMite to Java > and I discovered that numerous python modules have a C counterpart such > as _socket, _sre etc. So If you want to run these libs, you need to > translate these C files. ah, now I understand. This is the complete list of builtin modules that are enabled for pypy-jvm: [translation] [usemodules] [translation] __pypy__ = True [translation] _ast = True [translation] _codecs = True [translation] _pickle_support = True [translation] _random = True [translation] _sre = True [translation] _testing = True [translation] _weakref = True [translation] cStringIO = True [translation] errno = True [translation] gc = True [translation] itertools = True [translation] marshal = True [translation] math = True [translation] md5 = True [translation] operator = True [translation] parser = True [translation] posix = True [translation] rbench = True [translation] sha = True [translation] symbol = True [translation] time = True [translation] token = True As you can see, _socket is missing. It's surely possible to write a version of it that works for the jvm, but as usual it takes some time and nobody cared enough to implement it. ciao, Anto ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Does PyPy supports Java ME?
Jan Wedel wrote: > Hi Anto! > > Thanks for your answer. Hi Jan, (I'm cc-ing pypy-dev again) > Although it's now made by Sun, it shows the key points: > - No class loading > - No JNI > - No reflection > > Which means you cannot load dynamic code during run-time. On top, there > are a lot of libraries missing (XML parsing, Regex, logging...) so far pypy-jvm doesn't use any of those, so it should not be a problem. However, in the future reflection will be surely needed to access java libraries (but you don't seem to be interested in it) and class loading for the jit. > THAT might be a problem since we are working on memory constrained > systems with a maximum of a few MB of flash and approx. 200kB RAM. Are > there any estimations about the RAM usage of PyPy using classes and > other memory-consuming features? I think that 200kB are definitely not enough for pypy-jvm. I don't know for sure how to measure memory usage of java programs, but if I just launch pypy-jvm and measure the amount of memory used on linux, I get something like ~400 MB: this also includes the overhead of the jvm itself, but it's still a lot. > And is this 8-10MB including only the core interpreter? Or are there any > ways to strip it down by excluding unnecessary libs? it includes the core interpreter and the builtin modules; some of them can be stripped down, but I don't know how much you can save. > I didn't make myself clear. I meant python libs/modules. E.g. sys, > socket, sre etc... Because if these modules are supported I won't need a > lot of Java Libraries. As I explained, I already ported PyMite to Java > and I discovered that numerous python modules have a C counterpart such > as _socket, _sre etc. So If you want to run these libs, you need to > translate these C files. ah, now I understand. This is the complete list of builtin modules that are enabled for pypy-jvm: [translation] [usemodules] [translation] __pypy__ = True [translation] _ast = True [translation] _codecs = True [translation] _pickle_support = True [translation] _random = True [translation] _sre = True [translation] _testing = True [translation] _weakref = True [translation] cStringIO = True [translation] errno = True [translation] gc = True [translation] itertools = True [translation] marshal = True [translation] math = True [translation] md5 = True [translation] operator = True [translation] parser = True [translation] posix = True [translation] rbench = True [translation] sha = True [translation] symbol = True [translation] time = True [translation] token = True As you can see, _socket is missing. It's surely possible to write a version of it that works for the jvm, but as usual it takes some time and nobody cared enough to implement it. ciao, Anto ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Does PyPy supports Java ME?
Hi Jan, Jan Wedel wrote: > 1.) Does PyPy generate both C and Java Bytecode for the Interpreter out > of the box or with low effort? yes > 2.) Is the java code compatible with Java ME or is it possible to modify > the PyPy-code/Interpreter-code so that Java ME is supported? I don't know what are the requirements/restrictions for Java ME. If the restrictions are only about the library, it's possible that it works out of the box as pypy-jvm doesn't use many external libraries. However, note that you might have memory problems because the binary it's kind of big (the JAR file is ~8-10 MB, IIRC). > 3.) What standard libraries are supported? I coulnd't find a list. > Especially the communication (such as socket) functionality is > important. unfortunately, you cannot access any external java library from pypy-jvm so far. I agree that it makes it a bit pointless, but basically nobody ever cared enough/had time to add support for it. I plan to do it at some point, but surely not before I finish my PhD (which is supposed to finish at the end of next april). > 4.) Do you think PyPy is stable enough for commercial use? The MIT > license allows commercial use as far as I read. The Python interpreter in PyPy is quite stable and compliant with CPython. However, the resulting pypy-jvm has never been extensively tested as far as I know, so it's likely to have undiscovered bugs (but usually they are easy to fix). So, to sum up: I'm sorry but pypy-jvm is not usable for production right now; it could be made useful with relatively low effort if someone invested time and/or money on it, but I don't think it's going to happen very soon. ciao, Anto ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
[pypy-dev] Does PyPy supports Java ME?
Hi, I was currently working on a port of PyMite (8-Bit MCU port of CPython) to Java ME. I already got the interpreter working so that basic Python scripts could run as code images on a mobile java IMP device. Actually we need Python interpreters on different embedded platforms supporting either C or Java ME (IMP). I discovered a lot of problems including the missing class implementation of PyMite. Then, by recommendation, I found PyPy. It looks as if it is exactly what I was looking for and I read that it is specifically designed to support embedded platforms. But I want to clarify a few things: 1.) Does PyPy generate both C and Java Bytecode for the Interpreter out of the box or with low effort? 2.) Is the java code compatible with Java ME or is it possible to modify the PyPy-code/Interpreter-code so that Java ME is supported? 3.) What standard libraries are supported? I coulnd't find a list. Especially the communication (such as socket) functionality is important. 4.) Do you think PyPy is stable enough for commercial use? The MIT license allows commercial use as far as I read. Thank you very much! Regards, Jan ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
