Re: [pypy-dev] testing floating point
On Thu, Feb 3, 2011 at 8:01 AM, Maciej Fijalkowski fij...@gmail.com wrote: On Thu, Feb 3, 2011 at 1:43 AM, Fredrik Johansson fredrik.johans...@gmail.com wrote: On Thu, Feb 3, 2011 at 12:32 AM, Alex Gaynor alex.gay...@gmail.com wrote: If you have any benchmarks where we are slower that don't involve longs that'd be great, since for float operations we really should be able to beat up on CPython. A simple example: mpmath.fp.fsum(mpmath.fp.sqrt(k) for k in xrange(100)) A brief follow up. * PyPy trunk is faster (by quite a bit). * I noticed that you happily use mixture of old and new style classes. As of now this is a really bad case for PyPy. Example: [isinstance(i, type) for i in mpmath.fp.__class__.__mro__] [True, True, True, True, False, False, True, True, False, True, True, True, True, True, True] when I replace it with newstyle classes it runs much faster Interesting. The mixture of old and new style classes is a mistake, of course. I'm going to add a test to make sure this doesn't happen. Thanks for pointing it out. In fact this speeds up another benchmark I did -- [fp.lambertw(k) for k in xrange(5)]-- by 10x, which is quite a ridiculous ratio! Other things that speed up both CPython and PyPy: * Put this things into function instead of at global scope Do you mean in the benchmark or did you have some other code you saw in mind? * Use list comprehension instead of generator expression. I hope PyPy can do more in the future to speed up generator expressions. Fredrik ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] testing floating point
On Thu, Feb 3, 2011 at 11:56 AM, Fredrik Johansson fredrik.johans...@gmail.com wrote: On Thu, Feb 3, 2011 at 8:01 AM, Maciej Fijalkowski fij...@gmail.com wrote: On Thu, Feb 3, 2011 at 1:43 AM, Fredrik Johansson fredrik.johans...@gmail.com wrote: On Thu, Feb 3, 2011 at 12:32 AM, Alex Gaynor alex.gay...@gmail.com wrote: If you have any benchmarks where we are slower that don't involve longs that'd be great, since for float operations we really should be able to beat up on CPython. A simple example: mpmath.fp.fsum(mpmath.fp.sqrt(k) for k in xrange(100)) A brief follow up. * PyPy trunk is faster (by quite a bit). * I noticed that you happily use mixture of old and new style classes. As of now this is a really bad case for PyPy. Example: [isinstance(i, type) for i in mpmath.fp.__class__.__mro__] [True, True, True, True, False, False, True, True, False, True, True, True, True, True, True] when I replace it with newstyle classes it runs much faster Interesting. The mixture of old and new style classes is a mistake, of course. I'm going to add a test to make sure this doesn't happen. Thanks for pointing it out. In fact this speeds up another benchmark I did -- [fp.lambertw(k) for k in xrange(5)]-- by 10x, which is quite a ridiculous ratio! Mixture of old and new style classes is not only preventing us from doing optimizations but also hits a bad case of tradeoffs. However, we decided we don't care that much. You should use new style classes anyway :) Other things that speed up both CPython and PyPy: * Put this things into function instead of at global scope Do you mean in the benchmark or did you have some other code you saw in mind? The benchmark. * Use list comprehension instead of generator expression. I hope PyPy can do more in the future to speed up generator expressions. It doesn't speed up things by much. Yeah, I can imagine we can improve on this, but it's also a bit hard. Fredrik ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] testing floating point
On Thu, Feb 3, 2011 at 11:14 AM, Maciej Fijalkowski fij...@gmail.com wrote: Mixture of old and new style classes is not only preventing us from doing optimizations but also hits a bad case of tradeoffs. However, we decided we don't care that much. You should use new style classes anyway :) Yes, this is perfectly reasonable. You should make PyPy print warnings when it encounters mixed-type classes :) Fredrik ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] testing floating point
On Thu, Feb 3, 2011 at 12:23 PM, Fredrik Johansson fredrik.johans...@gmail.com wrote: On Thu, Feb 3, 2011 at 11:14 AM, Maciej Fijalkowski fij...@gmail.com wrote: Mixture of old and new style classes is not only preventing us from doing optimizations but also hits a bad case of tradeoffs. However, we decided we don't care that much. You should use new style classes anyway :) Yes, this is perfectly reasonable. You should make PyPy print warnings when it encounters mixed-type classes :) Fredrik That's not a bad idea actually. Maybe with something like some -X option? ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] testing floating point
2011/2/3 Maciej Fijalkowski fij...@gmail.com: You should make PyPy print warnings when it encounters mixed-type classes :) That's not a bad idea actually. Maybe with something like some -X option? If we use the warnings module to emit JitWarnings, the option could be -Wd::JitWarning -- Amaury Forgeot d'Arc ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] testing floating point
On Thu, Feb 3, 2011 at 2:10 PM, Amaury Forgeot d'Arc amaur...@gmail.com wrote: 2011/2/3 Maciej Fijalkowski fij...@gmail.com: You should make PyPy print warnings when it encounters mixed-type classes :) That's not a bad idea actually. Maybe with something like some -X option? If we use the warnings module to emit JitWarnings, the option could be -Wd::JitWarning I'm not sure about warnings module. How about --jit warnings=1 ? That would fit with other jit options. I was more thinking about this being optional -- Amaury Forgeot d'Arc ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] testing floating point
On 03/02/11 13:13, Maciej Fijalkowski wrote: I'm not sure about warnings module. How about --jit warnings=1 ? That would fit with other jit options. not really. The other jit options really belongs to the JIT engine, while this one is dependent on the Python interpreter ciao, Anto ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] testing floating point
On Thu, Feb 3, 2011 at 2:21 PM, Antonio Cuni anto.c...@gmail.com wrote: On 03/02/11 13:13, Maciej Fijalkowski wrote: I'm not sure about warnings module. How about --jit warnings=1 ? That would fit with other jit options. not really. The other jit options really belongs to the JIT engine, while this one is dependent on the Python interpreter true. ciao, Anto ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Link errors while translating with VS2010 and 64bit
Hi Tasos, On Wed, Feb 2, 2011 at 1:15 AM, Tasos Vogiatzoglou tvog...@gmail.com wrote: You are right. My mistake. It was picking the 2.6 32bit python. Now is working and tests are passing A few extra notes. What is needed to make this work is three-fold: * make sure that in the C code, intptr_t is generated instead of long for lltype.Signed, and similarly uintptr_t instead of unsigned long for lltype.Unsigned. This involves mainly fixing translator/c/primitive.py to declare types using intptr_t instead of long, and probably fixing a number of other uses of long in the .py files in that directory. * make sure that in the hand-written headers translator/c/src/*.h, we use intptr_t instead of long whenever it's needed (mostly everywhere, I suppose, but not absolutely everywhere, e.g. not in calls to external C functions that are declared to take or return a long). The changes above should fix the generation of code. Now the third part is the longest, probably. You need to distinguish the two types used in RPython, which are lltype.Signed and rffi.LONG, and which are equal so far. Of course rffi.LONG should remain equivalent to the C long. This probably needs looking around in all modules and in pypy/rlib/, to make sure that we use the correct one, either lltype.Signed or rffi.LONG. Fortunately, errors resulting from this confusion will probably just give translation-time errors, so we can fix the places one after the other. But you need to carefully check the declaration of external functions that are implemented in translator/c/src/*.h, like pypy/module/signal/interp_signal.py (this one happens to use mostly rffi.INT, so it should work right now, with the exception of LONG_STRUCT that uses a lltype.Signed). A bientôt, Armin. ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Link errors while translating with VS2010 and 64bit
Armin, thanks for the directions. I'll start working on them and I'll report back. Regards, Tasos On 3 Φεβ 2011, at 14:30, Armin Rigo ar...@tunes.org wrote: Hi Tasos, On Wed, Feb 2, 2011 at 1:15 AM, Tasos Vogiatzoglou tvog...@gmail.com wrote: You are right. My mistake. It was picking the 2.6 32bit python. Now is working and tests are passing A few extra notes. What is needed to make this work is three-fold: * make sure that in the C code, intptr_t is generated instead of long for lltype.Signed, and similarly uintptr_t instead of unsigned long for lltype.Unsigned. This involves mainly fixing translator/c/primitive.py to declare types using intptr_t instead of long, and probably fixing a number of other uses of long in the .py files in that directory. * make sure that in the hand-written headers translator/c/src/*.h, we use intptr_t instead of long whenever it's needed (mostly everywhere, I suppose, but not absolutely everywhere, e.g. not in calls to external C functions that are declared to take or return a long). The changes above should fix the generation of code. Now the third part is the longest, probably. You need to distinguish the two types used in RPython, which are lltype.Signed and rffi.LONG, and which are equal so far. Of course rffi.LONG should remain equivalent to the C long. This probably needs looking around in all modules and in pypy/rlib/, to make sure that we use the correct one, either lltype.Signed or rffi.LONG. Fortunately, errors resulting from this confusion will probably just give translation-time errors, so we can fix the places one after the other. But you need to carefully check the declaration of external functions that are implemented in translator/c/src/*.h, like pypy/module/signal/interp_signal.py (this one happens to use mostly rffi.INT, so it should work right now, with the exception of LONG_STRUCT that uses a lltype.Signed). A bientôt, Armin. ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] pypy jit
Hi, On Wed, Feb 2, 2011 at 4:48 PM, Maciej Fijalkowski fij...@gmail.com wrote: As of now there is no such guide. However, the rule of thumb is simple is better than complex. To some extend that is true; however, note that the second rule is but even on messily complicated code the JIT can do something :-) For example, translate.py was not written with the JIT in mind at all, but turns out to be twice as fast on PyPy. To get a better summary, I think that we can say that, to some extent, there is little point in spending ages tweaking your Python code to make it more JIT-friendly. If there is any Python code that would get seriously faster by some trivial tweaking, then it's somehow a bug, and we need to fix it. Of course, if you use introspection of the stack frames or even pdb (the Python debugger) all over the place, you need to expect the code to be hard to optimize for us. But I guess that it should somehow be clear. For the rest, any decision that can be resolved locally can nicely be optimized by the JIT, whatever the answer is (for example, is this '+' going to just add integers, or does it invoke some __add__() method?). A bientôt, Armin. ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] testing floating point
Maciej Fijalkowski, 03.02.2011 11:14: On Thu, Feb 3, 2011 at 11:56 AM, Fredrik Johansson wrote: On Thu, Feb 3, 2011 at 8:01 AM, Maciej Fijalkowski wrote: * Use list comprehension instead of generator expression. I hope PyPy can do more in the future to speed up generator expressions. It doesn't speed up things by much. Yeah, I can imagine we can improve on this, but it's also a bit hard. Does PyPy generate inlined looping code for them if applicable? (e.g. for any(), all(), sum(), sorted(), ...) Stefan ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] testing floating point
On Thu, Feb 3, 2011 at 6:15 PM, Stefan Behnel stefan...@behnel.de wrote: Maciej Fijalkowski, 03.02.2011 11:14: On Thu, Feb 3, 2011 at 11:56 AM, Fredrik Johansson wrote: On Thu, Feb 3, 2011 at 8:01 AM, Maciej Fijalkowski wrote: * Use list comprehension instead of generator expression. I hope PyPy can do more in the future to speed up generator expressions. It doesn't speed up things by much. Yeah, I can imagine we can improve on this, but it's also a bit hard. Does PyPy generate inlined looping code for them if applicable? (e.g. for any(), all(), sum(), sorted(), ...) We don't inline generator expr. The reason is that it's hard to judge when generator frame escapes. We can probably do something with it, but it's not done yet. Stefan ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
