Re: [pypy-dev] Slow sqlite user defined functions with pypy.
On 11/17/2011 02:56 AM, Alex Gaynor wrote: The JIT compiles functions without loops too now, so this should be jitted. ctypes callbacks still go through the old _rawffi, so it's possible that this introduces some unneeded overhead. ciao, Anto ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] Slow sqlite user defined functions with pypy.
Thanks to all for your answers. I took some time to think some more about the results and: For the simple function (which returns 1), CPython roughly takes 1 sec and pypy 13 secs. IMHO, this case reveals pypy's callback overhead. For the complex function case, CPython roughly takes 6 secs and pypy 15 secs. If we assume that the overhead will be the same as in the simple function case, and subtract it, then CPython roughly took 5 secs to execute the complex function and pypy 2 secs. From the above, i think that my jit guess was wrong (as Alex said), and jit indeed works, but the overhead of the pypy callbacks is indeed quite large. I know that callbacks from C code aren't so frequent in Python programs, but my project: http://code.google.com/p/madis/ lives on them. So i would be glad if it could be solved (and madis's data processing would become a lot faster). Alternatively, is there something that i could do in my code that would overcome this overhead? Like signalling to the jit engine that this function should be always jitted and to do not waste time updating statistics (or other tracking information) on it? thanks lefteris. On 17/11/11 10:42, Antonio Cuni wrote: On 11/17/2011 02:56 AM, Alex Gaynor wrote: The JIT compiles functions without loops too now, so this should be jitted. ctypes callbacks still go through the old _rawffi, so it's possible that this introduces some unneeded overhead. ciao, Anto ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] Slow sqlite user defined functions with pypy.
Ack. On 17 November 2011 12:23, William ML Leslie william.leslie@gmail.com wrote: On 17 November 2011 12:13, Elefterios Stamatogiannakis est...@gmail.com wrote: Pypy seems to not jit at all when a (pypy) Python function is called from C. Calls to native functions must be residualised, as there is no way to tell what state gives rise to the call to the UDF. If there was a loop inside the UDF, that would still be compiled. But the loop that occurs in the query must just call the C function in the usual way, as the JIT has no idea what it might do. -- William Leslie ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] Slow sqlite user defined functions with pypy.
On Wed, Nov 16, 2011 at 8:24 PM, William ML Leslie william.leslie@gmail.com wrote: Ack. On 17 November 2011 12:23, William ML Leslie william.leslie@gmail.com wrote: On 17 November 2011 12:13, Elefterios Stamatogiannakis est...@gmail.com wrote: Pypy seems to not jit at all when a (pypy) Python function is called from C. Calls to native functions must be residualised, as there is no way to tell what state gives rise to the call to the UDF. If there was a loop inside the UDF, that would still be compiled. But the loop that occurs in the query must just call the C function in the usual way, as the JIT has no idea what it might do. -- William Leslie ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev The JIT compiles functions without loops too now, so this should be jitted. Alex -- I disapprove of what you say, but I will defend to the death your right to say it. -- Evelyn Beatrice Hall (summarizing Voltaire) The people's good is the highest law. -- Cicero ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
