Re: [pypy-dev] Slow sqlite user defined functions with pypy.

[Alex Gaynor]

On Wed, Nov 16, 2011 at 8:24 PM, William ML Leslie <
william.leslie@gmail.com> wrote:

> Ack.
>
> On 17 November 2011 12:23, William ML Leslie
>  wrote:
> > On 17 November 2011 12:13, Elefterios Stamatogiannakis 
> wrote:
> >> Pypy seems to not jit at all when a (pypy) Python function is called
> from C.
> >
> > Calls to native functions must be residualised, as there is no way to
> > tell what state gives rise to the call to the UDF.
> >
> > If there was a loop inside the UDF, that would still be compiled.  But
> > the loop that occurs in the query must just call the C function in the
> > usual way, as the JIT has no idea what it might do.
>
> --
> William Leslie
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>

The JIT compiles functions without loops too now, so this should be jitted.

Alex

-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Slow sqlite user defined functions with pypy.

[William ML Leslie]

Ack.

On 17 November 2011 12:23, William ML Leslie
 wrote:
> On 17 November 2011 12:13, Elefterios Stamatogiannakis  
> wrote:
>> Pypy seems to not jit at all when a (pypy) Python function is called from C.
>
> Calls to native functions must be residualised, as there is no way to
> tell what state gives rise to the call to the UDF.
>
> If there was a loop inside the UDF, that would still be compiled.  But
> the loop that occurs in the query must just call the C function in the
> usual way, as the JIT has no idea what it might do.

-- 
William Leslie
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

[pypy-dev] Slow sqlite user defined functions with pypy.

[Elefterios Stamatogiannakis]

The following code is a lot slower with pypy as compared to CPython. The 
code mainly measures the time taken to execute a simple SQLite user 
defined function (UDF) and a more complex one, 100 times each.


Execution time for both queries is:
CPython 2.7: 7 sec 489 msec
Pypy nightly build: 28 sec 753 msec

Execution time for simple query only is:
CPython 2.7: 1 sec 39 msec
Pypy nightly build: 13 sec 645 msec

Pypy seems to not jit at all when a (pypy) Python function is called from C.

Also based on the simple query times, pypy seems to have massive 
overhead (nearly 13 times slower) when executing callbacks from C code.


lefteris.

--- code --

import sqlite3
import datetime

# Simple function
def testfun(*args):

return 1

# Complex function
def sectohuman(*args):

secs=int(args[0])
h=''
days=secs/86400
if days > 0:
h+=str(days)+' day'
if days > 1:
h+='s'
h+=' '
secs=secs % 86400
hours=secs/3600
if hours > 0:
h+=str(hours)+' hour'
if hours > 1:
h+='s'
h+=' '
secs=secs % 3600
mins=secs/60
if mins > 0:
h+=str(mins)+' min '
secs=secs % 60
if secs > 0:
h+=str(secs)+' sec'

return h

con=sqlite3.Connection('')

con.create_function('testfun', -1, testfun)
con.create_function('sectohuman', -1, sectohuman)

cur=con.cursor()

cur.execute('create table l(a);')
cur.execute('begin;')

for i in xrange(100):
cur.execute('insert into l values(?)',(i,))

before=datetime.datetime.now()

# Simple query
a=list(cur.execute('select sum(testfun(a)) as s from l'))

# Complex query
a=list(cur.execute('select sum(length(sectohuman(a))) as s from l'))

after=datetime.datetime.now()

tmdiff=after-before

print "Execution time is %s min. %s sec %s msec" 
%((int(tmdiff.days)*24*60+(int(tmdiff.seconds)/60),(int(tmdiff.seconds)%60),(int(tmdiff.microseconds)/1000)))


___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Pure Python Math Library

[Blaine]

Thanks Charlie, that's really helpful!

Yeah ceil is easy. Pow can be done if log() and exp() are available, and
exp() shouldn't be too hard I think.
def pow(x,y):
return exp(y*log(x))

Blaine


On Wed, Nov 16, 2011 at 1:03 PM, Charlie Hui  wrote:

> http://en.literateprograms.org/Logarithm_Function_(Python)
> ceil seems pretty easy to implement...
> pow? Integer only?
>
> --C
>
> --
> *From:* Blaine 
> *To:* pypy-dev@python.org
> *Sent:* Wednesday, November 16, 2011 9:21 AM
> *Subject:* [pypy-dev] Pure Python Math Library
>
> Does anyone know of a pure python math library? I've been playing around
> with berp , which is a python3 to
> haskell translator and compiler, and it works great as long as you don't go
> crazy with C extensions. It's highly experimental but fun to play around
> with. The only thing that I really miss is being able to use the math
> module. I asked the maintainer if it is possible to map into haskell's math
> library, but in the mean time a pure python math library would fit nicely
> since it would be compiled along with the rest of the python.
>
> I'm looking for log, log10, ceil, and pow mostly for my personal needs.
>
> It's funny now that things like pypy and berp exist. I find myself trying
> to locate pure python routines (like DFT) that would have no reason to
> exist with cpython, but make lots of sense with pypy.
>
> Anyway, just wondering if anyone had heard of such a thing. How do you
> even implement log?
>
> Thanks,
> Blaine
>
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>
>
>
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Pure Python Math Library

[Charlie Hui]

http://en.literateprograms.org/Logarithm_Function_(Python)

ceil seems pretty easy to implement...
pow? Integer only?

--C



From: Blaine 
To: pypy-dev@python.org
Sent: Wednesday, November 16, 2011 9:21 AM
Subject: [pypy-dev] Pure Python Math Library


Does anyone know of a pure python math library? I've been playing around with 
berp, which is a python3 to haskell translator and compiler, and it works great 
as long as you don't go crazy with C extensions. It's highly experimental but 
fun to play around with. The only thing that I really miss is being able to use 
the math module. I asked the maintainer if it is possible to map into haskell's 
math library, but in the mean time a pure python math library would fit nicely 
since it would be compiled along with the rest of the python.

I'm looking for log, log10, ceil, and pow mostly for my personal needs.

It's funny now that things like pypy and berp exist. I find myself trying to 
locate pure python routines (like DFT) that would have no reason to exist with 
cpython, but make lots of sense with pypy.

Anyway, just wondering if anyone had heard of such a thing. How do you even 
implement log?

Thanks,
Blaine

___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Pure Python Math Library

[Blaine]

Oh, neat. I didn't know that. Thank you for the information, I didn't know
that was possible.

Blaine


On Wed, Nov 16, 2011 at 12:40 PM, Alex Gaynor  wrote:

>
>
> On Wed, Nov 16, 2011 at 12:38 PM, Blaine  wrote:
>
>> I imagine pypy uses libc math through ctypes, since pypy and ctypes play
>> well together.
>>
>> Thanks for all the tips, I'll probably just wait to see what the
>> maintainer thinks about mapping to haskell's math library.
>> Blaine
>>
>>
>>
>> On Wed, Nov 16, 2011 at 12:36 PM, Benjamin Peterson 
>> wrote:
>>
>>> 2011/11/16 Blaine :
>>> > Thanks Alex and Benjamin.
>>> >   I'm sorry - you're right it isn't exactly related to pypy. I hope I
>>> didn't
>>> > break any rules. I was hoping that someone else may have come across
>>> this
>>> > because the only time I've needed to port compiled modules to python
>>> is when
>>> > I wanted to use them with pypy.
>>> > Blaine
>>>
>>> I don't PyPy will be very helpful to you, since it uses the libc math
>>> functions.
>>>
>>>
>>>
>>> --
>>> Regards,
>>> Benjamin
>>>
>>
>>
> No, PyPy has an RPython math module which calls libc.
>
> Alex
>
> --
> "I disapprove of what you say, but I will defend to the death your right
> to say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
> "The people's good is the highest law." -- Cicero
>
>
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Pure Python Math Library

[Alex Gaynor]

On Wed, Nov 16, 2011 at 12:38 PM, Blaine  wrote:

> I imagine pypy uses libc math through ctypes, since pypy and ctypes play
> well together.
>
> Thanks for all the tips, I'll probably just wait to see what the
> maintainer thinks about mapping to haskell's math library.
> Blaine
>
>
>
> On Wed, Nov 16, 2011 at 12:36 PM, Benjamin Peterson 
> wrote:
>
>> 2011/11/16 Blaine :
>> > Thanks Alex and Benjamin.
>> >   I'm sorry - you're right it isn't exactly related to pypy. I hope I
>> didn't
>> > break any rules. I was hoping that someone else may have come across
>> this
>> > because the only time I've needed to port compiled modules to python is
>> when
>> > I wanted to use them with pypy.
>> > Blaine
>>
>> I don't PyPy will be very helpful to you, since it uses the libc math
>> functions.
>>
>>
>>
>> --
>> Regards,
>> Benjamin
>>
>
>
No, PyPy has an RPython math module which calls libc.

Alex

-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Pure Python Math Library

[Blaine]

I imagine pypy uses libc math through ctypes, since pypy and ctypes play
well together.

Thanks for all the tips, I'll probably just wait to see what the maintainer
thinks about mapping to haskell's math library.
Blaine


On Wed, Nov 16, 2011 at 12:36 PM, Benjamin Peterson wrote:

> 2011/11/16 Blaine :
> > Thanks Alex and Benjamin.
> >   I'm sorry - you're right it isn't exactly related to pypy. I hope I
> didn't
> > break any rules. I was hoping that someone else may have come across this
> > because the only time I've needed to port compiled modules to python is
> when
> > I wanted to use them with pypy.
> > Blaine
>
> I don't PyPy will be very helpful to you, since it uses the libc math
> functions.
>
>
>
> --
> Regards,
> Benjamin
>
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Pure Python Math Library

[Benjamin Peterson]

2011/11/16 Blaine :
> Thanks Alex and Benjamin.
>   I'm sorry - you're right it isn't exactly related to pypy. I hope I didn't
> break any rules. I was hoping that someone else may have come across this
> because the only time I've needed to port compiled modules to python is when
> I wanted to use them with pypy.
> Blaine

I don't PyPy will be very helpful to you, since it uses the libc math functions.



-- 
Regards,
Benjamin
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Pure Python Math Library

[Alex Gaynor]

On Wed, Nov 16, 2011 at 12:26 PM, Blaine  wrote:

> I think that python's math module (which I use) is a compiled C extension,
> right? I'm looking for pure python that berp can use.
>
> Blaine
>
>
>
> On Wed, Nov 16, 2011 at 12:25 PM, Benjamin Peterson 
> wrote:
>
>> 2011/11/16 Blaine :
>> > Does anyone know of a pure python math library? I've been playing around
>> > with berp, which is a python3 to haskell translator and compiler, and it
>> > works great as long as you don't go crazy with C extensions. It's highly
>> > experimental but fun to play around with. The only thing that I really
>> miss
>> > is being able to use the math module. I asked the maintainer if it is
>> > possible to map into haskell's math library, but in the mean time a pure
>> > python math library would fit nicely since it would be compiled along
>> with
>> > the rest of the python.
>> > I'm looking for log, log10, ceil, and pow mostly for my personal needs.
>>
>> Well, python has a math module too:
>>
>> >>> import math
>> >>> math.log(23)
>> 3.1354942159291497
>>
>>
>>
>> --
>> Regards,
>> Benjamin
>>
>
>
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>
>
It's a part of the standard library, it's implemented however the Python VM
provides it.

Alex

-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Pure Python Math Library

[Blaine]

Thanks Alex and Benjamin.
  I'm sorry - you're right it isn't exactly related to pypy. I hope I
didn't break any rules. I was hoping that someone else may have come across
this because the only time I've needed to port compiled modules to python
is when I wanted to use them with pypy.

Blaine


On Wed, Nov 16, 2011 at 12:29 PM, Alex Gaynor  wrote:

>
>
> On Wed, Nov 16, 2011 at 12:26 PM, Blaine  wrote:
>
>> I think that python's math module (which I use) is a compiled C
>> extension, right? I'm looking for pure python that berp can use.
>>
>> Blaine
>>
>>
>>
>> On Wed, Nov 16, 2011 at 12:25 PM, Benjamin Peterson 
>> wrote:
>>
>>> 2011/11/16 Blaine :
>>> > Does anyone know of a pure python math library? I've been playing
>>> around
>>> > with berp, which is a python3 to haskell translator and compiler, and
>>> it
>>> > works great as long as you don't go crazy with C extensions. It's
>>> highly
>>> > experimental but fun to play around with. The only thing that I really
>>> miss
>>> > is being able to use the math module. I asked the maintainer if it is
>>> > possible to map into haskell's math library, but in the mean time a
>>> pure
>>> > python math library would fit nicely since it would be compiled along
>>> with
>>> > the rest of the python.
>>> > I'm looking for log, log10, ceil, and pow mostly for my personal needs.
>>>
>>> Well, python has a math module too:
>>>
>>> >>> import math
>>> >>> math.log(23)
>>> 3.1354942159291497
>>>
>>>
>>>
>>> --
>>> Regards,
>>> Benjamin
>>>
>>
>>
>> ___
>> pypy-dev mailing list
>> pypy-dev@python.org
>> http://mail.python.org/mailman/listinfo/pypy-dev
>>
>>
> It's a part of the standard library, it's implemented however the Python
> VM provides it.
>
> Alex
>
> --
> "I disapprove of what you say, but I will defend to the death your right
> to say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
> "The people's good is the highest law." -- Cicero
>
>
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Pure Python Math Library

[Peter Cock]

On Wed, Nov 16, 2011 at 5:27 PM, Benjamin Peterson  wrote:
> 2011/11/16 Blaine :
>> I think that python's math module (which I use) is a compiled C extension,
>> right? I'm looking for pure python that berp can use.
>
> I'm not really sure why this is relevant to pypy.
>

How does pypy implement the math module? It that pure Python?

Peter
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Pure Python Math Library

[Benjamin Peterson]

2011/11/16 Blaine :
> I think that python's math module (which I use) is a compiled C extension,
> right? I'm looking for pure python that berp can use.

I'm not really sure why this is relevant to pypy.



-- 
Regards,
Benjamin
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Pure Python Math Library

[Blaine]

I think that python's math module (which I use) is a compiled C extension,
right? I'm looking for pure python that berp can use.

Blaine


On Wed, Nov 16, 2011 at 12:25 PM, Benjamin Peterson wrote:

> 2011/11/16 Blaine :
> > Does anyone know of a pure python math library? I've been playing around
> > with berp, which is a python3 to haskell translator and compiler, and it
> > works great as long as you don't go crazy with C extensions. It's highly
> > experimental but fun to play around with. The only thing that I really
> miss
> > is being able to use the math module. I asked the maintainer if it is
> > possible to map into haskell's math library, but in the mean time a pure
> > python math library would fit nicely since it would be compiled along
> with
> > the rest of the python.
> > I'm looking for log, log10, ceil, and pow mostly for my personal needs.
>
> Well, python has a math module too:
>
> >>> import math
> >>> math.log(23)
> 3.1354942159291497
>
>
>
> --
> Regards,
> Benjamin
>
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Pure Python Math Library

[Benjamin Peterson]

2011/11/16 Blaine :
> Does anyone know of a pure python math library? I've been playing around
> with berp, which is a python3 to haskell translator and compiler, and it
> works great as long as you don't go crazy with C extensions. It's highly
> experimental but fun to play around with. The only thing that I really miss
> is being able to use the math module. I asked the maintainer if it is
> possible to map into haskell's math library, but in the mean time a pure
> python math library would fit nicely since it would be compiled along with
> the rest of the python.
> I'm looking for log, log10, ceil, and pow mostly for my personal needs.

Well, python has a math module too:

>>> import math
>>> math.log(23)
3.1354942159291497



-- 
Regards,
Benjamin
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

[pypy-dev] Pure Python Math Library

[Blaine]

Does anyone know of a pure python math library? I've been playing around
with berp , which is a python3 to
haskell translator and compiler, and it works great as long as you don't go
crazy with C extensions. It's highly experimental but fun to play around
with. The only thing that I really miss is being able to use the math
module. I asked the maintainer if it is possible to map into haskell's math
library, but in the mean time a pure python math library would fit nicely
since it would be compiled along with the rest of the python.

I'm looking for log, log10, ceil, and pow mostly for my personal needs.

It's funny now that things like pypy and berp exist. I find myself trying
to locate pure python routines (like DFT) that would have no reason to
exist with cpython, but make lots of sense with pypy.

Anyway, just wondering if anyone had heard of such a thing. How do you even
implement log?

Thanks,
Blaine
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Updated 'High Performance Python' tutorial (the one from EuroPython 2011)

[Jérémie Roquet]

2011/11/16 Ian Ozsvald :
> From memory the 'native' flag made a difference (I think it allows use
> of SSE?).

It depends on the machine, of course, but yes, on most machines it
enables SSE. Just compare the output of

$ < /dev/null g++ -E -v - |& grep cc1

and

$ < /dev/null g++ -march=native -E -v - |& grep cc1

The following flags are added for me :
-march=core2 -mcx16 -msahf --param l1-cache-size=32 --param
l1-cache-line-size=64 --param l2-cache-size=4096 -mtune=core2

-mcx16 and -msahf enable some additional instructions (see
http://gcc.gnu.org/onlinedocs/gcc/i386-and-x86_002d64-Options.html)
-mtune=core2 enables Intel's 64-bit extensions, MMX, SSE, SSE2, SSE3 and SSSE3

Best regards,

-- 
Jérémie
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Updated 'High Performance Python' tutorial (the one from EuroPython 2011)

[Ian Ozsvald]

>From memory the 'native' flag made a difference (I think it allows use
of SSE?). I guess that is something I'll normalise for a future v0.3
release of my handbook :-)
Cheers,
Ian.

2011/11/15 Jérémie Roquet :
> Hi,
>
> 2011/11/15 Armin Rigo :
>> On Tue, Nov 15, 2011 at 15:54, Ian Ozsvald  wrote:
>>> ShedSkin (from memory)
>>> requests fast math and a few other things in the generated Makefile.
>>
>> Ah, it is cheating that way.  Indeed, I didn't try to play with gcc
>> options; I just used -O2 (or -O3, which made no difference).
>
> FYI, here is the default FLAGS file for shedskin:
>
> CC=g++
> CCFLAGS=-O2 -march=native -Wno-deprecated $(CPPFLAGS)
> LFLAGS=-lgc -lpcre $(LDFLAGS)
>
> But of course you can change the compiler and play with its flags to
> improve performance.
>
> Best regards,
>
> --
> Jérémie
>



-- 
Ian Ozsvald (A.I. researcher)
i...@ianozsvald.com

http://IanOzsvald.com
http://MorConsulting.com/
http://StrongSteam.com/
http://SocialTiesApp.com/
http://TheScreencastingHandbook.com
http://FivePoundApp.com/
http://twitter.com/IanOzsvald
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] [pypy-commit] pypy ppc-jit-backend: setarrayitem and getarrayitem offsets are immediate values.

[Armin Rigo]

Hi Maciej,

On Wed, Nov 16, 2011 at 08:05, Maciej Fijalkowski  wrote:
> Unless I'm missing something offset for get/setarrayitem are not
> necesarilly intermediate values

Assuming you mean "immediate" values instead of "intermediate": the
content of the checkin shows that it meant that the *scale* is a
constant (address = base + baseoffset + item * scale).


A bientôt,

Armin.
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev