Re: [pypy-dev] Partial Escape Analysis and Scalar Replacement

[Amaury Forgeot d7;Arc]

Hi George,

2016-02-17 0:23 GMT+01:00 George Papanikolaou :

> Hey list,
>
> I'd like to announce that I will attempt to implement an optimization
> technique
> in PyPy as part of my thesis for my CS diplomma.


Welcome to PyPy development!
PyPy was designed upfront to experiment with new development techniques,
and we always need more speed :-)

I wish you good luck!


>

In addition to the local advisor from my uni, Carl Friedrich Bolz (@cfbolz)
> accepted to "mentor" me for the attempt.
>
> I'll be reporting the progress frequently here (so you will hear soon more
> details) and of course I'll also be availabe on the IRC channel, which I'll
> also use for the daily communication with Carl and for any other questions
> or
> problems I may have with the codebase.
>
> The optimization is based on the paper "Partial Escape Analysis and Scalar
> Replacement on Java" by L. Stadler, T. Würthinger and H. Mössenböck.
>
> Feel free to add any comments, remarks, ideas, etc. Have a good day.
>
> Cheers,
> George
>
> --
> George 'papanikge' Papanikolaou
> ___________
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>



-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] [pypy-commit] pypy py3.3: Fix unicode.capitalize() test to pass with CPython3.3,

[Amaury Forgeot d7;Arc]

2016-02-01 9:59 GMT+01:00 Maciej Fijalkowski :

> Those things are probably why pypy3 is so much slower than pypy2
>

Agreed. I prefer to have things correct first, and work on performance
later.
For this particular case though, do you have an idea on how to improve this?
Pass the UnicodeBuilder to unicodedb.toupper_full()?


>
> On Mon, Feb 1, 2016 at 12:42 AM, amauryfa  wrote:
> > Author: Amaury Forgeot d'Arc 
> > Branch: py3.3
> > Changeset: r82021:44aa48e4d16a
> > Date: 2016-02-01 00:31 +0100
> > http://bitbucket.org/pypy/pypy/changeset/44aa48e4d16a/
> >
> > Log:Fix unicode.capitalize() test to pass with CPython3.3, and
> implement
> > it for PyPy. Probably not the fastest implementation...
> >
> > diff --git a/pypy/objspace/std/test/test_unicodeobject.py
> b/pypy/objspace/std/test/test_unicodeobject.py
> > --- a/pypy/objspace/std/test/test_unicodeobject.py
> > +++ b/pypy/objspace/std/test/test_unicodeobject.py
> > @@ -217,7 +217,7 @@
> >  # check that titlecased chars are lowered correctly
> >  # \u1ffc is the titlecased char
> >  assert ('\u1ff3\u1ff3\u1ffc\u1ffc'.capitalize() ==
> > -'\u1ffc\u1ff3\u1ff3\u1ff3')
> > +'\u03a9\u0399\u1ff3\u1ff3\u1ff3')
> >  # check with cased non-letter chars
> >  assert ('\u24c5\u24ce\u24c9\u24bd\u24c4\u24c3'.capitalize() ==
> >  '\u24c5\u24e8\u24e3\u24d7\u24de\u24dd')
> > diff --git a/pypy/objspace/std/unicodeobject.py
> b/pypy/objspace/std/unicodeobject.py
> > --- a/pypy/objspace/std/unicodeobject.py
> > +++ b/pypy/objspace/std/unicodeobject.py
> > @@ -155,13 +155,16 @@
> >  return unicodedb.islinebreak(ord(ch))
> >
> >  def _upper(self, ch):
> > -return unichr(unicodedb.toupper(ord(ch)))
> > +return u''.join([unichr(x) for x in
> > + unicodedb.toupper_full(ord(ch))])
> >
> >  def _lower(self, ch):
> > -return unichr(unicodedb.tolower(ord(ch)))
> > +return u''.join([unichr(x) for x in
> > + unicodedb.tolower_full(ord(ch))])
> >
> >  def _title(self, ch):
> > -return unichr(unicodedb.totitle(ord(ch)))
> > +return u''.join([unichr(x) for x in
> > + unicodedb.totitle_full(ord(ch))])
> >
> >  def _newlist_unwrapped(self, space, lst):
> >      return space.newlist_unicode(lst)
> > ___
> > pypy-commit mailing list
> > pypy-com...@python.org
> > https://mail.python.org/mailman/listinfo/pypy-commit
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>



-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Looking for clues to consistent Seg Fault in PyPy 2.6.1

[Amaury Forgeot d7;Arc]

2015-10-12 11:10 GMT+02:00 Armin Rigo :

> Hi Jeff,
>
> Ah, I think I understand now what is going on.  The special-casing for
> PyPy should not be needed at all, but when I removed it, I get
> assertion failures.  Now I get the reason---yes, it is yet another bug
> inside cpyext, caused by yet another access pattern of "PyObject *"
> that is somehow unusual but should be allowed.
>
> I can try to fix it inside cpyext, and then the "if IS_PYPY" cases in
> proxy.pxi should be removed.  (I'm unsure if it is easy to change them
> to "if the version of PyPy is <= 2.6.1"...)
>

We define it as a string (in pypy/module/cpyext/include/patchlevel.h)
#define PYPY_VERSION "2.6.0-alpha0"

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Looking for clues to consistent Seg Fault in PyPy 2.6.1

[Amaury Forgeot d7;Arc]

2015-10-12 9:10 GMT+02:00 Armin Rigo :

> Hi Laura,
>
> On Sun, Oct 11, 2015 at 10:59 AM, Laura Creighton  wrote:
> > Has https://github.com/amauryfa/lxml/tree/lxml-cffi been renamed
> > to https://github.com/amauryfa/lxml/tree/cffi  ?
> > The first one now is 404 on github, despite being the first
> > google hit for lxml-cffi
>
> Uh, that's very new.  I accessed that page about two days ago.
> Amaury, do you have a comment?
>

It's quite possible that early versions of the repo had this branch, and I
somehow deleted it;
I cannot remember how.
But I'm sure nothing has changed in the last months.

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Working version of pypy3 that support Python 3.3

[Amaury Forgeot d7;Arc]

2015-10-07 13:26 GMT+02:00 Armin Rigo :

> Hi Ofir,
>
> On Wed, Oct 7, 2015 at 10:51 AM, Ofir Shwartz  wrote:
> > Is there any update on that matter?
> > I tried it now and got the same results for the most updated jit and
> no-jit
> > pypy3.3.
>
> I looked into it a bit more.  (I think it's sad that no-one had a look
> since June, but I suppose it just shows that our pypy3 involvement is
> rather thin.)  The problem is not that the archive is incomplete; it
> is complete as far as I can see.  The problem is that the binary only
> works if it runs inside a directory with exactly the same name as it
> was compiled with.
>
> The crash I get is in _getfilesystemencoding, called from
> pypy/module/sys/__init__py:startup().  It is indeed extremely early,
> before we have got any chance to set up sys.path correctly.  I tried
> to fix the problem but gave up after hitting several nested issues.
> The failing check is pushed in 88a921f5ae6b; somebody please fix it
> :-)
>

Maybe move setup_bootstrap_path() from app_main.py to
module/sys/__init__.py?


> A bientôt,
>
> Armin.
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>



-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Working version of pypy3 that support Python 3.3

[Amaury Forgeot d7;Arc]

2015-10-07 10:51 GMT+02:00 Ofir Shwartz :

> Is there any update on that matter?
> I tried it now and got the same results for the most updated jit and
> no-jit pypy3.3.
> Obviously, pypy 3.2 runs perfectly good.
> Please assist.
>

There are still many features missing from our pypy3.3.
Help needed!
You can track the progress here:
http://buildbot.pypy.org/summary?branch=py3.3


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] About gurobipy

[Amaury Forgeot d7;Arc]

2015-09-30 20:10 GMT+02:00 Vasily Evseenko :

> Hi,
> Could you explain about closed file objects (do you mean open file
> descriptors)?
> I've never heard about this restriction before.
>

PyPy uses a different garbage collection strategy, and unlike CPython,
objects are not deallocated as soon as the variable goes out of scope:
http://pypy.readthedocs.org/en/latest/cpython_differences.html#differences-related-to-garbage-collection-strategies

FWIW, Java has a very similar behavior.




>
> 30.09.2015 19:35, Phyo Arkar пишет:
>
>
> On Wed, Sep 30, 2015 at 6:54 PM, Luis José Novoa 
> wrote:
>
>> gurobipy
>
>
> Never heard of it.
>
> Here are things PyPy Can help:
>
> - If it is Pure Python
> - If CExtensions are with CFFI
> - If file objects (if any) are closed
>
> PyPy will work fine and you can expect average 7x performance gain , up to
> 20-30x.
>
>
>
>
> ___
> pypy-dev mailing 
> listpypy-dev@python.orghttps://mail.python.org/mailman/listinfo/pypy-dev
>
>
>
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
>


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] pypy 2.6 memory leak bug!! very importent

[Amaury Forgeot d7;Arc]

Hi,

2015-09-14 8:59 GMT+02:00 于大伟 :

> hi,
> i use pypy 2.6 with BeautifulSoup module and pymongo.  i make a spider
> with the pypy.
> when i run pypy ,i found a problem ,the pypy use memory higher then 6GB
> when i run pypy at once , the pypy use only  240mb memory!
> the screenshot in the attachment!
>

This might be because of the differences in memory management.
Unlike CPython, PyPy does not free resources as soon as the last reference
to an object goes away. See
http://pypy.readthedocs.org/en/latest/cpython_differences.html#differences-related-to-garbage-collection-strategies

The solution is often to close resources explicitly or use "with" blocks.





>
>
>
>
>
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
>


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] pypy segment fault bug

[Amaury Forgeot d7;Arc]

Hi,

Please be more specific and indicate what kind of program you were running:
which extensions modules were used, and which function was called.

For the redis module: which error did you get?


2015-09-14 8:52 GMT+02:00 于大伟 :

> hi manager:
>   i use pypy do something. i found a bug when i run my code!
>
>
> Fatal error in cpyext, CPython compatibility layer, calling PyTuple_New
> Either report a bug or consider not using this particular extension
> 
> RPython traceback:
>   File "pypy_module_pypyjit_interp_jit.c", line 340, in portal_5
>   File "pypy_interpreter_pyopcode.c", line 3578, in
> handle_bytecode__AccessDirect_None
>   File "pypy_interpreter_pyopcode.c", line 5853, in
> dispatch_bytecode__AccessDirect_None
> Segmentation fault (core dumped)
>
>  and i found pypy can not use the redis module, i use pypy 2.6.
>
> https://github.com/andymccurdy/redis-py
>
>
>
>
>
>
>
>
>
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
>


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] problems with pypy and pip

[Amaury Forgeot d7;Arc]

2015-07-07 19:20 GMT+02:00 Matti Picus :

>  This is all documented in our FAQ
> http://pypy.readthedocs.org/en/latest/faq.html#do-cpython-extension-modules-work-with-pypy
>
> Some modules, like wxPython, that use the python c API or ctypes will
> either not work with PyPy or will be much slower than cPython. We highly
> recommend cffi as an alternative foreign function interface, since it is
> nicer than ctypes and is fast on PyPy, but it may be a while until module
> authors see the light of the one true way.
>

Old versions of wxPython ("classic") used SWIG to wrap the C++ libraries,
and could be compiled with PyPy. It worked, but it was a long time ago, was
the result was quite slow...
http://morepypy.blogspot.fr/2010/05/running-wxpython-on-top-of-pypy.html
(unfortunately the screenshots are lost)

Newer versions of wxPython ("Phoenix") use SIP to wrap the C++ libraries,
and there is no way it can be supported directly by PyPy (SIP uses
metaclasses with custom tp_alloc slots to subclass int... I tried).

There was an attempt to port have SIP file generate a cffi interface, it
did not go to completion but looks very promising.
There is a nice story here: http://waedt.blogspot.fr/
It seems that all basic infrastructure are there, but many widgets extend
SIP wrappers and use the C API: they need to be rewritten in a
cffi-friendly fashion.

Good luck with wxPython.


>
> The wiki page https://bitbucket.org/pypy/compatibility/wiki/Home may have
> some hints about module compatibility, note that your mileage may vary and
> updates are welcome.
>
> Also, you should be using visual studio 2008 (for both PyPy 2.6.0 and
> cPython 2.7), using this link should get you started
> https://www.microsoft.com/en-us/download/details.aspx?id=44266
> Matti
>
>
>
> On 05/07/15 07:44, germano carella wrote:
>
> Hi,
> I downloaded pypy 2.6.0 and I ran pypy -m ensurepip. All worked correctly.
> Now, in some cases, when i do pip install somepackage, it say could not
> find any download that satisfy requirements.
> This even if packages apear in pip search command.
> I tried wxPython, but i'm unable to install it...
> In other cases, pip says that windows ssdk is not found. But I have visual
> studio 2013 installed and windows sdk is on my pc.
> There are variables Can I set?
> Thanks!
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
>
>
> ___________
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
>


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] For Donations, List How Much is Left (or Spent)?

[Amaury Forgeot d7;Arc]

2015-06-25 9:11 GMT+02:00 Armin Rigo :

> Hi Amaury,
>
> On 25 June 2015 at 01:10, Amaury Forgeot d'Arc  wrote:
> > Currently I'm kind of blocked on finishing the "more-rposix" branch
> > (needed for the recent expansion of the posix module:
> os.supports_dir_fd...)
> > It has still some failures: win32 does not translate, for example.
> > Since it's mostly changes in rpython/ I wanted to make it land in the
> > "default" branch first,
> > but I'll probably move along and merge it into the py3.3 branch.
>
> If it's only adding new functions, then I think it's fine to merge it
> to default even if some of these functions cause translation errors on
> Windows.  The default pypy won't call them anyway.
>

No, this branch is actually a full rewrite of the ll_os.py file,
all the code has been moved to rposix.py.

The code is much simpler (no lazy registering callback function),
and we have a consistent way to add new wrapper to C functions,
even if they don't map to any 2.7 feature.


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] For Donations, List How Much is Left (or Spent)?

[Amaury Forgeot d7;Arc]

2015-06-24 23:20 GMT+02:00 Phyo Arkar :

> What do you guys think about skipping directly to pythin 3.5 and release
> it in-time when it releases?
>

Yes, it would be great to release 3.5 at the same time as CPython,
but skipping releases won't make it faster.

There are so many features to implement, many new modules.
At least intermediate releases (3.3 and 3.4) make stable consistent
milestones.

Currently I'm kind of blocked on finishing the "more-rposix" branch
(needed for the recent expansion of the posix module: os.supports_dir_fd...)
It has still some failures: win32 does not translate, for example.
Since it's mostly changes in rpython/ I wanted to make it land in the
"default" branch first,
but I'll probably move along and merge it into the py3.3 branch.



>
> On Fri, Jun 5, 2015 at 2:05 PM, Armin Rigo  wrote:
>
>> Hi Donald, hi everybody,
>>
>> On 1 June 2015 at 22:00, Donald Stufft  wrote:
>> > So out of someone complaining about PyPy's lack of an up to date port of
>> > Python 3, it struck me that for someone just now looking into PyPy it
>> looks
>> > like PyPy is sitting on almost 60k USD in order to port PyPy3 from 3.2
>> to 3.4.
>> > However the reality (via fijall) is that most of it, roughly 50k has
>> already
>> > been spent going from nothing to 3.2.
>>
>> Thanks for bringing this up.  I added to pypy.org the display of the
>> amount left in our account for each project.
>>
>> Like the amount received, this number is updated from a source that
>> depends on a manual step somewhere.  This manual step is typically
>> done only a few times per month.  Don't be surprised if you donate and
>> it does not show up immediately!
>>
>> Current situation:
>>
>> numpy: $52184 received, $15999 left
>> py3k: $59578 received, $5045 left
>> stm (2nd call): $29112 received, $22016 left
>>
>> We also have two Summer of Code projects, one related to numpy
>> (performance via vectorization, not general compatibility) and one on
>> py3k (planning a first 3.3 version by midterm, but knowing that 3.4 is
>> the version most used, trying to push forward).  STM is progressing
>> too, but I am alone on consuming the funds and  recently I've been
>> "side-tracked" by CFFI 1.0 (coming back to STM now).
>>
>>
>> A bientôt,
>>
>> Armin.
>> _______
>> pypy-dev mailing list
>> pypy-dev@python.org
>> https://mail.python.org/mailman/listinfo/pypy-dev
>>
>
>
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
>


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] tkinter tests broken

[Amaury Forgeot d7;Arc]

Thanks Armin,

2015-06-23 9:48 GMT+02:00 Armin Rigo :

> Hi Amaury,
>
> On 21 June 2015 at 14:11, Amaury Forgeot d'Arc  wrote:
> > 2015-06-21 12:33 GMT+02:00 Armin Rigo :
> >> Can you fix the situation?  Thanks!
> >
> > Should be fixed with 6453697f3a30.
>
> Thanks for fixing half of the failures :-)  However, the other half
> still remains:
> http://buildbot.pypy.org/summary/longrepr?testname=unmodified&builder=pypy-c-jit-linux-x86-32&build=2730&mod=lib-python.2.7.test.test_tcl


This is now fixed (with 9fa66d2aa78b):
http://buildbot.pypy.org/summary?category=linux32&branch=%3Ctrunk%3E&recentrev=78278:9fa66d2aa78b


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] rogue CFFI speedups #....@k....$...#

[Amaury Forgeot d7;Arc]

2015-06-19 17:12 GMT+02:00 anatoly techtonik :

> Hi,
>
> Just wanted to let you know of awesome 3x
> speedup after python-tdl was ported to CFFI.
>

Thanks for sharing!
Yes, CFFI is awsesome.


>
> python-tdl is a libtcod binding for Python.
>
> The wrapped libtcod is the very rad “Doryen Library"
> used to build roguelikes in C world. I am sure that
> having it in Python will pave a road to more pleasant
> text mode interfaces. =)
>
> There were some issues during the porting and
> packaging process. I think you might be interested
> to check them out:
>
> https://github.com/HexDecimal/python-tdl/issues/5
>
> --
> anatoly t.
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>



-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] tkinter tests broken

[Amaury Forgeot d7;Arc]

Hi Armin,

2015-06-21 12:33 GMT+02:00 Armin Rigo :

> Hi Amaury,
>
> In fd331e4bf733 you did an untested change to the tkinter library; as
> it turns out, the CPython tests find problems:
>
>
> http://buildbot.pypy.org/summary/longrepr?testname=unmodified&builder=pypy-c-jit-linux-x86-32&build=2727&mod=lib-python.2.7.test.test_tcl
>
> Can you fix the situation?  Thanks!


Actually my changes were trying to fix the test failures in this test_tcl
file, since the merge of stdlib-2.7.10.
This Tcl_NewWideIntObj function is used probably only in the 32bit case.

Should be fixed with 6453697f3a30.




-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] pypy 2.6.0 failing with compile errors

[Amaury Forgeot d7;Arc]

2015-06-14 14:54 GMT+02:00 Armin Rigo :

> Hi Roshan,
>
> On 13 June 2015 at 21:22, Roshan Cherian  wrote:
> > Our build system uses gcc 4.4.6 as a result I am kind of confined to it.
> I
> > am copying the part which has lead to the error. Please let me know how I
> > can provide more info.
>
> Maybe you have an old version of openssl installed, too?  One which
> would not define the type 'EC_KEY'.  Try to grep for 'EC_KEY' in
> '/usr/include/openssl/*.h'.
>

Or this installation of openssl is built with "OPENSSL_NO_ECDH".
CPython respects this flag, PyPy does not.

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Working version of pypy3 that support Python 3.3

[Amaury Forgeot d7;Arc]

Sorry, forget what I said.
There is a big issue with the archive file, most stdlib files are missing.

2015-06-13 10:50 GMT+02:00 Amaury Forgeot d'Arc :

> 2015-06-13 10:06 GMT+02:00 Ram Rachum :
>
>> Hello Armin,
>>
>> I already said I tried it 3 days ago and showed you the error I got, I'm
>> not sure why you're assuming I haven't followed the instructions correctly.
>> Is it so hard to believe that PyPy has a bug or the instructions need
>> tweaking? (BTW your instructions weren't precise, the file to run is pypy3.)
>>
>> Here's my shell session trying your instructions:
>>
>> root@ubuntu:/home/ubuntu/Desktop# curl -O
>> http://buildbot.pypy.org/nightly/py3.3/pypy-c-jit-76553-deae634f291c-linux64.tar.bz2
>>   % Total% Received % Xferd  Average Speed   TimeTime Time
>>  Current
>>  Dload  Upload   Total   SpentLeft
>>  Speed
>> 100 19.7M  100 19.7M0 0   458k  0  0:00:44  0:00:44 --:--:--
>>  674k
>> root@ubuntu:/home/ubuntu/Desktop# tar jxf
>> pypy-c-jit-76553-deae634f291c-linux64.tar.bz2
>> root@ubuntu:/home/ubuntu/Desktop# cd
>> pypy-c-jit-76553-deae634f291c-linux64/bin/
>> root@ubuntu:/home/ubuntu/Desktop/pypy-c-jit-76553-deae634f291c-linux64/bin#
>> ./pypy
>> bash: ./pypy: No such file or directory
>> root@ubuntu:/home/ubuntu/Desktop/pypy-c-jit-76553-deae634f291c-linux64/bin#
>> ./pypy3
>> debug: OperationError:
>> debug:  operror-type: ImportError
>> debug:  operror-value: No module named 'encodings'
>> debug: OperationError:
>> debug:  operror-type: AttributeError
>> debug:  operror-value: stdout
>> root@ubuntu
>> :/home/ubuntu/Desktop/pypy-c-jit-76553-deae634f291c-linux64/bin#
>>
>>
> Can you try with
> LANG=C ./pypy3
>
> And if it does, what's the value of LANG in your environment?
>
>
>
>>
>>
>>
>> On Sat, Jun 13, 2015 at 10:01 AM, Armin Rigo  wrote:
>>
>>> Hi Ram,
>>>
>>> On 11 June 2015 at 16:28, Ram Rachum  wrote:
>>> > Didn't work.
>>>
>>> All we can do is repeat the instructions.  Try following them exactly,
>>> at first:
>>>
>>> - assuming you have downloaded pypy-c-jit-X.tar.bz2, for some value of X
>>>
>>> - run: tar jxf pypy-c-jit-X.tar.bz2
>>>
>>> - cd pypy-c-jit-X/bin
>>>
>>> - ./pypy
>>>
>>>
>>> A bientôt,
>>>
>>> Armin.
>>>
>>
>>
>> ___
>> pypy-dev mailing list
>> pypy-dev@python.org
>> https://mail.python.org/mailman/listinfo/pypy-dev
>>
>>
>
>
> --
> Amaury Forgeot d'Arc
>



-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Working version of pypy3 that support Python 3.3

[Amaury Forgeot d7;Arc]

2015-06-13 10:06 GMT+02:00 Ram Rachum :

> Hello Armin,
>
> I already said I tried it 3 days ago and showed you the error I got, I'm
> not sure why you're assuming I haven't followed the instructions correctly.
> Is it so hard to believe that PyPy has a bug or the instructions need
> tweaking? (BTW your instructions weren't precise, the file to run is pypy3.)
>
> Here's my shell session trying your instructions:
>
> root@ubuntu:/home/ubuntu/Desktop# curl -O
> http://buildbot.pypy.org/nightly/py3.3/pypy-c-jit-76553-deae634f291c-linux64.tar.bz2
>   % Total% Received % Xferd  Average Speed   TimeTime Time
>  Current
>  Dload  Upload   Total   SpentLeft
>  Speed
> 100 19.7M  100 19.7M0 0   458k  0  0:00:44  0:00:44 --:--:--
>  674k
> root@ubuntu:/home/ubuntu/Desktop# tar jxf
> pypy-c-jit-76553-deae634f291c-linux64.tar.bz2
> root@ubuntu:/home/ubuntu/Desktop# cd
> pypy-c-jit-76553-deae634f291c-linux64/bin/
> root@ubuntu:/home/ubuntu/Desktop/pypy-c-jit-76553-deae634f291c-linux64/bin#
> ./pypy
> bash: ./pypy: No such file or directory
> root@ubuntu:/home/ubuntu/Desktop/pypy-c-jit-76553-deae634f291c-linux64/bin#
> ./pypy3
> debug: OperationError:
> debug:  operror-type: ImportError
> debug:  operror-value: No module named 'encodings'
> debug: OperationError:
> debug:  operror-type: AttributeError
> debug:  operror-value: stdout
> root@ubuntu
> :/home/ubuntu/Desktop/pypy-c-jit-76553-deae634f291c-linux64/bin#
>
>
Can you try with
LANG=C ./pypy3

And if it does, what's the value of LANG in your environment?



>
>
>
> On Sat, Jun 13, 2015 at 10:01 AM, Armin Rigo  wrote:
>
>> Hi Ram,
>>
>> On 11 June 2015 at 16:28, Ram Rachum  wrote:
>> > Didn't work.
>>
>> All we can do is repeat the instructions.  Try following them exactly, at
>> first:
>>
>> - assuming you have downloaded pypy-c-jit-X.tar.bz2, for some value of X
>>
>> - run: tar jxf pypy-c-jit-X.tar.bz2
>>
>> - cd pypy-c-jit-X/bin
>>
>> - ./pypy
>>
>>
>> A bientôt,
>>
>> Armin.
>>
>
>
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
>


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Freezing the ".pypy-26.so" name?

[Amaury Forgeot d7;Arc]

2015-06-13 9:31 GMT+02:00 Donald Stufft :

>
>
> On June 13, 2015 at 3:23:21 AM, Armin Rigo (ar...@tunes.org) wrote:
> > Hi Donald,
> >
> > On 13 June 2015 at 09:13, Donald Stufft wrote:
> > > For the uses of pip, it would be preferable if there was an exported
> stable ABI
> > > that we could leverage to tag wheels with. For the people *making*
> Wheels, the
> > > less often that stable ABI changed the better, since a single wheel
> would be
> > > useful for more versions of PyPy then.
> >
> > Sorry, I meant that nothing changed in the ABI, in addition to the
> > API. You can just rename an existing "foo.pypy-25.so" to
> > "foo.pypy-26.so" and it should work.
> >
> > Note that you have to assume that I don't know the problems related to
> > making wheels, and give some more details in your answer. I'm ready
> > to hear about any suggestion if fixing "imp.get_suffixes()" is either
> > not a good idea, or ok but the same information needs to be exported
> > some other way too. The point is that using sys.version_info[:2] is
> > too vague, given that it is always (2, 7), and using
> > sys.pypy_version_info[:2] is (or will become) too precise.
> >
> >
> > A bientôt,
> >
> > Armin.
> >
>
> Sorry, to be clear, I wasn't saying your idea was bad. I was just making it
> clear that whatever it is should reflect the actual ABI of the module. So
> if
> there are compile options (does JIT vs not change it?) then that should be
> reflected in it too. Ideally it'd contain enough information to know if
> something compiled for a particular PyPy is ABI compatible with another
> one.
>

But cffi extension modules are not real modules, right?
When compiled for PyPy, the init function has a different name, and fills a
C struct that cannot be considered as a Python module.

They can be imported only because there is special support in the import
framework.

In this case, why not have a completely different extension or SOABI?
Like _sqlite.cffi1.0.so



>
> As far as how it's exported, the suffixes is a good starting place, and if
> the
> ABI tag can also be added to sysconfig.get_config_var("SOABI") that would
> be
> good too.
>
> For example, CPython (in the 3.x branch) has::
>
> >>> import sysconfig
> >>> sysconfig.get_config_var("SOABI")
> 'cpython-34m'
>
> This is a CPython 3.4 compiled with --with-pymalloc. The CPython bits are
> here: https://www.python.org/dev/peps/pep-3149/
>
> Generally though, I'm +1 on starting to define an actual ABI (and not
> breaking
> it without updating the tag to reflect that). As far as wheels goes, the
> important thing is that we'll make the SOABI part of the wheel filename
> and use
> it to select which wheel we'll download and install. You don't need to
> worry
> about that much other than the fact that defining one makes it better and
> being conservative in changing it makes it better.
>
> ---
> Donald Stufft
> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>
>
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>



-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Working version of pypy3 that support Python 3.3

[Amaury Forgeot d7;Arc]

2015-06-09 20:01 GMT+02:00 Ram Rachum :

> Ah, so I guess I have no choice but to download the linux version. I never
> downloaded Linux software before (only installed with package managers).
> How do you do that? Is it just a folder you download and run or is there
> some extra process involved?
>

Just unpack the archive in any directory, and run pypy from there.
http://pypy.org/download.html#installing

I usually use a command like:
  tar xvaf /path/to/pypy-c.tar.bz2 .



>
> On Tue, Jun 9, 2015 at 8:15 PM, Amaury Forgeot d'Arc 
> wrote:
>
>> 2015-06-09 19:11 GMT+02:00 Amaury Forgeot d'Arc :
>>
>>> 2015-06-09 19:03 GMT+02:00 Ram Rachum :
>>>
>>>> Ah... Can I get that in Windows please?
>>>>
>>>
>>> I just started a translation.
>>> Please be patient, and it's very possible that it breaks in the middle...
>>>
>>
>> Actually it broke at the beginning :-/
>>
>>
>> http://buildbot.pypy.org/builders/pypy-c-jit-win-x86-32/builds/1921/steps/translate/logs/stdio
>> [translation:info]File
>> "c:\pypy\buildbot\pypy-c-jit-win-x86-32\build\rpython\rlib\rposix.py", line
>> 273, in replace
>> [translation:info] 'On windows, os.replace() should overwrite the
>> destination')
>> [translation:ERROR] NotImplementedError: On windows, os.replace() should
>> overwrite the destination
>>
>>
>>
>>>
>>>
>>>>
>>>> On Tue, Jun 9, 2015 at 8:02 PM, Amaury Forgeot d'Arc <
>>>> amaur...@gmail.com> wrote:
>>>>
>>>>> 2015-06-09 18:58 GMT+02:00 Ram Rachum :
>>>>>
>>>>>> Hi everyone,
>>>>>>
>>>>>> I want to test my PyPI package on pypy3, but my package only support
>>>>>> Python 3.3 and above. I know that a version of pypy3 that support Python
>>>>>> 3.3 is not ready yet, but is there a development version of it that I can
>>>>>> run so I could test my package and find any bugs in it? (And possibly 
>>>>>> also
>>>>>> bugs in pypy.)
>>>>>>
>>>>>
>>>>> Sure, we have builds for the linux64 platform:
>>>>> http://buildbot.pypy.org/nightly/py3.3/
>>>>>
>>>>>
>>>>> --
>>>>> Amaury Forgeot d'Arc
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Amaury Forgeot d'Arc
>>>
>>
>>
>>
>> --
>> Amaury Forgeot d'Arc
>>
>
>


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Working version of pypy3 that support Python 3.3

[Amaury Forgeot d7;Arc]

2015-06-09 19:11 GMT+02:00 Amaury Forgeot d'Arc :

> 2015-06-09 19:03 GMT+02:00 Ram Rachum :
>
>> Ah... Can I get that in Windows please?
>>
>
> I just started a translation.
> Please be patient, and it's very possible that it breaks in the middle...
>

Actually it broke at the beginning :-/

http://buildbot.pypy.org/builders/pypy-c-jit-win-x86-32/builds/1921/steps/translate/logs/stdio
[translation:info]File
"c:\pypy\buildbot\pypy-c-jit-win-x86-32\build\rpython\rlib\rposix.py", line
273, in replace
[translation:info] 'On windows, os.replace() should overwrite the
destination')
[translation:ERROR] NotImplementedError: On windows, os.replace() should
overwrite the destination



>
>
>>
>> On Tue, Jun 9, 2015 at 8:02 PM, Amaury Forgeot d'Arc 
>> wrote:
>>
>>> 2015-06-09 18:58 GMT+02:00 Ram Rachum :
>>>
>>>> Hi everyone,
>>>>
>>>> I want to test my PyPI package on pypy3, but my package only support
>>>> Python 3.3 and above. I know that a version of pypy3 that support Python
>>>> 3.3 is not ready yet, but is there a development version of it that I can
>>>> run so I could test my package and find any bugs in it? (And possibly also
>>>> bugs in pypy.)
>>>>
>>>
>>> Sure, we have builds for the linux64 platform:
>>> http://buildbot.pypy.org/nightly/py3.3/
>>>
>>>
>>> --
>>> Amaury Forgeot d'Arc
>>>
>>
>>
>
>
> --
> Amaury Forgeot d'Arc
>



-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Working version of pypy3 that support Python 3.3

[Amaury Forgeot d7;Arc]

2015-06-09 19:03 GMT+02:00 Ram Rachum :

> Ah... Can I get that in Windows please?
>

I just started a translation.
Please be patient, and it's very possible that it breaks in the middle...


>
> On Tue, Jun 9, 2015 at 8:02 PM, Amaury Forgeot d'Arc 
> wrote:
>
>> 2015-06-09 18:58 GMT+02:00 Ram Rachum :
>>
>>> Hi everyone,
>>>
>>> I want to test my PyPI package on pypy3, but my package only support
>>> Python 3.3 and above. I know that a version of pypy3 that support Python
>>> 3.3 is not ready yet, but is there a development version of it that I can
>>> run so I could test my package and find any bugs in it? (And possibly also
>>> bugs in pypy.)
>>>
>>
>> Sure, we have builds for the linux64 platform:
>> http://buildbot.pypy.org/nightly/py3.3/
>>
>>
>> --
>> Amaury Forgeot d'Arc
>>
>
>


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Working version of pypy3 that support Python 3.3

[Amaury Forgeot d7;Arc]

2015-06-09 18:58 GMT+02:00 Ram Rachum :

> Hi everyone,
>
> I want to test my PyPI package on pypy3, but my package only support
> Python 3.3 and above. I know that a version of pypy3 that support Python
> 3.3 is not ready yet, but is there a development version of it that I can
> run so I could test my package and find any bugs in it? (And possibly also
> bugs in pypy.)
>

Sure, we have builds for the linux64 platform:
http://buildbot.pypy.org/nightly/py3.3/


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] How to use interperter level code from a module for app level code in another module

[Amaury Forgeot d7;Arc]

2015-06-09 11:51 GMT+02:00 Yicong Huang :

> "I guess it's because __pypy__ is always enabled, but other modules can
> be enabled or disabled"
>
> How to enable the module?
> We had tried "spaceconfig", but it only worked for writing tests.
>

Again, how are you running PyPy?
If it is the compiled binary, you need to pass --withmod-select at
translation time.
Same thing, if you started the (slow) untranslated bin/pyinteractive.py


>
> On Tue, Jun 9, 2015 at 5:38 PM, Armin Rigo  wrote:
>
>> Hi,
>>
>> On 9 June 2015 at 11:34, Yicong Huang  wrote:
>> > import select =>  ImportError: No module named select
>> > import pypy.module.select =>ImportError: No module named pypy
>> >
>> > But __pypy__ module is able to be imported.
>>
>> I guess it's because __pypy__ is always enabled, but other modules can
>> be enabled or disabled (for example using "spaceconfig" if you're
>> writing tests in the same style as ours).
>>
>>
>> A bientôt,
>>
>> Armin.
>>
>
>


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] How to use interperter level code from a module for app level code in another module

[Amaury Forgeot d7;Arc]

2015-06-09 11:34 GMT+02:00 Yicong Huang :

> No, I' am trying to call interp code from app code, and found the module
> could not be imported:
>
> import select =>  ImportError: No module named select
> import pypy.module.select =>ImportError: No module named pypy
>

"import select" is the one which should work at applevel.
But for this, the PyPy object space must have the option "spaceconfig =
dict(usemodules=['select'])", or be translated with --withmod-select.
In which context are you running the code above?


>
> But __pypy__ module is able to be imported.
>

some modules are always selected: sys, posix, __pypy__... that's why you
see a difference here.
Put usemodules=['select'], somewhere.



>
> On Tue, Jun 9, 2015 at 5:27 PM, Amaury Forgeot d'Arc 
> wrote:
>
>> 2015-06-09 11:15 GMT+02:00 Yicong Huang :
>>
>>> Great thanks!
>>> "posix" module is a good example. But this interp code reference is in
>>> the same module.
>>>
>>> How about a new module's app code to use interp code from other module?
>>> One method to follow "posix" example is to define interp code in the
>>> same module and wrap the interp function from other module for app code in
>>> the same module to use.
>>>
>>
>> This is not clear to me.
>> Are you trying to call interp code from interp code? Then just import the
>> interp module (pypy.module.xxx.yyy)
>>
>>
>>>
>>> But I observed __pypy__ module is different. Interp functions of _pypy_
>>> module could be directly import in app code for other modules.
>>> We are curious how does it work?
>>>
>>>
>>>
>>> On Tue, Jun 9, 2015 at 4:25 PM, Amaury Forgeot d'Arc >> > wrote:
>>>
>>>> 2015-06-09 10:20 GMT+02:00 Yicong Huang :
>>>>
>>>>> "The interpreter gateway has more options for exposing interp-level
>>>>> functions and types to app-level​."
>>>>>
>>>>> How does this work?
>>>>> We tried several builtin modules, as well as "select" module that you
>>>>> mentioned, all thses modules's interp-level code are invisble to other
>>>>> module's app-level.
>>>>>
>>>>> We would like to take your advices to make a new module. But the
>>>>> problem is how to make the new module's interp-level code visible to
>>>>> app-level?
>>>>>
>>>>
>>>> See the "posix" module for example. Most functions are defined at
>>>> interp-level,
>>>> but there is also a file "app_posix.py" which just does "import posix"
>>>> and happily calls interp-level code.
>>>>
>>>>
>>>>
>>>>>
>>>>>
>>>>> On Tue, Jun 9, 2015 at 9:30 AM, William ML Leslie <
>>>>> william.leslie@gmail.com> wrote:
>>>>>
>>>>>> On 8 June 2015 at 21:22, Yicong Huang  wrote:
>>>>>>
>>>>>>> By reading modules in pypy/module, finally we found out the
>>>>>>> workaround method.
>>>>>>>
>>>>>>> For most of modules in 'pypy/module', their interp code could not be
>>>>>>> imported or used by other module's app code.
>>>>>>> But there is special module '__pypy__', it exposed several interp
>>>>>>> functions, and those code are used by other module's app code.
>>>>>>>
>>>>>>
>>>>>> ​There's nothing special about __pypy__ for exposing interp-level
>>>>>> code, the modules in pypy/modules​ usually include interp level code.
>>>>>>
>>>>>> ​For example, the select module exposes the interp-level function
>>>>>> select_interp.select.  Here's how it does so:
>>>>>>
>>>>>> ​
>>>>>> https://bitbucket.org/pypy/pypy/src/b0c01840baea472c834635de627bb596c05d3bd5/pypy/module/select/__init__.py?at=default
>>>>>>
>>>>>> The interpreter gateway has more options for exposing interp-level
>>>>>> functions and types to app-level​.
>>>>>>
>>>>>> The workaround solution is simple: add one more interface in __pypy__
>>>>>>> module's __init__.py

Re: [pypy-dev] How to use interperter level code from a module for app level code in another module

[Amaury Forgeot d7;Arc]

2015-06-09 11:15 GMT+02:00 Yicong Huang :

> Great thanks!
> "posix" module is a good example. But this interp code reference is in the
> same module.
>
> How about a new module's app code to use interp code from other module?
> One method to follow "posix" example is to define interp code in the same
> module and wrap the interp function from other module for app code in the
> same module to use.
>

This is not clear to me.
Are you trying to call interp code from interp code? Then just import the
interp module (pypy.module.xxx.yyy)


>
> But I observed __pypy__ module is different. Interp functions of _pypy_
> module could be directly import in app code for other modules.
> We are curious how does it work?
>
>
>
> On Tue, Jun 9, 2015 at 4:25 PM, Amaury Forgeot d'Arc 
> wrote:
>
>> 2015-06-09 10:20 GMT+02:00 Yicong Huang :
>>
>>> "The interpreter gateway has more options for exposing interp-level
>>> functions and types to app-level​."
>>>
>>> How does this work?
>>> We tried several builtin modules, as well as "select" module that you
>>> mentioned, all thses modules's interp-level code are invisble to other
>>> module's app-level.
>>>
>>> We would like to take your advices to make a new module. But the problem
>>> is how to make the new module's interp-level code visible to app-level?
>>>
>>
>> See the "posix" module for example. Most functions are defined at
>> interp-level,
>> but there is also a file "app_posix.py" which just does "import posix"
>> and happily calls interp-level code.
>>
>>
>>
>>>
>>>
>>> On Tue, Jun 9, 2015 at 9:30 AM, William ML Leslie <
>>> william.leslie@gmail.com> wrote:
>>>
>>>> On 8 June 2015 at 21:22, Yicong Huang  wrote:
>>>>
>>>>> By reading modules in pypy/module, finally we found out the workaround
>>>>> method.
>>>>>
>>>>> For most of modules in 'pypy/module', their interp code could not be
>>>>> imported or used by other module's app code.
>>>>> But there is special module '__pypy__', it exposed several interp
>>>>> functions, and those code are used by other module's app code.
>>>>>
>>>>
>>>> ​There's nothing special about __pypy__ for exposing interp-level code,
>>>> the modules in pypy/modules​ usually include interp level code.
>>>>
>>>> ​For example, the select module exposes the interp-level function
>>>> select_interp.select.  Here's how it does so:
>>>>
>>>> ​
>>>> https://bitbucket.org/pypy/pypy/src/b0c01840baea472c834635de627bb596c05d3bd5/pypy/module/select/__init__.py?at=default
>>>>
>>>> The interpreter gateway has more options for exposing interp-level
>>>> functions and types to app-level​.
>>>>
>>>> The workaround solution is simple: add one more interface in __pypy__
>>>>> module's __init__.py
>>>>>
>>>>
>>>> ​You should make a new module rather than adding to one existing one.​
>>>>
>>>>
>>>>>  PYC_MAGIC = get_pyc_magic(self.space)
>>>>
>>>>  self.extra_interpdef('PYC_MAGIC', 'space.wrap(%d)' % PYC_MAGIC)
>>>>
>>>>
>>>>> Are the above code do the magic thing to expose interp code?
>>>>>
>>>>
>>>> ​They are exposing an integer value called PYC_MAGIC, which is the
>>>> value for determining what python the .pyc file was compiled for.​
>>>>
>>>> ​This is about exposing an integer rather than a function, it's
>>>> probably not relevant to you.
>>>>
>>>>
>>>> --
>>>> William Leslie
>>>>
>>>> Notice:
>>>> Likely much of this email is, by the nature of copyright, covered under
>>>> copyright law.  You absolutely MAY reproduce any part of it in accordance
>>>> with the copyright law of the nation you are reading this in.  Any attempt
>>>> to DENY YOU THOSE RIGHTS would be illegal without prior contractual
>>>> agreement.
>>>>
>>>
>>>
>>> ___
>>> pypy-dev mailing list
>>> pypy-dev@python.org
>>> https://mail.python.org/mailman/listinfo/pypy-dev
>>>
>>>
>>
>>
>> --
>> Amaury Forgeot d'Arc
>>
>
>


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] How to use interperter level code from a module for app level code in another module

[Amaury Forgeot d7;Arc]

2015-06-09 10:20 GMT+02:00 Yicong Huang :

> "The interpreter gateway has more options for exposing interp-level
> functions and types to app-level​."
>
> How does this work?
> We tried several builtin modules, as well as "select" module that you
> mentioned, all thses modules's interp-level code are invisble to other
> module's app-level.
>
> We would like to take your advices to make a new module. But the problem
> is how to make the new module's interp-level code visible to app-level?
>

See the "posix" module for example. Most functions are defined at
interp-level,
but there is also a file "app_posix.py" which just does "import posix" and
happily calls interp-level code.



>
>
> On Tue, Jun 9, 2015 at 9:30 AM, William ML Leslie <
> william.leslie@gmail.com> wrote:
>
>> On 8 June 2015 at 21:22, Yicong Huang  wrote:
>>
>>> By reading modules in pypy/module, finally we found out the workaround
>>> method.
>>>
>>> For most of modules in 'pypy/module', their interp code could not be
>>> imported or used by other module's app code.
>>> But there is special module '__pypy__', it exposed several interp
>>> functions, and those code are used by other module's app code.
>>>
>>
>> ​There's nothing special about __pypy__ for exposing interp-level code,
>> the modules in pypy/modules​ usually include interp level code.
>>
>> ​For example, the select module exposes the interp-level function
>> select_interp.select.  Here's how it does so:
>>
>> ​
>> https://bitbucket.org/pypy/pypy/src/b0c01840baea472c834635de627bb596c05d3bd5/pypy/module/select/__init__.py?at=default
>>
>> The interpreter gateway has more options for exposing interp-level
>> functions and types to app-level​.
>>
>> The workaround solution is simple: add one more interface in __pypy__
>>> module's __init__.py
>>>
>>
>> ​You should make a new module rather than adding to one existing one.​
>>
>>
>>>  PYC_MAGIC = get_pyc_magic(self.space)
>>
>>  self.extra_interpdef('PYC_MAGIC', 'space.wrap(%d)' % PYC_MAGIC)
>>
>>
>>> Are the above code do the magic thing to expose interp code?
>>>
>>
>> ​They are exposing an integer value called PYC_MAGIC, which is the value
>> for determining what python the .pyc file was compiled for.​
>>
>> ​This is about exposing an integer rather than a function, it's probably
>> not relevant to you.
>>
>>
>> --
>> William Leslie
>>
>> Notice:
>> Likely much of this email is, by the nature of copyright, covered under
>> copyright law.  You absolutely MAY reproduce any part of it in accordance
>> with the copyright law of the nation you are reading this in.  Any attempt
>> to DENY YOU THOSE RIGHTS would be illegal without prior contractual
>> agreement.
>>
>
>
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
>


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Is there similar PyEval_EvalFrameEx() function in PyPy?

[Amaury Forgeot d7;Arc]

Did you consider the trace module?

https://docs.python.org/2/library/trace.html

2015-05-26 11:45 GMT+02:00 Yicong Huang :

> We would like to monitor and log all external function calls from a class:
> when there the function call happen, we capture and log the frame.
>
> In python, there is the function PyEval_EvalFrameEx() we might add a
> callback inside.
> Is there a similar function in PyPy that would be called when the function
> call happens?
>
>
>
>
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
>


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Cannot instantiate ctype 'struct *' of unknown size

[Amaury Forgeot d7;Arc]

Haha, but there is a bug in cffi I think.

I just filed
https://bitbucket.org/cffi/cffi/issue/193/forward-declaration-of-struct-does-not
with a simpler test case.

2015-05-20 12:03 GMT+02:00 Yicong Huang :

> Oh, great thanks for you help!
> Finally, I found out the real issue.
> The whold code is this:
>
> @ffi.callback("struct api_ret*(char *)")
> def udf_api(arg0):
>return None
>
> ffi.cdef('''
>   struct API {
>  struct api_ret* (*pyudf)(char* str);
>   };
> ''')
>
> ffi.cdef('''
>   struct api_ret {
>  char *ret1;
>  char *ret2;
>  char *ret3;
>   };
> ''')
>
> result = ffi.new('struct api_ret*')
>
> And we should put the function callback annotion below struct api_ret
> definition. :)
>
> On Wed, May 20, 2015 at 5:51 PM, Amaury Forgeot d'Arc 
> wrote:
>
>> 2015-05-20 11:44 GMT+02:00 Yicong Huang :
>>
>>> Hi,
>>>
>>> The below is the code we met troubles;
>>>
>>> ffi.cdef('''
>>>   struct API {
>>>  struct api_ret* (*pyudf)(char* str);
>>>   };
>>> ''')
>>>
>>> ffi.cdef('''
>>>   struct api_ret {
>>>  char *ret1;
>>>  char *ret2;
>>>  char *ret3;
>>>   };
>>> ''')
>>>
>>> result = ffi.new('struct api_ret*')
>>>
>>
>> This code works for me. What's the issue exactly?
>>
>>
>>>
>>> At first, we defined struct API with a function pointer that return a
>>> pointer to struct api_ret.
>>> And then we defined api_ret, and tried to new the object.
>>> The problem seems to happen when the function pointer and new structure
>>> are together.
>>> The below code had no problems.
>>>
>>> ffi.cdef('''
>>>   struct API {
>>>  struct api_ret* (*pyudf)(char* str);
>>>   };
>>> ''')
>>>
>>> ffi.cdef('''
>>>   struct api_ret2 {
>>>  char *ret1;
>>>  char *ret2;
>>>  char *ret3;
>>>   };
>>> ''')
>>> #we used different name api_ret2 to work around the issue.
>>> result = ffi.new('struct api_ret2*')
>>>
>>> ___
>>> pypy-dev mailing list
>>> pypy-dev@python.org
>>> https://mail.python.org/mailman/listinfo/pypy-dev
>>>
>>>
>>
>>
>> --
>> Amaury Forgeot d'Arc
>>
>
>


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Cannot instantiate ctype 'struct *' of unknown size

[Amaury Forgeot d7;Arc]

2015-05-20 11:44 GMT+02:00 Yicong Huang :

> Hi,
>
> The below is the code we met troubles;
>
> ffi.cdef('''
>   struct API {
>  struct api_ret* (*pyudf)(char* str);
>   };
> ''')
>
> ffi.cdef('''
>   struct api_ret {
>  char *ret1;
>  char *ret2;
>  char *ret3;
>   };
> ''')
>
> result = ffi.new('struct api_ret*')
>

This code works for me. What's the issue exactly?


>
> At first, we defined struct API with a function pointer that return a
> pointer to struct api_ret.
> And then we defined api_ret, and tried to new the object.
> The problem seems to happen when the function pointer and new structure
> are together.
> The below code had no problems.
>
> ffi.cdef('''
>   struct API {
>  struct api_ret* (*pyudf)(char* str);
>   };
> ''')
>
> ffi.cdef('''
>   struct api_ret2 {
>  char *ret1;
>  char *ret2;
>  char *ret3;
>   };
> ''')
> #we used different name api_ret2 to work around the issue.
> result = ffi.new('struct api_ret2*')
>
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
>


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] For embedding pypy, how to deal with None value

[Amaury Forgeot d7;Arc]

2015-05-20 9:54 GMT+02:00 Yicong Huang :

> Yes, C could *not* have a function that returns either a double or NULL.
> But python could.
> In addition, python could have a function paramter that is either int or
> None.
>
> The problem is we might get a python function, and we would like to call
> this python function in C.
> We're not sure whether this function return int or return None.
> Follow the document ,we might define the function callback as below:
> @ffi.callback("int (int, int)")
> However, sometimes the python function will return None, which cause
> Operation Error from CFFI.
>
> On the other hand, it is valid to python for receving the function
> parameter as None.
> How could we pass NULL to python function via ffi.callback definition?
>

cffi basic usage is to call C functions from Python, and is was designed
for this. C is a simple language.
Your use case is the opposite: call Python from C, and it's going to be
more complex because
Python types are way more flexible than C.

To call from C a Python function that can return multiple types,
you will have to design (in C) some way to embed these types in a single
value,
and more code to manipulate this value.
For example, the CPython API uses the PyObject* type to represent anything,
and has more than 400 functions like PyFloat_AsDouble.

Yes, this is involved: cffi.new_handle() can be used to return any Python
object to C, but the handle
needs to stay alive on the Python side, so a pool of returned handles is
necessary, but it needs to
be cleared sometimes, and there will be many helper functions...

I once wrote some code to demonstrate this, I need to clean it a bit and
put it in the docs.

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] How to get the return value from pypy function for C code

[Amaury Forgeot d7;Arc]

2015-05-13 9:55 GMT+02:00 Yicong Huang :

> Thanks for your advices!
> But your code seems to only solve the problem about how to pass the
> function parameter from C to Python.
> In the document example:
>
> char source[] = "from cffi import FFI\n\ffi = 
> FFI()\n\@ffi.callback('int(int)')\n\def func(a):\n\print 'Got from C %d' 
> % a\n\return a * 2\n\ffi.cdef('int callback(int (*func)(int));')\n\c_func 
> = ffi.cast('int(*)(int(*)(int))', c_argument)\n\c_func(func)\n\print 
> 'finished the Python part'\n\";
> int callback(int (*func)(int)){
> printf("Calling to Python, result: %d\n", func(3));}
>
> We are able to pass the function paramter '3' to python function func(a).
> But how could we extract the function result '6' to C code?
> In CPython, there is  'PyObject_CallObject', which returns python
> fucntion result.
> Does pypy have similar API?
>

But don't you have it already? the func(3) above should return the integer
6!



>
> On Wed, May 13, 2015 at 3:25 PM, Amaury Forgeot d'Arc 
> wrote:
>
>> Hi,
>>
>> 2015-05-13 7:39 GMT+02:00 Yicong Huang :
>>
>>> From the document " "Embedding Pypy", we got the method to call python
>>> function from C code.
>>> But there are no examples on how to get the function return value.
>>> Does pypy have this feature?
>>>
>>
>>
>> I think you need to use the function "pypy_execute_source_ptr", and pass
>> a pointer to a C variable.
>> From the Python side, it's turned into an int, that you can cast back to
>> a ffi pointer.
>> Something like this:
>>
>> import cffi
>> ffi = cffi.FFI()
>> result_ptr = ffi.cast('double *', c_argument)
>> result_ptr[0] = 42.0
>>
>> --
>> Amaury Forgeot d'Arc
>>
>
>


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] How to call the "same pypy function" from C for thousands of times?

[Amaury Forgeot d7;Arc]

Hi,

2015-05-13 4:42 GMT+02:00 Yicong Huang :

> However, the code is quite slow for reasons:
> 1. Pypy treats the same function call for a new function every time. It
> took time to do parsing, analysing and such a lot of work.
> 2. JIT could not give any benifits for the single function call.
>

As said in the docs, the trick is to call pypy_execute_source once, and
make it return a set of C function pointers.

Here is how I would do it:

- in a file "interface.py"

import cffi

ffi = cffi.FFI()
ffi.cdef('''
struct API {
double (*add_numbers)(double x, double y);
};
''')

# Better define callbacks at module scope, it's important to keep this
object alive.
@ffi.callback("double (double, double)")
def add_numbers(x, y):
return x + y

def fill_api(ptr):
api = ffi.cast("struct API*", ptr)
api.add_numbers = add_numbers


- From C code:

struct API {
double (*add_numbers)(double x, double y);
};

API api;
const char source[] = "import interface;
interface.fill_api(c_argument)";
res = pypy_execute_source_ptr(source, &api);

// Now call this in a loop:
api.add_numbers(12, 13);


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] How to get the return value from pypy function for C code

[Amaury Forgeot d7;Arc]

Hi,

2015-05-13 7:39 GMT+02:00 Yicong Huang :

> From the document " "Embedding Pypy", we got the method to call python
> function from C code.
> But there are no examples on how to get the function return value.
> Does pypy have this feature?
>


I think you need to use the function "pypy_execute_source_ptr", and pass a
pointer to a C variable.
>From the Python side, it's turned into an int, that you can cast back to a
ffi pointer.
Something like this:

import cffi
ffi = cffi.FFI()
result_ptr = ffi.cast('double *', c_argument)
result_ptr[0] = 42.0

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] (no subject)

[Amaury Forgeot d7;Arc]

2015-05-05 1:55 GMT+02:00 Leonardo Santagada :

> As we are transitioning to internationalized messages should/could we also
> get more descritive error messages? It could always be just a phrase and a
> link, eg:
> instead of
> TypeError: x() takes no arguments (1 given)
> we output:
> TypeError: x() takes no arguments (1 given) (maybe you forgot the self
> attribute (https://docs.python.org/2/tutorial/classes.html#random-remarks
> ))
>
> I think for this to move forward someone (me) would need to write a pep
> about it, I just want to know if you think it is a good idea, or there is
> an obvious problem that I'm not seeing.
>

This could be implemented as a custom language ("en-DESCRIPTIVE") so that
you are free to experiment without changing any code...


>
>
> On Mon, May 4, 2015 at 5:26 PM, Yury V. Zaytsev  wrote:
>
>> On Mon, 2015-05-04 at 22:24 +0200, Carl Friedrich Bolz wrote:
>> >
>> > Yes, well. It's precisely *not* about professional programmers in this
>> > issue.
>>
>> That's why, as I said, I'm totally +1 on it ;-)
>>
>> --
>> Sincerely yours,
>> Yury V. Zaytsev
>>
>>
>> ___
>> pypy-dev mailing list
>> pypy-dev@python.org
>> https://mail.python.org/mailman/listinfo/pypy-dev
>>
>
>
>
> --
>
> Leonardo Santagada
>
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
>


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Porting PyPy/rpython to Python 3

[Amaury Forgeot d7;Arc]

2015-04-16 14:02 GMT+02:00 Maciej Fijalkowski :

> > I think *some* conversion should be allowed, for example when the
> unicode is
> > a constant.
> > (maybe with a SomeAsciiString annotation)
> > Otherwise, do we need to rewrite all calls like `space.call_method(w_x,
> > "split")`?
> >
> > Another issue will be the int/long distinction.
>
> Wait, what? You're messing two things. If you want to convert, you can
> always call encode/decode


Should space.call_method() take a bytes string or a unicode string?
And can we have a unique spelling that covers all cases?
python{2|3} translating a pypy{2|3} interpreter

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Porting PyPy/rpython to Python 3

[Amaury Forgeot d7;Arc]

2015-04-16 10:48 GMT+02:00 Maciej Fijalkowski :

> >
> > 2. I am initially doing this work in a way that maintains 2/3
> compatibility
> > - my check before each major commit is whether I can still build pypy
> using
> > pypy2. Would the pypy devs be willing to make building pypy be 2.7+ only?
> > That way I could use __future__ imports to ease some of the porting.
>
> Generally speaking the small changes are mostly a no-brainer for us.
> RPython is already 2.7 only. However, we generally want to avoid being
> Python 3 compatible as a major barrier, so things that complicate
> stuff need to be discussed first. One thing that we need to discuss is
> how to support unicode in RPython. Unicode-everywhere is definitely a
> model we would not like to pursue, you *have to* be able to use bytes
> efficiently and all over the place in RPython. Right now unicode
> support is a bit rudimentary and I would welcome a way to structure it
> better. I'm happy to discuss this (note that automatic conversion
> between unicode and bytes in rpython is illegal anyway)
>

I think *some* conversion should be allowed, for example when the unicode
is a constant.
(maybe with a SomeAsciiString annotation)
Otherwise, do we need to rewrite all calls like `space.call_method(w_x,
"split")`?

Another issue will be the int/long distinction.

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Which pypy with >=3.3 Python compatibility

[Amaury Forgeot d7;Arc]

2015-04-13 17:50 GMT+02:00 Ludovic Gasc :

> From my experience, the main issue I have is that setuptools/pip
> bootstrapping doesn't work, at least on my setup.
> You need to install dependencies manually and play with PYTHONPATH
> environment variable.
>

ensurepip is a 3.4 feature, right?
I guess we need to wait a bit more.

Or get involved and help us finish the 3.3 port first...

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Which pypy with >=3.3 Python compatibility

[Amaury Forgeot d7;Arc]

Hi,

2015-04-13 17:29 GMT+02:00 Mike Müller :

> I need pypy that is Python 3.3 or, even better, Python 3.4 compatible.
> I can found the nightly builds at http://buildbot.pypy.org//nightly


There is a branch for the Python 3.3 port, named "py3.3".
It's not complete, but probably enough for most usages.

The builds are in the corresponding subdirectory:
http://buildbot.pypy.org/nightly/py3.3/
Note that this branch is not built nightly, but only on-demand.

I just started a new one ("pypy-c-jit-linux-x86-64"),
let's see if it completes :-)

As you have guessed, it's still Work In Progress,
and should be considered as highly experimental.



> Which one should I use py3.3 or py3k? There are many more version.
> Should I use one of them?
>
> Thanks,
> Mike
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>



-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Release 2.5.1 binaries now available for download

[Amaury Forgeot d7;Arc]

2015-03-25 9:22 GMT+01:00 Maciej Fijalkowski :

> maybe we should actually write up (or copy-paste) what stdlib 2.7.9
> means (it changes quite a bit)
>

Agreed. Also the stdlib contains the test suite,
so the changes also impact the core interpreter.


>
> On Wed, Mar 25, 2015 at 9:59 AM, Maciej Fijalkowski 
> wrote:
> > No releases on weekends please :-) (It's a PR disaster)
> >
> > On Tue, Mar 24, 2015 at 11:09 PM, Matti Picus 
> wrote:
> >> I have uploaded releases bundles to
> >> https://bitbucket.org/pypy/pypy/downloads
> >> Could people please take a few minutes to test them out on their
> favorite
> >> platform? Also, any corrections, additions, or edits to the release
> notice
> >> would be welcome, it is here
> >> http://doc.pypy.org/en/release-2.5.x/release-2.5.1.html
> >> Note it has a broken link, the whatsnew is here instead
> >> http://doc.pypy.org/en/release-2.5.x/whatsnew-2.5.1.html
> >> This will be fixed when I merge the release branch back into default
> >>
> >> Barring any issues, the release notice should hit the blog by the
> weekend,
> >> and I will update the web sites appropriately before then.
> >> Matti
> >> ___
> >> pypy-dev mailing list
> >> pypy-dev@python.org
> >> https://mail.python.org/mailman/listinfo/pypy-dev
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>



-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] How to help PyPy 3 ?

[Amaury Forgeot d7;Arc]

Hi,

2015-03-22 14:45 GMT+01:00 Ludovic Gasc :

> Hi,
>
> I want to try to help PyPy 3, especially to run AsyncIO on PyPy 3.
>
> For now, I've tried to compile PyPy from 3.3 branch, and install AsyncIO.
> I'd an issue with time.monotonic() and time.get_clock_info() in AsyncIO,
> because it isn't implemented in PyPy 3.
>
> I've replaced that by time.time() and hardcoded value of
> time.get_clock_info() return, only to see until AsyncIO should work on PyPy
> 3.
>
> I've launched several examples from AsyncIO documentation, it works,
> except when I launch a TCP client:
> https://docs.python.org/3/library/asyncio-protocol.html#tcp-echo-client-protocol
> I've the same issue with aiotest: https://bitbucket.org/haypo/aiotest All
> tests pass, except:
> $ ./pypy-c -Wd ../tulip/run_aiotest.py
> Run tests in debug mode
> ...E
> ==
> ERROR: test_tcp_hello (aiotest.test_network.NetworkTests)
> --
> Traceback (most recent call last):
>   File
> "/home/lg/Documents/IDEA/pypy/site-packages/aiotest/test_network.py", line
> 78, in test_tcp_hello
> self.loop.run_until_complete(coro)
>   File
> "/home/lg/Documents/IDEA/pypy/site-packages/asyncio/base_events.py", line
> 317, in run_until_complete
> return future.result()
>   File "/home/lg/Documents/IDEA/pypy/site-packages/asyncio/futures.py",
> line 275, in result
> raise self._exception
>   File "/home/lg/Documents/IDEA/pypy/site-packages/asyncio/tasks.py", line
> 238, in _step
> result = next(coro)
>   File "/home/lg/Documents/IDEA/pypy/site-packages/asyncio/coroutines.py",
> line 79, in __next__
> return next(self.gen)
>   File
> "/home/lg/Documents/IDEA/pypy/site-packages/asyncio/base_events.py", line
> 644, in create_connection
> sock, protocol_factory, ssl, server_hostname)
> TypeError: '_SelectorSocketTransport' object is not iterable
>

Please file a bug, this is an issue with "yield from" and return values.
The script below prints "(1, 2)" with CPython3, but "1" with pypy3.3

def f():
yield 3
return 1, 2

def g():
x = yield from f()
print(x)

list(g())



>
> --
> Ran 16 tests in 0.337s
>
> FAILED (errors=1)
>
> Finally, I've tested to launch the aiohttp server example:
> https://github.com/KeepSafe/aiohttp/blob/master/examples/srv.py
> The servers starts, but when I launch a HTTP request, I've a strange
> stackstrace:
> Error handling request
> Traceback (most recent call last):
>   File "/home/lg/Documents/IDEA/pypy/site-packages/aiohttp/server.py",
> line 240, in start
> yield from handler
>   File "/home/lg/tmp/asyncio_examples/7_aiohttp_server.py", line 25, in
> handle_request
> message.method, message.path, message.version))
> AttributeError: 'str' object has no attribute 'method'
> Traceback (most recent call last):
>   File "/home/lg/Documents/IDEA/pypy/site-packages/aiohttp/server.py",
> line 240, in start
> yield from handler
>   File "/home/lg/tmp/asyncio_examples/7_aiohttp_server.py", line 25, in
> handle_request
> message.method, message.path, message.version))
> AttributeError: 'str' object has no attribute 'method'
>
> After debugging a little bit with pdb, I see
> that aiohttp.HttpRequestParser parses correctly the request, but "message =
> yield from httpstream.read()" in aiohttp.server at line 226 returns "GET"
> string instead of a RawRequestMessage object.
>
> I'm a total newbie about PyPy internals, implement a monotonic timer seems
> to be over-complicated to me.
> But, I should maybe help with tests and fix small issues like with
> aiohttp.server if I've a clue to fix that.
> Are you interested if I create issues on PyPy project for each problem, or
> I will add only noise in PyPy project ?
>
> BTW, to help PyPy project not only with my brain time, I've did a small
> recurring donation for PyPy 3.
>
> Regards.
> --
> Ludovic Gasc (GMLudo)
>
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
>


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] UTF8 string passing in cffi and PyPy internal string optimizations

[Amaury Forgeot d7;Arc]

Hi,

2015-03-17 18:27 GMT+01:00 Eleytherios Stamatogiannakis :

> Hello,
>
> I'm sending the following here as they involve both cffi and PyPy.
>
> For the last few years i have been trying to find the most efficient way
> to pass UTF8 strings between PyPy and C code using cffi.
>
> Right now when PyPy receives a utf8 string (from a C function) it has to
> do 2 copies:
>
> 1. convert the cdata string to a pypy byte string via ffi.string
> 2. convert ffi.string to a unicode string
>
> When pypy sends a utf8 string it also does 2 copies:
>
> 1. convert pypy unicode string to utf8-encoded byte string
> 2. copy the byte string into a cdata string.
>
> From what i understand, there is a cffi optimization dealing with windows
> unicode (via set_unicode) where on windows platforms and when using the
> native windows unicode strings, cffi avoids doing one of the copies in both
> of above cases.
>
> On linux where the default unicode format for C libraries nowadays is
> UTF8, there is no such optimization, so we have to do the two copies in all
> string passing.
>
> PyPy at some point was going towards using utf8 string internally, but i
> don't know if this is still the plan or not. Using utf8 strings would
> optimize away one of the two copies on the linux platform (utf8
> encoding/decoding would become a nop operator).
>
> All of the above is the current status of cffi and pypy string handling as
> i understand it. So my proposal to reduce the string copies to a minimum is
> this:
>
> 1. If PyPy doesn't go towards using utf8 strings internally, maybe we need
> some special C type that denotes that the string is utf8 and pypy/cffi
> should do the conversion from-to it automatically. Something like "wchar_t"
> in windows but denoting a utf8 string. CFFI can define a special type
> ("__utf8char_t"?) for these strings.
>

This is a first step towards SWIG's typemaps:
http://www.swig.org/Doc3.0/Typemaps.html#Typemaps_nn4

That's also something I wanted to have in another projects: automatic
conversion to PYTHON_HANDLE, for example.

But typemaps are a tough thing, and they would likely differ between
CPython and PyPy.
Armin, what do you think?


Alternatively, an encoding parameter could be added in ffi.string, so that
> it'll do both the cdata and encoding conversions in one step.
>
> 2. If PyPy does go towards using utf8 string internally. Then it could
> call C functions that do not mutate the pypy strings and do not store
> pointers to them, by passing the strings directly. This could be
> accomplished by using a cffi annotation for these kind of
> non-string-mutating C functions.
>

Even utf8 is used internally (it is the case already in the py3k branch, as
a cached attribute), I'm not sure I would like fuctions like strlen() to
silently accept unicode strings...


>
> Above ideas are based on my understanding of the current status and the
> future directions of PyPy. If i have misunderstood something i would be
> glad to be set right :).
>
> Kind regards,
>
> l.
>
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>



-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Language Parser Ontology

[Amaury Forgeot d7;Arc]

Hi Anatoly,

2015-03-09 19:43 GMT+01:00 anatoly techtonik :

> I'll start from afar, so that it will be easier to understand what I
> am thinking about..
>
> CFFI uses pycparser, which parses C files, but! uses C compiler
> to strip comments from C files and process defines, but almost
> all .c files contain comments, so pycparser is basically useless
> as a parser, but maybe it has a good API for working with AST.
>
> Anyway, I tried to see if I can teach pycparser to strip
> comments itself, and in c_lexer.py I found a list of tokens,
> among which there were no token representing the comment
> start.


This looks off-topic for PyPy or CFFI.
Did you intend to send this message to some pycparser forum?


> Stripped list:
>
> ##
> ## All the tokens recognized by the lexer
> ##
> tokens = keywords + (
> # Identifiers
> 'ID',
>
> # Type identifiers (identifiers previously defined as
> # types with typedef)
> 'TYPEID',
>
> # constants
> 'INT_CONST_DEC', 'INT_CONST_OCT', 'INT_CONST_HEX',
> 'FLOAT_CONST', 'HEX_FLOAT_CONST',
> 'CHAR_CONST',
> 'WCHAR_CONST',
> ....
>
> So I thought that I need to add a name for a token
> corresponding to comments start //, /* and end */
> and it will be better if the token name would be somewhat
> common among parsers, so that people looking at token
> could immediately recognize that it is a comment related.
> Apparently, properly naming is a little bit ambiguous for a
> automated processing. Editors like Spyder could also
> benefit information about token and their meaning in
> different programming languages. The processing of text
> comments that can be catched from the parsing stream is
> same for any language and could be IDE independent.
> Right now you can't just reuse the language definitions
> (such as ASDL) to just feed the IDE so that it can
> automatically figure out, what parts of text it can attach
> its functions to.
>
> I read the ontologies is way to express relations between
> object in this automatic was as triples. Like;
>
>   COMMENTSTART is a TOKEN
>   COMMENTSTART starts a COMMENT
>
> And I wonder, have anybody tried to apply this ontology
> stuff to designing and analysing computer languages?
> If yes, maybe there are some databases with such
> information about parsers. I would like to query names of
> all tokens that represent a program comment.
>
> --
> anatoly t.
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>



-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] trsnslating pypy to another language besides C

[Amaury Forgeot d7;Arc]

2015-03-06 13:24 GMT+01:00 anatoly techtonik :

> On Wed, Mar 4, 2015 at 9:24 PM, Maciej Fijalkowski 
> wrote:
> > you need to write a backend. Backends are in translator/xxx.
>
> There are multiple dirs in rpython/translator:
>
>   backendopt
>   c
>   goal
>   platform
>   sandbox
>   test
>   tool
>
> Which of those are backends?
>

Only one: c

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Windows: pypyw.exe?

[Amaury Forgeot d7;Arc]

2015-02-06 11:27 GMT+01:00 Armin Rigo :

> Hi,
>
> On 6 February 2015 at 11:15, Amaury Forgeot d'Arc 
> wrote:
> > This has other implications.
> > For example, sys.stdout points to an invalid file descriptor,
> > and I remember that old versions of pythonw.exe used to freeze after
> > printing 8192 characters.
>
> I'm not finding anything special done with stdout if we're
> pythonw.exe.  Can you be more specific?
>

CPython3 has this code to set sys.stdout to None in this case:
https://hg.python.org/cpython/file/v3.3.4/Python/pythonrun.c#l1083

But that's probably only for python3, python2 chose to not change anything:
http://bugs.python.org/issue706263


> Otherwise I suppose the way forward is to produce pypyw.exe (by
> editing rpython/translator/platform/windows.py, gen_makefile(), rule
> for $(DEFAULT_TARGET), to run $(CC_LINK) twice instead of once), and
> then to wait until people report issues with it.
>
>
> A bientôt,
>
> Armin.
>



-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Windows: pypyw.exe?

[Amaury Forgeot d7;Arc]

2015-02-06 3:34 GMT+01:00 Yaacov Finkelman :

> The difference between pythonw.exe and python.exe is that one is a
> "windows" application and the other is a "console" application.
>

This has other implications.
For example, sys.stdout points to an invalid file descriptor,
and I remember that old versions of pythonw.exe used to freeze after
printing 8192 characters.


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] cppyy questions

[Amaury Forgeot d7;Arc]

2015-01-20 17:00 GMT+01:00 Omer Katz :

> I tried to pass a bytearray and that's also not currently supported.
> Any clue about what should I do with the exception? It definitely
> shouldn't crash the process. I need it to raise a python exception instead.
>
The only way to prevent a crash is to add a "catch" block somehow in C++,
and I don't see anything like this in cppyy.
This said, it's probably a bad idea to continue something after what the
library calls a "FatalError"...
Better add a check like "if e.IsInitialized()" before calling
SerializeToString.



> בתאריך 20 בינו 2015 17:49, ‏"Amaury Forgeot d'Arc" 
> כתב:
>
> 2015-01-20 16:07 GMT+01:00 Omer Katz :
>>
>>> That's correct but can't we handle those cases in cppyy?
>>> We should provide a native Python interface whenever it's possible.
>>>
>>
>> It's not possible to take a Python string as mutable reference.
>>
>> Here are some options that cppyy could implement:
>>
>> - Use bytearray, which is mutable.
>> a = bytearray()
>> e.SerializeToString(a)
>> s = str(a)
>>
>> - Pass a list, and expect the function to append a (python) string
>> l = []
>> e.SerializeToString(s)
>> s = l[0]
>>
>> - Change the signature of the function so that it *returns* the string
>> (like swig's OUTPUT
>> <http://www.swig.org/Doc3.0/Arguments.html#Arguments_nn5>)
>> result, s = e.SerializeToString()
>>
>> I don't know which method is the most convenient with cppyy.
>>
>>
>>
>>>
>>> 2015-01-20 15:40 GMT+02:00 Amaury Forgeot d'Arc :
>>>
>>>> Hi,
>>>>
>>>> 2015-01-20 14:14 GMT+01:00 Omer Katz :
>>>>
>>>>> The documentation is unclear how you can pass a pointer to a Python
>>>>> variable e.g.:
>>>>> str = ""
>>>>>
>>>>> e.SerializeToString(str)
>>>>>
>>>>
>>>> Message::SerializeToString() updates its argument in-place, but Python
>>>> strings are not mutable.
>>>> You should allocate a std::string from Python code, and pass it to the
>>>> function.
>>>> Maybe something like:
>>>>
>>>> s = cppyy.gbl.std.string()
>>>> e.SerializeToString(s)
>>>> print s
>>>>
>>>>
>>>>
>>>>
>>>>>
>>>>> ---
>>>>> TypeError Traceback (most recent call
>>>>> last)
>>>>>  in ()
>>>>> > 1 e.SerializeToString(str)
>>>>>
>>>>> TypeError: none of the 5 overloaded methods succeeded. Full details:
>>>>>   bool google::protobuf::MessageLite::SerializeToString(std::string*)
>>>>> =>
>>>>> TypeError: cannot pass str as basic_string
>>>>>   bool google::protobuf::MessageLite::SerializeToString(std::string*)
>>>>> =>
>>>>> TypeError: cannot pass str as basic_string
>>>>>   bool google::protobuf::MessageLite::SerializeToString(std::string*)
>>>>> =>
>>>>> TypeError: cannot pass str as basic_string
>>>>>   bool google::protobuf::MessageLite::SerializeToString(std::string*)
>>>>> =>
>>>>> TypeError: cannot pass str as basic_string
>>>>>   bool google::protobuf::MessageLite::SerializeToString(std::string*)
>>>>> =>
>>>>> TypeError: cannot pass str as basic_string
>>>>>
>>>>> Best Regards,
>>>>> Omer Katz.
>>>>>
>>>>> ___
>>>>> pypy-dev mailing list
>>>>> pypy-dev@python.org
>>>>> https://mail.python.org/mailman/listinfo/pypy-dev
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Amaury Forgeot d'Arc
>>>>
>>>
>>
>>
>> --
>> Amaury Forgeot d'Arc
>>
>


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] cppyy questions

[Amaury Forgeot d7;Arc]

2015-01-20 16:07 GMT+01:00 Omer Katz :

> That's correct but can't we handle those cases in cppyy?
> We should provide a native Python interface whenever it's possible.
>

It's not possible to take a Python string as mutable reference.

Here are some options that cppyy could implement:

- Use bytearray, which is mutable.
a = bytearray()
e.SerializeToString(a)
s = str(a)

- Pass a list, and expect the function to append a (python) string
l = []
e.SerializeToString(s)
s = l[0]

- Change the signature of the function so that it *returns* the string
(like swig's OUTPUT
<http://www.swig.org/Doc3.0/Arguments.html#Arguments_nn5>)
result, s = e.SerializeToString()

I don't know which method is the most convenient with cppyy.



>
> 2015-01-20 15:40 GMT+02:00 Amaury Forgeot d'Arc :
>
>> Hi,
>>
>> 2015-01-20 14:14 GMT+01:00 Omer Katz :
>>
>>> The documentation is unclear how you can pass a pointer to a Python
>>> variable e.g.:
>>> str = ""
>>>
>>> e.SerializeToString(str)
>>>
>>
>> Message::SerializeToString() updates its argument in-place, but Python
>> strings are not mutable.
>> You should allocate a std::string from Python code, and pass it to the
>> function.
>> Maybe something like:
>>
>> s = cppyy.gbl.std.string()
>> e.SerializeToString(s)
>> print s
>>
>>
>>
>>
>>>
>>> ---
>>> TypeError Traceback (most recent call
>>> last)
>>>  in ()
>>> > 1 e.SerializeToString(str)
>>>
>>> TypeError: none of the 5 overloaded methods succeeded. Full details:
>>>   bool google::protobuf::MessageLite::SerializeToString(std::string*) =>
>>> TypeError: cannot pass str as basic_string
>>>   bool google::protobuf::MessageLite::SerializeToString(std::string*) =>
>>> TypeError: cannot pass str as basic_string
>>>   bool google::protobuf::MessageLite::SerializeToString(std::string*) =>
>>> TypeError: cannot pass str as basic_string
>>>   bool google::protobuf::MessageLite::SerializeToString(std::string*) =>
>>> TypeError: cannot pass str as basic_string
>>>   bool google::protobuf::MessageLite::SerializeToString(std::string*) =>
>>> TypeError: cannot pass str as basic_string
>>>
>>> Best Regards,
>>> Omer Katz.
>>>
>>> ___
>>> pypy-dev mailing list
>>> pypy-dev@python.org
>>> https://mail.python.org/mailman/listinfo/pypy-dev
>>>
>>>
>>
>>
>> --
>> Amaury Forgeot d'Arc
>>
>


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] cppyy questions

[Amaury Forgeot d7;Arc]

Hi,

2015-01-20 14:14 GMT+01:00 Omer Katz :

> The documentation is unclear how you can pass a pointer to a Python
> variable e.g.:
> str = ""
>
> e.SerializeToString(str)
>

Message::SerializeToString() updates its argument in-place, but Python
strings are not mutable.
You should allocate a std::string from Python code, and pass it to the
function.
Maybe something like:

s = cppyy.gbl.std.string()
e.SerializeToString(s)
print s




> ---
> TypeError Traceback (most recent call last)
>  in ()
> > 1 e.SerializeToString(str)
>
> TypeError: none of the 5 overloaded methods succeeded. Full details:
>   bool google::protobuf::MessageLite::SerializeToString(std::string*) =>
> TypeError: cannot pass str as basic_string
>   bool google::protobuf::MessageLite::SerializeToString(std::string*) =>
> TypeError: cannot pass str as basic_string
>   bool google::protobuf::MessageLite::SerializeToString(std::string*) =>
> TypeError: cannot pass str as basic_string
>   bool google::protobuf::MessageLite::SerializeToString(std::string*) =>
> TypeError: cannot pass str as basic_string
>   bool google::protobuf::MessageLite::SerializeToString(std::string*) =>
> TypeError: cannot pass str as basic_string
>
> Best Regards,
> Omer Katz.
>
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
>


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Segmentation Fault

[Amaury Forgeot d7;Arc]

Hi,

Please file a bug at https://bitbucket.org/pypy/pypy/issues
Also, please run "bt" in gdb to show the full stack of the failure.

2015-01-07 16:06 GMT+01:00 Stuart Axon :

> Hi,
>I'm running pypy 2.4.0 on Ubuntu Utopic.
>
> Running the pypy works OK (though it outputs 'trusty' weirdly):
>
>  $ pypy
> Python 2.7.8 (2.4.0+dfsg-1~ppa2+trusty, Sep 25 2014, 04:35:04)
> [PyPy 2.4.0 with GCC 4.8.2] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>>>
>
> Running my program in it gives a segfault:
>
> $ sbot
> Segmentation fault
>
>
> If I run in CPython then things are OK
>
> $ sbot
> usage: usage: sbot [options] inputfile.bot [args] [-h] [-o FILE] [-w] [-f]
>   [-t TITLE] [-s] [-dv]
>   [-p SERVERPORT] [-r
> REPEAT]
>   [-g GRAMMAR] [-c] [-v
> VARS]
>   script [script_args]
> usage: sbot [options] inputfile.bot [args]: error: too few arguments
>
>
>
> I got this out of gdb:
>
> gdb -ex r --args `which pypy` `which sbot`
> GNU gdb (Ubuntu 7.8-1ubuntu4) 7.8.0.20141001-cvs
> Copyright (C) 2014 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <
> http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
> and "show warranty" for details.
> This GDB was configured as "x86_64-linux-gnu".
> Type "show configuration" for configuration details.
> For bug reporting instructions, please see:
> <http://www.gnu.org/software/gdb/bugs/>.
> Find the GDB manual and other documentation resources online at:
> <http://www.gnu.org/software/gdb/documentation/>.
> For help, type "help".
> Type "apropos word" to search for commands related to "word"...
> Reading symbols from
> /mnt/data/home/stu/.virtualenvs/shoebot-pgi-pypy/bin/pypy...(no debugging
> symbols found)...done.
> Starting program:
> /mnt/data/home/stu/.virtualenvs/shoebot-pgi-pypy/bin/pypy
> /mnt/data/home/stu/.virtualenvs/shoebot-pgi-pypy/bin/sbot
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
>
> Program received signal SIGSEGV, Segmentation fault.
> 0x7fffecdb686e in std::_Rb_tree const, std::pair >, std::_Select1st const, std::pair > >, std::less,
> std::allocator > >
> >::_M_get_insert_unique_pos(std::string const&) ()
>from /usr/lib/x86_64-linux-gnu/libprotobuf.so.8
> (gdb)
>
>
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
>


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] stdlib-2.7.9!

[Amaury Forgeot d7;Arc]

I suspect that some 2.7.9 changes should not go in 3.2, but are only
compatible with a 3.3 or 3.4 stdlib...
Is there a way to skip the merge so these changes directly go to the 3.3
branch?

2014-12-14 22:15 GMT+01:00 Alex Gaynor :

> Hey all,
>
> Earlier today I created the 2.7.9 branch, with the copy of the 2.7.9
> stdlib.
>
> http://buildbot.pypy.org/summary?branch=stdlib-2.7.9 is the branch
> summary.
>
> It's no surprise, the biggest work to be done is for the ssl module, 2.7.9
> contains a complete backport of 3.4's ssl module.
>
> We have up through 3.2s version of the ssl module implemented on the py3k
> branch. I'd like some feedback from folks on how you think we should best
> handle finishing the 2.7.9 work.
>
> Should I copy the work from py3k, finish anything missing, and then when
> we get to python 3.4 on the py3k branch the work is just "already done"?
> Something else?
>
> Feedback please!
> Alex
>
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
>


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Unify lib_pypy by vendoring six

[Amaury Forgeot d7;Arc]

Hi Van,

2014-12-05 16:38 GMT+01:00 VanL :

> Hi all,
>
> I’ve been doing some experiments with pypy and I would interested in
> making parts of the codebase more 3x compatible. As a first step, I notice
> that there are slight differences between the lib_pypy shipped in the 2.7
> and 3.2 releases. How would people feel about reducing the duplication by
> consolidating the lib_pypy implementations?
>

lib_pypy is a portion of the stdlib.
It contains the modules that CPython implements in C, and that PyPy decided
to implement in pure Python.

They describe a different version of Python, and have different features.
And why would you want to consolidate the code there, and not say in
urllib2.py or unicodeobject.py?

The strategy would be:
>
> - vendor six.py within lib_pypy
> - unify implementation as much as possible, using either compatible syntax
> or six helpers
> - if the implementation cannot be unified, putting individual
> implementations behind six.PY2 or six.PY3 conditionals
>
> Thoughts?
>
> Thanks,
> Van
>
>
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
>


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] GSoC 2015: cpyext project?

[Amaury Forgeot d7;Arc]

th Cython and Boost.Python extensions are
> functional with minimal effort on behalf of the user. Does anyone have
> any advice? Are there particular things I should familiarise myself
> with? I know there is the module/cpyext tree, but it is quite formidable
> for someone uninitiated!
>
> Of course, I recognise that cpyext is a much trickier proposition in
> comparison with things like cffi and cppyy. In particular, I'm very
> excited by cppyy and PyCling, but they seem quite bound up in CERN's
> ROOT infrastructure, which is a shame. But it's also clear that very
> many useful extensions currently use the CPython API, and so -- as I
> have often found -- the apparent relative immaturity of cpyext keeps
> people away from PyPy, which is also a shame!
>
> [1] https://pypi.python.org/pypi/pyviennacl
> [2] https://bitbucket.org/pypy/pypy/wiki/GSOC%202014
>
> Best,
>
> Toby
>
>
> --
> Toby St Clere Smithe
> http://tsmithe.net
>
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>



-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Segfault in Hy for PyPy 2.4

[Amaury Forgeot d7;Arc]

2014-11-05 1:57 GMT+01:00 Ryan Gonzalez :

> I just built the PyPy alpha yesterday. I can run the tests using nose for
> the Hy <http://docs.hylang.org/en/latest/> project under PyPy 2.5 alpha
> and PyPy 2.3. However, using the prebuilt PyPy 2.4 binaries fails with a
> segfault. Why does this happen?
>

Probably a bug.
Please file an issue here: https://bitbucket.org/pypy/pypy/issues
Also, try to come with a smaller test, it will be much faster for us to fix
it.


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Silly Question re PIP

[Amaury Forgeot d7;Arc]

On my Linux box, "virtualenv" creates an environment which already contains
pip.

2014-10-31 16:47 GMT+01:00 Gary Furash :

> I didn't understand the documentation, but is it the case that you cannot
> use PIP with pypy? When I run pypy get_pip.py, it fails.
>
>
> -- gary furash | furashg...@gmail.com, 520-907-2470 | calendar
> <https://www.google.com/calendar/embed?src=furashgary%40gmail.com&ctz=America/Phoenix&mode=Week>
>
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
>


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

[pypy-dev] PyPy Fractal Generator

[Amaury Forgeot d7;Arc]

It's implemented here:
https://bitbucket.org/pypy/pypy/src/tip/rpython/tool/ansi_mandelbrot.py


On Wed Feb 05 2014 at 7:04:01 AM, Kirk Liberty 
wrote:

> Hi everyone,
>
> I never really email on the list, but I like to lurk and read what
> other people have to say. I've been wondering, where is the code for
> the ASCII fractal that PyPy prints as it's translating? Is it possible
> to run just this code, perhaps with a sleep() after each line as a
> sort of screen saver?
>
> Thanks,
> Kirk
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Boost.Python and PyPy -- status?

[Amaury Forgeot d7;Arc]

2014/1/9 Toby St Clere Smithe 

> Hi there,
>
> I have written a (C)Python extension using Boost::Python, and would like
> to get it working on PyPy. I came across [1], but that's probably a bit
> out of date, and the link given there discussing the status then[2]
> doesn't work any more. Is there an up-to-date status page? Or, does
> anyone know more?
>

Hi,

I just did it again, with Boost 1.55:

- Apply the patch in

https://bitbucket.org/pypy/pypy/src/default/pypy/module/cpyext/patches/boost.patch

- EXCEPT the "using python" part, which should be something like:

using python : 2.7
 : $(PYPY_HOME)/bin/pypy
 : $(PYPY_HOME)/include
 : : : .pypy-22 ;

Then I got the test in "libs/python/example/tutorial" work correctly.
I get segfaults in other example, investigating.


> I don't currently have much time to invest in solving many residual
> issues myself, but I should do come spring / summer.
>
> [1] https://mail.python.org/pipermail/pypy-dev/2010-April/005816.html
> [2] http://paste.pocoo.org/show/203663/
>
> Regards,
>
> Toby Smithe
>
>
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>



-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] clarification cffi vs _ffi_ vs _ffi vs libffi vs _rawffi vs rffi

[Amaury Forgeot d7;Arc]

2013/12/2 Ryan Gonzalez 

> Check the _tkinter and _sqlite3 modules.
>

But they don't do advanced reference management, even if
_tkinter/tclobj.py has some calls to Tcl_IncrRefCount()
and Tcl_DecrRefCount().

The lxml module has interesting code to manage cross-references between C
and Python.
I'm afraid it's a bit difficult to read (it's an almost exact port of the
cython version)
But I found that ffi.new_handle() and ffi.from_handle() are really handy in
this case:
https://github.com/amauryfa/lxml/blob/cffi/src/lxml-cffi/proxy.py#L20



> On Mon, Dec 2, 2013 at 1:29 PM, Dima Tisnek  wrote:
>
>> Thanks, Armin, doc page looks good now!
>>
>> On a related note, I'm thinking to interface jvm/dalvik, similar to
>> jpype (C module) and following pyjnius (cython module), but in cffi.
>>
>> I'd be very grateful if there was a good pypy/cffi project you could
>> point me to as an example, especially of harder bits like reference
>> management.
>>
>> d.
>>
>> On 27 November 2013 18:08, Armin Rigo  wrote:
>> > Re-Hi,
>> >
>> > On Wed, Nov 27, 2013 at 5:24 PM, Armin Rigo  wrote:
>> >>> http://doc.pypy.org/en/latest/extending.html
>> >
>> > I updated that page.
>> >
>> >
>> > Armin
>> ___
>> pypy-dev mailing list
>> pypy-dev@python.org
>> https://mail.python.org/mailman/listinfo/pypy-dev
>>
>
>
>
> --
> Ryan
> When your hammer is C++, everything begins to look like a thumb.
>
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
>


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Preparing for the release of PyPy 2.2

[Amaury Forgeot d7;Arc]

2013/11/12 Ryan Gonzalez 

> Ok, just to make sure, this is what is needed:
>
> -Include a prebuilt TkInter???
>

Not only DLLs, but also header files and .lib.
cffi need a development environment.
The instructions in
http://hg.python.org/cpython/file/v2.7.5/Tools/buildbot/external-common.bat
and
http://hg.python.org/cpython/file/v2.7.5/Tools/buildbot/external.bat<http://hg.python.org/cpython/file/v2.7.5/Tools/buildbot/external-common.bat>
could be reused.


> -Fix the erratic sqlite versions
>
> Correct me if I'm wrong. Two more questions:
>
> -What version of sqlite is needed
>

According to the file above, and to match CPython 2.7.5: sqlite-3.6.21


> -How the heck do I upload this stuff or something?
>
>
> On Tue, Nov 12, 2013 at 1:42 PM, Matti Picus wrote:
>
>> I have been commenting in the bug reports. The general strategy we
>> recommend on the windows build page
>> http://doc.pypy.org/en/latest/windows.html#abridged-method-
>> for-ojit-builds-using-visual-studio-2008
>> is to download a zip file with all the recommended dependencies, and
>> build after adding the appropriate enviroment variables to INCLUDE, LIB,
>> and PATH.
>> So I have been finding all the pieces to add/update/document the needed
>> DLLs. Opinions on a desired version of sqlite3.dll would be appreciated,
>> see comments to issue 1441.
>> Once this is all resolved, we can update the buildbot to use the zip file
>> contents and package the needed dlls
>> Matti
>>
>>
>> On 12/11/2013 9:06 PM, Ryan Gonzalez wrote:
>>
>>> I could probably do the Windows DLL fixes, if someone told me how to
>>> upload the changes...
>>>
>>>
>>> On Mon, Nov 11, 2013 at 5:53 AM, Armin Rigo >> ar...@tunes.org>> wrote:
>>>
>>> Hi all,
>>>
>>> Preparing for the next release.  The Linux binaries of PyPy 2.2,
>>> release candidate, are here:
>>>
>>> http://buildbot.pypy.org/nightly/release-2.2.x/
>>>
>>> This corresponds to the "release-2.2.x" branch in the repo.
>>>
>>> There are a few bug reports that will likely *not* get fixed for lack
>>> of manpower: we would need someone involved with Windows, OS/X, or
>>> the
>>> BSDs.
>>>
>>> https://bugs.pypy.org/issue1626 - Tkinter on Windows
>>> https://bugs.pypy.org/issue1166 - kqueue on OSX/BSD
>>> https://bugs.pypy.org/issue1441 - version of sqlite3.dll on Win.
>>>
>>> Also, we need again someone to make the OS/X binary (Alex?); we can't
>>> use our buildbot because its old compiler builds slightly bogus
>>> binaries.
>>>
>>>
>>> A bientôt,
>>>
>>> Armin.
>>> ___
>>> pypy-dev mailing list
>>> pypy-dev@python.org <mailto:pypy-dev@python.org>
>>>
>>> https://mail.python.org/mailman/listinfo/pypy-dev
>>>
>>>
>>>
>>>
>>> --
>>> Ryan
>>>
>>>
>>> ___
>>> pypy-dev mailing list
>>> pypy-dev@python.org
>>> https://mail.python.org/mailman/listinfo/pypy-dev
>>>
>>
>>
>
>
> --
> Ryan
>
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
>


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] cffi python long

[Amaury Forgeot d7;Arc]

2013/11/8 Nathan Hurst 

> Is there a nicer way to pass python long ints (bigint) into C
> efficiently?  I'm currently cutting the value up into 64 bit chunks in
> python and passing in as an unsigned long*:
>
> cdef("int bigInt(int n, unsigned long* x);")
>
> x = sum(1 << i for i in [100,200,123])
> xs = []
> while x > 0:
> xs.append(x & ((1 << 64) - 1))
> x >>= 64
> print lib.bigInt(len(xs), xs)
>
> but this is quite slow, and it seems like this data must already be
> lurking somewhere in exactly the right form.
>

In Python3 you could use the to_bytes() method of ints,
but I could not find any way to have the same in python2.

Or just use hex(). You might have endianness issues, though.


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] More strategies

[Amaury Forgeot d7;Arc]

2013/11/7 Laurence Tratt 

> The patch itself is here:
>
>
> https://bitbucket.org/pypy/pypy/commits/599ed4285a6de348c7e7e732e303336d3160ce78
>

Hum, [1,2,3].__contains__(1.9)?


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] May we delete the pypy/tkinter repository?

[Amaury Forgeot d7;Arc]

2013/11/4 Maciej Fijalkowski 

> do we have any alternative?
>

Sure. _tkinter has a cffi implementation, in lib_pypy since 2.1.


>
> On Mon, Nov 4, 2013 at 1:16 PM, Matti Picus  wrote:
> > Someone recently tried to use this repository
> > https://bitbucket.org/pypy/tkinter
> > and failed, it would seem we would be better off without it.
> > May I delete it?
> > Matti
> >
> > ___
> > pypy-dev mailing list
> > pypy-dev@python.org
> > https://mail.python.org/mailman/listinfo/pypy-dev
> >
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>



-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] pypy about c-extension

[Amaury Forgeot d7;Arc]

Hi,

2013/11/4 KaShining 

> I use this method to enhance protobuf
> http://yz.mit.edu/wp/fast-native-c-protocol-buffers-from-python/


Don't.
With PyPy, protobuf are much faster with the pure-python version.
The C version is actually much slower on PyPy.

The Encode and Decode functions may look faster,
but accessing values or building a message in Python code will be much
slower.



>
> but after i gen the podpbpypy.so
> then import podpbpypy:
>
> Traceback (most recent call last):
> File "app_main.py", line 72, in run_toplevel
> File "t2.py", line 4, in 
> import podpbpypy
> ImportError: No module named podpbpypy
>
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
>


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] RPython Environment Variables

[Amaury Forgeot d7;Arc]

2013/11/1 Ryan Gonzalez 

> I am writing a tool in RPython that needs to access environment variables.
> Now, all has been going well, but I noticed something interesting. The
> RPython translator sees os.environ as constant. But, will it still be
> constant on other platforms?
>
> What I'm saying is, are the environment variables permanently embedded
> into the application? i.e., if I run it on another PC, the values will be
> the same? Or is there an RLib equivalent? Because I really don't feel like
> writing a C wrapper...
>

RPython magic...

In RPython, os.environ is not a dictionary, but a clever wrapper around
getenv() and putenv().
See the implementation in ll_os_environ.py:
https://bitbucket.org/pypy/pypy/src/default/rpython/rtyper/module/ll_os_environ.py
- as an object, os.environ is equivalent to None: nothing is embedded in
the application.
- only some methods are implemented, and redirect to some implementation
that use the C primitives.

Yes, I also found this "Controller" crazy the first time I saw it. But very
clever and powerful.

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Question on installing pypy and the source code

[Amaury Forgeot d7;Arc]

2013/10/31 Mark Roberts 

> > All the build instructions seem to assume that you have a previous
> version of pypy already installed, in my case, I don't have a previous
> version of pypy already installed and am trying to create the first one..
> :( ..


Assuming you are following "Building from Source":
http://pypy.org/download.html#building-from-source
- The link in §2, "install dependencies" says:
It is, of course, possible to translate with a CPython 2.6 or later
[...]

- §4 says:
works also with 'python' instead of 'pypy'

IMO this is clear enough,
but maybe you found another place where pypy cannot be replaced by python?

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] pip install rbtree fail in pypy env!

[Amaury Forgeot d7;Arc]

2013/10/31 KaShining 

> Tks!
>
> but
>
> >>>>>>>>>>>>>>>
> This is a typical example of a program that will be
> incredibly slower on PyPy if used as a C extension module (via
> cpyext).
> >>>>>>>>>>>>>>>
>
> why?
>

See this blog post:
http://morepypy.blogspot.ch/2011/05/numpy-follow-up.html
specially the paragraph starting with "cpyext is slow"



>
>
> -- Original --
> *From: * "Armin Rigo";;
> *Date: * Thu, Oct 31, 2013 04:53 PM
> *To: * "KaShining"; **
> *Cc: * "pypy-dev"; **
> *Subject: * Re: [pypy-dev] pip install rbtree fail in pypy env!
>
> Hi KaShining,
>
> On Thu, Oct 31, 2013 at 9:32 AM, KaShining  wrote:
> > src/rbtree.c: In function '__pyx_f_6rbtree_6rbtree_byOffset':
>
> This is using Cython.  Compiling Cython modules with PyPy
> kind-of-works but is shaky.  You may have more luck if you upgrade
> Cython to the latest version, but I don't know the details myself.
>
> Note that according to the name "rbtree", it's about a data structure
> that --- I guess --- is meant to contain tons of references to Python
> objects.  This is a typical example of a program that will be
> incredibly slower on PyPy if used as a C extension module (via
> cpyext).  You'd get much better results by running a pure Python
> version of it.  Check "rbtree" to see if it already has a pure Python
> version.
>
>
> A bientôt,
>
> Armin.
>
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
>


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] numpy repo under pypy on bitbucket

[Amaury Forgeot d7;Arc]

Hi,

2013/10/30 Matti Picus 

> Thanks to Brian Kearns, the pypy-compat branch of the numpy repo at
> https://bitbucket.org/pypy/**numpy <https://bitbucket.org/pypy/numpy> <
> https://bitbucket.org/pypy/**numpy/ <https://bitbucket.org/pypy/numpy/>>
> is starting to become the prefrered method of using numpy on pypy. Note
> this is a git repo.
>

Will nightly builds and pypy release still contain a full numpy module?
Or is it necessary to install it manually?

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] trying out STM for some numbers on more cores

[Amaury Forgeot d7;Arc]

2013/10/30 Taavi Burns 

> Unless I missed something (possible!) the JIT and STM are mutually
> exclusive (until implemented).
>

In the last post about STM:
http://morepypy.blogspot.ch/2013/10/update-on-stm.html
"For comparison, disabling the JIT gives 492ms on PyPy-2.1 and 538ms on
PyPy-STM."

JIT on PyPy-STM gives only a slight improvement so far, but it will
definitely slow down things a lot during the tracing phase.


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] trying out STM for some numbers on more cores

[Amaury Forgeot d7;Arc]

2013/10/30 Davide Del Vento 

> Hi Wim,
>
> Thanks for posting your numbers. I think they are interesting and the 11x
> speedup for 16 threads is not bad, however the overhead of STM is still too
> high compared to PyPy. Maybe you need also a larger dataset, besides a
> longer time?
>
>
> > I should run this multiple
> > times and average, but this is just for fun
>
> I think for this purpose you need the best timing, not the average,
> especially if you are using a desktop/laptop. The best timing is something
> that happened and therefore "can happen". The average is affected by a
> variety of other things which may be running on your machine, which are
> better left out for this purpose (it's interesting to study them in order
> to see if one can eliminate them on a production environment, but that's a
> completely different job).
>

Also, the process should perform 1000 iterations before you start the
timings.
The JIT needs a lot of iterations to be warm-up correctly.

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] merging pypy-pyarray branch

[Amaury Forgeot d7;Arc]

2013/9/22 Matti Picus 

> Does anyone have objections to merging pypy-pyarray into default (assuming
> the buildbots come up with no failures)?


I think I have some remarks about some details in the change, I will have a
closer look later.
Nothing that prevents the merge though.
+1

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] PyPy's PyDateTime_Delta definition

[Amaury Forgeot d7;Arc]

2013/9/7 Armin Rigo 

> It seems that "struct PyDateTime_Delta" is supposed to be purely
> read-only.  In case situation, wouldn't it make sense to expose a
> structure with a similar layout?  I don't think we care about the
> overhead of copying three words; the only annoying case is about
> read-write objects, when we want the changes in C to be reflected back
> to the Python world.
>

This is right.
But datetime classes are difficult to handle in cpyext, because they are
pure-python classes. They don't have a specific "TypeDef", and cpyext ties
the C structs to those typedefs.
See sliceobject.py for a simple example of filling the C struct; I don't
know how to do this for datetime.

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] PyPy's PyDateTime_Delta definition

[Amaury Forgeot d7;Arc]

2013/9/7 Skip Montanaro 

> On Fri, Sep 6, 2013 at 6:13 PM, Skip Montanaro  wrote:
> >> Not sure I understand. Or did you mean "why are those declarations not
> in
> >> datetime.h"?
> >
> > Yes, sorry about the typo.
>
> Alas, I am still confused. The PyDateTime_DELTA_GET_* macros aren't
> defined for CPython until the 3.x series. (There are macros in 2.7,
> but they are defined in datetimemodule.c, not in a public header file.
> PyPy declares them as functions.
>
> Is there some predefined "This is PyPy" macro I can check?
>

Now I feel embarrassed.
I was the one who added these macros to CPython 3.3, exactly for this
reason.
see the patch we propose for cx_Oracle:
https://bitbucket.org/pypy/pypy/src/tip/pypy/module/cpyext/patches/cx_Oracle.patch
"#ifdef PyDateTime_DELTA_GET_DAYS" should do the trick for any version.

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] PyPy's PyDateTime_Delta definition

[Amaury Forgeot d7;Arc]

2013/9/7 Skip Montanaro 

> > Sure. Use macros like PyDateTime_DELTA_GET_DAYS(). The structure is not
> part
> > of the public API anyway.
> > With CPython the macros don't make any difference, but PyPy implements
> them
> > as actual function calls.
>
> Ah, that makes sense.  The code in question was written when we were
> still using Python 2.3 or 2.4, before there was any datetime C API. A
> little tweak is in order I guess. Why are those PyDateTime*GET*
> declarations in datetime.h? I might well have noticed them before
> asking a basic question.
>

Not sure I understand. Or did you mean "why are those declarations not in
datetime.h"?
Then the answer is: because all these functions are generated during PyPy
translation.
The corresponding declarations are written in pypy_decl.h

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] PyPy's PyDateTime_Delta definition

[Amaury Forgeot d7;Arc]

2013/9/6 Skip Montanaro 

> Python 2.7's PyDateTime_Delta structure is defined like this:
>
> typedef struct
> {
> PyObject_HEAD
> long hashcode;  /* -1 when unknown */
> int days;   /* -MAX_DELTA_DAYS <= days <=
> MAX_DELTA_DAYS */
> int seconds;/* 0 <= seconds < 24*3600 is invariant */
> int microseconds;   /* 0 <= microseconds < 100 is
> invariant */
> } PyDateTime_Delta;
>
> In contrast, PyPy's version leaves out everything except
> PyObject_HEAD:
>
> typedef struct {
> PyObject_HEAD
> } PyDateTime_Delta;
>
> That doesn't seem to be compatible with C extension modules that want
> to manipulate such objects.  This is my first foray into trying to
> compile some Boost.Python wrappers for libraries at work which
> manipulate Python datetime objects. Why are the other four fields not
> defined? Is there a way to get around this?
>

Sure. Use macros like PyDateTime_DELTA_GET_DAYS(). The structure is not
part of the public API anyway.
With CPython the macros don't make any difference, but PyPy implements them
as actual function calls.

Remember that with PyPy, the C API needs to create a copy of each Python
object (at a fixed address). So to implement the additional fields, it
would be necessary to allocate more and copy more, even when the attributes
are not needed.
Oh, by the way, PyPy uses a pure Python implementation of datetime.py. So
there is little chance we can implement some shortcut in C to directly
reference some internal storage of the datetime types.

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] [pypy-commit] pypy default: Unroll list.count() when the list is virtual or very small and constant lenght

[Amaury Forgeot d7;Arc]

2013/8/30 Carl Friedrich Bolz 

> Hi Alex,
>
> please revert this change, it can lead to combinatorial explosion: it
> can give a bridge per pattern of where in the list the object is found.
>

Doesn't list.count() traverse all the list in all cases?

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] PyPy 2x slower using cpickle

[Amaury Forgeot d7;Arc]

2013/7/18 Eleytherios Stamatogiannakis 

> Yes you are right, cPickle.dump most probably does a lot more small writes.
>
> On the other hand, shouldn't this also affect CPython? Or is CPython so
> much faster in "".join-ing strings?
>

CPython is not affected, because files are implemented with the C fopen,
fwrite...
and use a completely different buffer.


> Is "".join-ing something that we should generally avoid doing in PyPy?
>

This is in RPython code, not in the PyPy interpreter.
But yes, a buffering method that is optimized for append() + flush() is
better
than an all-purpose object.

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] PyPy 2x slower using cpickle

[Amaury Forgeot d7;Arc]

2013/7/18 Eleytherios Stamatogiannakis 

> Why is there such a huge speed difference between cPickle.dump( ... f) and
> f.write(cPickle.dumps(...)) ?


Did you count the number of calls to f.write?
pickle call write() once per pickled object.

Now, pypy's implementation of buffered file uses a (RPython) list of
strings, and does a final ''.join.
This is probably much less efficient than the RStringIO implementation.

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] ndarray-subtype branch looking for review

[Amaury Forgeot d7;Arc]

2013/7/17 Maciej Fijalkowski 

> you go to your checkout (default), you do hg merge  and
> then hg diff. if you don't like it, you do hg up -C to clean stuff up.
> Easy to do in a separate dir (I suggest hg share for that)
>

Actually I created a Pull Request:
https://bitbucket.org/pypy/pypy/pull-request/164/ndarray-subtype-branch-review/diff
Easy to comment on each diff line.

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] ndarray-subtype branch looking for review

[Amaury Forgeot d7;Arc]

2013/7/16 Amaury Forgeot d'Arc 

> I could do it.
> Is there a way to get a full diff of the changes about to be merged?
>

Ok, found it.
https://bitbucket.org/pypy/pypy/compare/ndarray-subtype..default#diff


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] ndarray-subtype branch looking for review

[Amaury Forgeot d7;Arc]

I could do it.
Is there a way to get a full diff of the changes about to be merged?


2013/7/16 Matti Picus 

> The ndarray-branch passes tests both translated and untranslated. I ran it
> through the numpy test suite as well, specifically the matrix class tests,
> and it seems to pass them.
> Would anyone like to review, or should I just go ahead and merge?
> Specifically, I may have missed some w_ prefixes on app-level instances,
> or may not have tested all the code paths, and I had to hack the FakeSpace
> in compile.py to allow subclassing.
> Matti
> __**_
> pypy-dev mailing list
> pypy-dev@python.org
> http://mail.python.org/**mailman/listinfo/pypy-dev<http://mail.python.org/mailman/listinfo/pypy-dev>
>



-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] subclassing ndarray, was Memap in numpypy

[Amaury Forgeot d7;Arc]

Hi Matti,

I think you still have to implement the __array_finalize__ protocol.
Otherwise methods like .transpose() won't work on a memmap object.

2013/7/4 Matti Picus 

> You correctly point out, in other words, that it was an ugly hack.
> It is on the way to being removed by adding a space argument to calls to
> from_space...().
> Let us know how your tests progress.
> Matti
>
> On 07/04/2013 09:57 PM, Mike Beller wrote:
>
>> Matti
>>
>> That's great.  I was stuck trying to figure out how to deal with the fact
>> that if subtype was not-none, the from_shape (and friends) would also need
>> access to 'space'.  I see you addressed that with
>> subtype_and_space=(None,None).It's not a pattern I'd seen in the pypy
>> codebase.  I keep thinking that there are already very specific pypy-ish
>> ways of doing things at interpreter level, and digging to try to learn
>> those patterns, when perhaps things are a bit more in flux than I thought.
>>
>> On awesome side, I think now with your subclassing code, and Armin's
>> modification to mmap.py to support more of the buffer interface, the memap
>> code I've already written may just work.  Will try that next.
>>
>> Mike
>>
>>
>>
>> On Wed, Jul 3, 2013 at 3:29 PM, Matti Picus > matti.pi...@gmail.com>**> wrote:
>>
>> I got motivated and implemented parts of subtypes on ndarray in
>> the ndarray-subtype branch.
>> This page
>> 
>> http://docs.scipy.org/doc/**numpy/user/basics.subclassing.**html<http://docs.scipy.org/doc/numpy/user/basics.subclassing.html>
>> describes the numpy way, so I did
>> ndarray.view(subtype)
>> as well, but I did not yet do the slice (they call it template)
>> nor did I do __array_finalize__
>>
>> Failing tests and patches are welcome
>>
>> Matti
>>
>>
>>
> __**_
> pypy-dev mailing list
> pypy-dev@python.org
> http://mail.python.org/**mailman/listinfo/pypy-dev<http://mail.python.org/mailman/listinfo/pypy-dev>
>



-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] PyPy 2x slower using cpickle

[Amaury Forgeot d7;Arc]

2013/7/3 Eleytherios Stamatogiannakis 

> Hello,
>
> We also found a case where PyPy is 2x slower than CPython. The following
> code:
>

This is because of I/O.
If I replace the file with a custom class which has an empty write() method,
pypy is twice faster than CPython.

Note: with pypy, io.open() is even slower :-(



> <<<<
>
> import cPickle
>
> fileIter=open("pypytesting", "w+b")
> mylist = ["qwerty"] * 100
>
> for i in xrange(100):
> cPickle.dump(mylist, fileIter,1)
>
> >>>>
>
> Runs at:
> CPython 2.7.3:  13.114 sec
> PyPy nightly:29.239 sec
>
> [Warning: it'll produce a file (pypytesting) that is 205 MB in size]
>
> Kind regards,
>
> lefteris.
> __**_
> pypy-dev mailing list
> pypy-dev@python.org
> http://mail.python.org/**mailman/listinfo/pypy-dev<http://mail.python.org/mailman/listinfo/pypy-dev>
>



-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Memap in numpypy

[Amaury Forgeot d7;Arc]

Hello,

2013/6/30 Mike Beller 

> Amaury
>
> Sorry for the high latency.  Day job intervenes.  (This may take some time
> :-) )
>

No problem, we are all volunteers here. There is no pressure.
The worst that can happen is someone may need the feature and implement it
before you :-)


> I have reviewed your advice (below) and I can certainly modify class
> W_NDimArray as proposed.  Here is my question.  It seems from looking at
> other similar modules, the convention in pypy/module is to have w_subtype
> as the second positional parameter to the factory method.   Changing the 4
> factory methods of W_NDimArray to follow this convention would involve
> changing the following references (found using egrep/awk/uniq so I may not
> be exactly right but you get the right order of magnitude):
>
>   2 arrayimpl/concrete.py:
>   2 arrayimpl/scalar.py:
>   1 arrayimpl/sort.py:
>   1 base.py:
>   3 compile.py:
>   5 interp_arrayops.py:
>   1 interp_dtype.py:
>   1 interp_flatiter.py:
>  19 interp_numarray.py:
>   2 interp_support.py:
>   3 interp_ufuncs.py:
>   2 iter.py:
>   3 loop.py:
>   2 test/test_numarray.py:
>
> 47 references in 14 files.
>

Ouch!


> I'm a bit hesitant to change code spewed all over micronumpy unless
> someone with more ownership over the code tells me for sure that's the
> right thing to do.  So I can also propose 2 alternatives:
>
> 1) Create a parallel set of methods:  from_shape_and_subtype,
> from_shape_and_storage_and_subtype, for example, which take w_subtype, but
> this would cause method proliferation.
>
> 2) Add a named parameter to the existing factory methods, so they are like
>   from_shape(shape, subtype=None), but this would be kind of
> nonstandard. On a related matter, I found other modules in the code base
> where w_subtype is ignored (e.g. select/interp_epoll/W_EPoll, and others)
> -- seems a common problem -- so maybe "kind of nonstandard" is not that
> important -- or perhaps this is an area which was simply left to be cleaned
> up later.
>

Both approach can work IMO:
for 1), these are only two new functions, and I'm sure they can share most
of the code with existing functions.
method 2) would work as well, and is proably even easier to implement.
And there is no standard that forbids named parameter! Some calls to
from_shape() already use them...

Do as you prefer, and what makes the code more natural to you.

Cheers,

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Memap in numpypy

[Amaury Forgeot d7;Arc]

2013/6/30 Armin Rigo 

> Hi Mike,
>
> On Sun, Jun 30, 2013 at 7:15 PM, Mike Beller  wrote:
> > is ignored (e.g. select/interp_epoll/W_EPoll, and others) -- seems a
> common
> > problem
>
> At the end of that file we have
> "W_Epoll.typedef.acceptable_as_base_class = False", meaning that it
> cannot be subclassed as app-level; so ignoring w_subtype is correct.
> Are there other examples which don't have "acceptable_as_base_class =
> False" ?
>

I tried to make a list. In the "module/" subdirectory, here are the
__new__ methods that don't use "allocate_instance" and don't define
"acceptable_as_base_class":

clr._CliObject_internal
_ffi.CDLL
_ffi.WinDLL
_ffi.Field
_ffi._StructDescr
numpypy.string_
numpypy.unicode_
numpypy.dtype
numpypy.ndarray
pypyjit.Box
_rawffi.CallbackPtr
_rawffi.CDLL
_winreg.HKEYType
zipimport.zipimporter

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] adding numpy test target to buildbot

[Amaury Forgeot d7;Arc]

2013/6/28 Maciej Fijalkowski 

> > - can buildbot parse output from nose?
>
> buildbot does not parse output "summary" is our custom hack.


How are these summaries implemented btw? A custom pytest reporter?

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] installing after translating

[Amaury Forgeot d7;Arc]

Hi,

2013/6/20 Davide Del Vento 

> Folks,
>
> I've successfully translated pypy 2.0.2 as described here:
> http://pypy.org/download.html#**building-from-source<http://pypy.org/download.html#building-from-source>(which
>  is slightly different from running make).
>
> Now I want to install it in a global location for others to use (it's on a
> machine shared by many users). The problem is, I've done this before, but I
> don't remember what I did and I did not write in my notes about translating.
>
> Do I just rename pypy-c to pypy, place it in a bin directory somewhere in
> the PATH and do the same with site-packages and PYTHONPATH (and
> corresponding actions to include, lib_pypy and lib-python directories)?
>
> Is there any reason why the makefile does not have an install target,
> possibly with prefix option (other than lack of interest or time to write
> it)?
>

The makefile is only to build the C binary. It is generated, by the way.

To build a pypy distribution, after translation you can call
pypy/tool/release/package.py; this is how the nightly downloads are made.

PyPy does not use any prefix or anything: to install, just unpack the .tgz
somewhere and eventually add a link in /usr/local/bin:
http://doc.pypy.org/en/latest/getting-started.html#installing-pypy


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Memap in numpypy

[Amaury Forgeot d7;Arc]

2013/6/20 Mike Beller 

> Thank you both, for your encouragement.
>
> So I have made progress.  I now have some unit tests, and they can compile
> and run.  I have imported the memmap.py from numpy, and modified it so it
> gets the needed items from micronumpy.  I can run the unit tests and they
> fail for the correct reason -- that reason being the buffer attribute is
> not supported in interp_numarray.descr_new_array() .  So that's great.
>

Great indeed!


>
> Here are my next questions:
>
> 1) The way the numpy mmap module works, it  calls ndarray.__new__(), and
> monkey-patches the return value to add  _mmap, offset, and mode attributes.
>  This, for example, ensures the mmap object is kept around until the array
> is deleted.  However, I can't monkey-patch a numpy ndarray object.  I
> presume this is because it is an interpreter level object rather than an
> app level one?  Anyway -- not sure how to deal with this situation.
>

No, that's because ndarray.__new__(subtype, ...) returns a ndarray. This is
wrong, it should return a instance of the subtype (the memmap.memmap class
in this case).
ndarray has no __dict__, OTOH memmap is a Python-defined class and has a
__dict__ where you can store attributes.
(It's also possible to have a __dict__ on ndarray, but it's not necessary
here)

In interp_numarray.py, descr_new_array() does not use w_subtype at all!
This means that ndarray cannot be subclassed in Python...
To make the necessary changes, you can pick one module and see how it's
done there.
For example, in the bz2 module, __new__ could simply have written "return
W_BZ2File(space)", but instead it handles subclasses correctly:

def descr_bz2file__new__(space, w_subtype, __args__):
bz2file = space.allocate_instance(W_BZ2File, w_subtype)
W_BZ2File.__init__(bz2file, space)
return space.wrap(bz2file)

ndarray should do the same, maybe by changing W_NDimArray.from_shape() (and
friends) this way:

@classmethod
def from_shape(w_subtype, shape, dtype, order='C'):
...
if w_subtype is None:
return W_NDimArray(impl)
else:
...allocate_instance, __init__ and wrap...

This subclassing feature should have its own unit tests, btw.


> 2) Secondly, the mmap object itself doesn't really provide a usable buffer
> implementation.  The implementation of buffer(mmap) is currently
> W_MMap.descr_buffer(), (found in interp_mmap.py), which returns a
> StringLikeBuffer object.  This object (implemented in
> pypy/interpreter/buffer.py) is a subclass of Buffer, which does not
> implement get_raw_address().  Our current plan clearly requires the buffer
> object to implement get_raw_address so it can be used by
> ndarray.from_shape_and_storage().  Interestingly, it seems as if the
> interp_mmap author anticipated this shortcoming -- there is a comment:
>  "improve to work directly on low-level address" right in the descr_buffer
> method.
>

> So -- am I on the wrong path?  Should I not even bother trying to use the
> mmap?  (since I can't monkey patch it and it doesn't do what I want?)  This
> would mean perhaps using the underlying rffi mmap to build my own memmap
> module.  Alternatively, can I fix the monkey-patching problem some other
> way, and then take the advice of interp_mmap's author to "improve to work
> directly on low-level address" by returning something better than a
> StringLikeBuffer object.
>

This was the third task I mentioned earlier. It turns out that Armin
implemented it just this morning, thanks! :-)

Mike, you are doing well. Please keep going.

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Memap in numpypy

[Amaury Forgeot d7;Arc]

2013/6/16 Mike Beller 

> Ok -- made some progress.  Built a unit test -- it's similar to the
> doctest from memmap.py, but rewritten in py.test format (uses
> BaseNumpyAppTest from pypy.module.micronumpy.test.test_base, just like
> test_numeric.py does).
>
> Here is the next problem I face:  To run my unit test, I added in an
> import of my new memmap.py, which I have placed in
> lib_pypy/numpypy/core/memmap.py, and linked into core/__init__.py.  This
> module imports mmap.py for obvious reasons (it wants to use mmap.py to
> create the mmap object).  That import fails when I run the test: python2.7
> pytest.py pypy/module/test_lib_pypy/numpypy/core/test_memmap.py ("Import
> error: no module named mmap")
>

great progress anyway!


> It fails because it can not import the module mmap.  Whereas if I just
> fire up a normal interpreter-level pypy I have no problem importing mmap.
>  Clearly it does exist in pypy.
>

> I don't understand which modules will be loadable and which modules will
> not be loadable when I run in pytest.py.  Somehow, clearly, the
> BaseNumpyAppTest inheritance mechanism has allowed all the numpypy stuff to
> show up (but only inside the class definition, not at module scope), but
> other pypy application-level modules are not available.  I need to somehow
> invoke the same magic for mmap if I want to be able to use the mmap module
> to implement the numpypy memmap functionality.I read coding-guide.rst
> but it's still not obvious to me.  (Alternatively I suppose the answer
> could be full translation, but testing this way would take a full 90-minute
> translation every time I wanted to change a line of code.)
>

That's because most built-in modules are not made available by default. You
need to pass the equivalent of "--withmod-mmap" to the interpreter. In a
test, this is done in a statement like:
spaceconfig = dict(usemodules=["mmap"])
at the top of the test class. There certainly is already one (in order to
import numpy), you could add "mmap" to the existing list.

>

>
> If you feel this detailed emailing is wasting your time, just let me know
> and I can drop it.   (Or if you want to take the conversation off pypy-dev
> that's ok too.)
>

You are welcome. PyPy is a wonderful platform with powerful tools, but they
are often not documented enough.
Pypy-dev traffic is not that high these days; to get immediate answers
there many of us are hanging on IRC: #pypy at freenode.net.

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Memap in numpypy

[Amaury Forgeot d7;Arc]

2013/6/14 Mike Beller 

> It seems to me one needs to modify ndarray.__new__ so it can take a named
> buffer= argument, where that argument can be an mmap object.   Currently
> the python ndarray.__new__ can accept such an argument, but the numpypy one
> does not appear to.


Correct.


> When I try to figure out the changes to interp_numarray.py to make it
> work, it goes way over my head.


Hey, you need to get used to our RPython language. At least you opened the
correct file, that's a great first step :-). In interp_numarray.py:

- the block starting with "W_NDimArray.typedef" describe the type and the
methods. There is a "__new__" entry, which describes the constructor. The
implementation is descr_new_array().

- in descr_new_array(), the variables that start with "w_" are Wrapped
objects, i.e. visible to Python (similar to PyObject* in the CPython
implementation). At the moment there is a check for
"space.is_none(w_buffer)", we'll have to remove it of course.

- The goal is to use the memory allocated for w_buffer memory instead of
allocating a new one.
So don't call W_NDimArray.from_shape(),
call W_NDimArray.from_shape_and_storage() instead, it takes the raw address
of the buffer. That's it!

Of course I skipped over all the details, so the first thing to do is to
write a unit test to be sure that everything works correctly before
building a new PyPy.

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Memap in numpypy

[Amaury Forgeot d7;Arc]

Hi, this is definitely doable, but needs some work.
Here are the tasks I identified:

- copy "memmap.py" from the official numpy, and write a unit test (there is
a nice docstring in the previous file)

- add support in numpy for buffers with a fixed address
  (in interp-level terms: a RWBuffer with a get_raw_address() method)

- have buffer(mmap) return such a buffer, very very similar to
array.ArrayType.

The three tasks are quite independent, not too difficult, and could be a
nice start for newcomers...
I'll be happy to help.



2013/6/14 Mike Beller 

> Hello
>
> Do you think it is likely that the memap capabilities of numpy will find
> their way in to numpypy any time soon?
>
> It seems to me that some people think memap is a relatively unimportant
> aspect of numpy.  But I do not think so.  Because of the way the linux IO
> subsystem and virtual memory systems interact, memap enables numpy to have
> high performance access to very large data sets -- it helps make numpy
> relevant to "Big Data".
>
> The code to allow numpy to support memap doesn't seem very large.  But,
> while I have tried reading through the code, I really can not tell whether
> the same is true for numpypy, or whether it is a large endeavor (for
> example, due to some kind of pypy memory management architectural issue).
>
> I'm interested in any input on this.
>
> Mike Beller
>
>
>
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>
>


-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Compatibilidad with boost python

[Amaury Forgeot d7;Arc]

2013/6/6 Riccardo Rossi 

> Dear Amaury,
>  first of all thank you for your answer. I think i could hack the
> boost to compile with a different python, however your last warning is
> actuall the answer i was interested in getting.
>
> if you tell me that the interface will be actually slower than in cpython,
> than most probably it will not fit my needs.
>
> i will keep monitoring your project (i really believe you do a very
> interesting job) in the hope the link to C/C++ will get better (or that
> boost_python will be ported based on cffi)
>

There are fundamental reasons why the CPython API is slow with PyPy
(one of them is that PyPy objects move in memory, so cannot be represented
as simple pointers)

Porting boost::python to cffi is... a challenging idea.
First, a cffi module is written in Python, when a boost::python module is
C++ code.
Then, cffi is currently designed to pass only simple types: int, float,
pointers, struct.
No C++ class instance, and no Python object.

A boost::python::object could be emulated with the help of ffi.new_handle(),
but I don't know whether this integrates easily with C++ memory management
(the "handle" object
needs to stay alive, and this can only be done from Python code)

It's worth exploring though -- and no specific PyPy knowledge is needed for
this.
Help is welcome!

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Compatibilidad with boost python

[Amaury Forgeot d7;Arc]

Hi Riccardo,

2013/6/6 Armin Rigo 

> On Sat, Jun 1, 2013 at 11:59 PM, Riccardo Rossi 
> wrote:
> >   I am writing since I would like to understand if it is
> > possible to use pypy together with modulus and types exported by
> > boost_python.
>
> I am sorry not to be able to answer you.  Hopefully others here have a
> better idea about boost_python than what I do.  Hi there, please speak
> up if you do :-)
>

There was some work a while ago to support boost::python in PyPy.
In any case you have to recompile the boost::python library with pypy
headers
(and .lib on Windows) and your C++ code as well.

I can't remember the details, but I know modified some user-config.jam file
(or was it site-config.jam?) to set the correct Python paths.
And be sure to name your extension module foo.pypy-20.so (or
foo.pypy-20.pyd on Windows)

Then, the usual warnings about cpyext apply:
The interface between C/C++ and Python is very slow,
and not all of the CPython API is implemented.
But it did pass a lot of the test suite.

Sorry I don't have any turnkey solution.

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] comparing strings using is does not works

[Amaury Forgeot d7;Arc]

2013/6/6 Maciej Fijalkowski 

> it *also* depends where the string comes from. I think encodings don't
> respect caching small strings for example


It's really a detail of the C implementation:
PyString_FromString() returns a prebuilt value when the length is 0 or 1,
but _PyString_Resize() won't replace the string being built with a shared
object.
Moreover those details completely changed with Python 3.3 and the new
Unicode implementation.

Don't use "is" with immutable objects (except with the singletons: None,
True, False)

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] blist, was Re: pypy-dev Digest, Vol 26, Issue 5

[Amaury Forgeot d7;Arc]

2013/6/5 Phyo Arkar 

> or write CFFI version?


Unfortunately there is no library to wrap.
All the C code is deeply integrated with the CPython API.

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] why does jit.backend.x86.test.test_fficall crash python on win32?

[Amaury Forgeot d7;Arc]

2013/6/2 Matti Picus 

>  I have been trying to whip win32 own tests into passing, or at least to
> an understanding of why they fail.
> I am seeing the interpreter crash on the
> jit.backend.x86.test.test_fficall.py, here is a traceback
> http://bpaste.net/show/cM9YptahKH0PkZjp5gtN/
> and the original fail is here (not very helpful, sorry)
>
> http://buildbot.pypy.org/summary/longrepr?testname=&builder=own-win-x86-32&build=39&mod=jit%2Fbackend%2Fx86%2Ftest%2Ftest_fficall.py
>

That page has a hint though: the signal number 1073741783 corresponds to
0xC029,
which is STATUS_INVALID_UNWIND_TARGET, and suggests a corruption of the
stack.

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] PyPy doesn't make code written in C faster

[Amaury Forgeot d7;Arc]

2013/5/30 Nathan Hurst 

> > Does it still make sense to add programs like this to our benchmarks?
> > So far, our benchmarks are "real-life" examples.  The benchmarks like
> > above are completely missing the point of PyPy, as they don't stress
> > at all the Python interpreter part.  There are also other cases where
> > PyPy's performance is very bad, like cpyext on an extension module
> > with lots of small C API calls.  I believe that it would still make
> > sense to list such cases in the official benchmark, and have the
> > descriptions of the benchmarks explain what's wrong with them.
>
> I agree that you should include them, I disagree that they are
> 'wrong'.  They measure the overhead of a C call.  Why should a C call
> be slower in pypy than cpython?  Presumably it could be compiled down
> to the appropriate instructions and then out-perform cpy.
>

The C API here is the one of the CPython interpreter (PyLong_FromLong &co)
To support it PyPy has to emulate many aspects, specially the fact that pypy
objects are movable memory, and a PyObject* pointer is not supposed to
change.

To get fair benchmarks, those extension modules should be rewritten,
with cffi for example: its C calls have very little overhead,
and it integrates very well with the rest of the PyPy interpreter.

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] RPython

[Amaury Forgeot d7;Arc]

Hello,

2013/5/28 Dirk Hünniger 

> Hello,
> I am working on a compiler for MediaWiki to LaTeX. Currently it is written
> in Haskell and Python3. I feel very insecure about the Python part and I
> would feel much safer if I had static typechecking in the Python part.
> Still I want the Python part to be able to run with normal Python
> interpreter. So it seems to me that converting the code to RPython might
> solve this issue for. Everything else is Ok, in particular speed is not an
> issue. What do you think.
>

RPython is not the language you are looking for.
No urllib, no xml, no codecs... open() is not even supported!
If you want a statically typed language, write C or Java.

But Python has strong type checks, only at runtime.
I think you'd better write unit tests, or some scripts to exercise the
various tools.
Good code coverage will catch all typos and also many mistakes a compiler
wouldn't tell you.

Cheers,

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] SSL version?

[Amaury Forgeot d7;Arc]

2013/5/14 Stefano Rivera 

> so I'm patching pypy a bit [1] (mainly, PEP3147 support)

(i.e. the __pycache__ directory)

Could you compare your patch with the py3k version of pypy?
I already found interesting differences in importing.py,
with bugs or inefficiencies on both sides.

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] PyPy 2.0 - Einstein Sandwich

[Amaury Forgeot d7;Arc]

2013/5/10 Phyo Arkar 

> Unfortunately there is a dependency on libtinfo.so :
>
> ./bin/pypy: error while loading shared libraries: libtinfo.so.5: cannot
> open shared object file: No such file or directory
>

Yes, this is certainly pulled by the _curses module.

But this was also the case with pypy-1.9.
On which system are you running?

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] x is y <=> id(x)==id(y)

[Amaury Forgeot d7;Arc]

2013/5/6 Armin Rigo 

> On Sun, May 5, 2013 at 10:40 PM, Michael Hudson-Doyle 
> wrote:
> > I want to say something about negative zeroes here
>
> Right: on floats it's not actually the usual equality, but equality of
> the bit pattern (using float2longlong).


Except for NaN...

-- 
Amaury Forgeot d'Arc
___
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev