Maciej Fijalkowski, 13.12.2012 09:13: > On Thu, Dec 13, 2012 at 9:35 AM, Stefan Behnel wrote: >> Maciej Fijalkowski, 12.12.2012 20:10: >>> On Wed, Dec 12, 2012 at 7:06 PM, Joe Hillenbrand wrote: >>>> I was able to fix the issue with scrapy. >>>> >>>> https://github.com/joehillen/scrapy/commit/8778af5c5be50a5d746751352f8d710d1f24681c >>>> >>>> Unfortunately, scrapy takes twice as long in PyPy than in CPython. I >>>> suspect >>>> this is because lxml is twice as slow in PyPy vs CPython, which I found in >>>> lxml's benchmarks. >>>> >>>> Should lxml be added to the set of speed tests? >>> >>> no. lxml uses cpyext (CPython extension compatibility) that is and >>> will forever be slow. >> >> Well, I don't think it would be hard for any PyPy core developer to make it >> twice as fast. Shouldn't be more than a day's work. > > I'm not so sure, we wouldn't know until someone tries it. What > optimizations did you have in mind?
Anything that creates a proper fast-path in the ref-counting functions and that generally takes pressure off them, e.g. by keeping PyObjects alive in a weakref dict as long as the corresponding PyPy object lives, so that useless re-allocation cycles are avoided. I'm sure that really simple changes can bring a substantial improvement here. > For what is worth, cpyext is not twice as slow, lxml is. cpyext is > likely 10-20x slower or so. I presume lowering the overhead would not > automatically make lxml twice as fast, since it's doing quite a lot of > other work. lxml's API performance suffers a lot from object/reference creation and deallocation time, so making object deallocation faster and making it happen only when necessary would certainly improve the overall performance. Stefan _______________________________________________ pypy-dev mailing list [email protected] http://mail.python.org/mailman/listinfo/pypy-dev
