Author: Manuel Jacob <m...@manueljacob.de> Branch: py3.3 Changeset: r82835:a05bd4b30cfe Date: 2016-03-07 02:56 +0100 http://bitbucket.org/pypy/pypy/changeset/a05bd4b30cfe/
Log: hg merge py3k diff too long, truncating to 2000 out of 2700 lines diff --git a/pypy/doc/contributor.rst b/pypy/doc/contributor.rst --- a/pypy/doc/contributor.rst +++ b/pypy/doc/contributor.rst @@ -11,29 +11,29 @@ Amaury Forgeot d'Arc Antonio Cuni Samuele Pedroni + Matti Picus Alex Gaynor Brian Kearns - Matti Picus Philip Jenvey Michael Hudson + Ronan Lamy David Schneider + Manuel Jacob Holger Krekel Christian Tismer Hakan Ardo - Manuel Jacob - Ronan Lamy Benjamin Peterson + Richard Plangger Anders Chrigstrom Eric van Riet Paap Wim Lavrijsen - Richard Plangger Richard Emslie Alexander Schremmer Dan Villiom Podlaski Christiansen + Remi Meier Lukas Diekmann Sven Hager Anders Lehmann - Remi Meier Aurelien Campeas Niklaus Haldimann Camillo Bruni @@ -42,8 +42,8 @@ Romain Guillebert Leonardo Santagada Seo Sanghyeon + Ronny Pfannschmidt Justin Peel - Ronny Pfannschmidt David Edelsohn Anders Hammarquist Jakub Gustak @@ -65,6 +65,7 @@ Tyler Wade Michael Foord Stephan Diehl + Vincent Legoll Stefan Schwarzer Valentino Volonghi Tomek Meka @@ -75,9 +76,9 @@ Jean-Paul Calderone Timo Paulssen Squeaky + Marius Gedminas Alexandre Fayolle Simon Burton - Marius Gedminas Martin Matusiak Konstantin Lopuhin Wenzhu Man @@ -86,16 +87,20 @@ Ivan Sichmann Freitas Greg Price Dario Bertini + Stefano Rivera Mark Pearse Simon Cross Andreas Stührk - Stefano Rivera + Edd Barrett Jean-Philippe St. Pierre Guido van Rossum Pavel Vinogradov + Jeremy Thurgood Paweł Piotr Przeradowski + Spenser Bauman Paul deGrandis Ilya Osadchiy + marky1991 Tobias Oberstein Adrian Kuhn Boris Feigin @@ -104,14 +109,12 @@ Georg Brandl Bert Freudenberg Stian Andreassen - Edd Barrett + Tobias Pape Wanja Saatkamp Gerald Klix Mike Blume - Tobias Pape Oscar Nierstrasz Stefan H. Muller - Jeremy Thurgood Rami Chowdhury Eugene Oden Henry Mason @@ -123,6 +126,8 @@ Lukas Renggli Guenter Jantzen Ned Batchelder + Tim Felgentreff + Anton Gulenko Amit Regmi Ben Young Nicolas Chauvat @@ -132,12 +137,12 @@ Nicholas Riley Jason Chu Igor Trindade Oliveira - Tim Felgentreff + Yichao Yu Rocco Moretti Gintautas Miliauskas Michael Twomey Lucian Branescu Mihaila - Yichao Yu + Devin Jeanpierre Gabriel Lavoie Olivier Dormond Jared Grubb @@ -161,33 +166,33 @@ Stanislaw Halik Mikael Schönenberg Berkin Ilbeyi - Elmo M?ntynen + Elmo Mäntynen + Faye Zhao Jonathan David Riehl Anders Qvist Corbin Simpson Chirag Jadwani Beatrice During Alex Perry - Vincent Legoll + Vaibhav Sood Alan McIntyre - Spenser Bauman + William Leslie Alexander Sedov Attila Gobi + Jasper.Schulz Christopher Pope - Devin Jeanpierre - Vaibhav Sood Christian Tismer Marc Abramowitz Dan Stromberg Arjun Naik Valentina Mukhamedzhanova Stefano Parmesan + Mark Young Alexis Daboville Jens-Uwe Mager Carl Meyer Karl Ramm Pieter Zieschang - Anton Gulenko Gabriel Lukas Vacek Andrew Dalke @@ -195,6 +200,7 @@ Jakub Stasiak Nathan Taylor Vladimir Kryachko + Omer Katz Jacek Generowicz Alejandro J. Cura Jacob Oscarson @@ -209,6 +215,7 @@ Lars Wassermann Philipp Rustemeuer Henrik Vendelbo + Richard Lancaster Dan Buch Miguel de Val Borro Artur Lisiecki @@ -220,18 +227,18 @@ Tomo Cocoa Kim Jin Su Toni Mattis + Amber Brown Lucas Stadler Julian Berman Markus Holtermann roberto@goyle Yury V. Zaytsev Anna Katrina Dominguez - William Leslie Bobby Impollonia - Faye Zhao t...@eistee.fritz.box Andrew Thompson Yusei Tahara + Aaron Tubbs Ben Darnell Roberto De Ioris Juan Francisco Cantero Hurtado @@ -243,6 +250,7 @@ Christopher Armstrong Michael Hudson-Doyle Anders Sigfridsson + Nikolay Zinov Yasir Suhail Jason Michalski rafalgalczyn...@gmail.com @@ -252,6 +260,7 @@ Gustavo Niemeyer Stephan Busemann Rafał Gałczyński + Matt Bogosian Christian Muirhead Berker Peksag James Lan @@ -286,9 +295,9 @@ Stefan Marr jiaaro Mads Kiilerich - Richard Lancaster opassembler.py Antony Lee + Jason Madden Yaroslav Fedevych Jim Hunziker Markus Unterwaditzer @@ -297,6 +306,7 @@ squeaky Zearin soareschen + Jonas Pfannschmidt Kurt Griffiths Mike Bayer Matthew Miller @@ -311,4 +321,3 @@ Julien Phalip Roman Podoliaka Dan Loewenherz - diff --git a/pypy/doc/index-of-release-notes.rst b/pypy/doc/index-of-release-notes.rst --- a/pypy/doc/index-of-release-notes.rst +++ b/pypy/doc/index-of-release-notes.rst @@ -6,6 +6,7 @@ .. toctree:: + release-5.0.0.rst release-4.0.1.rst release-4.0.0.rst release-2.6.1.rst diff --git a/pypy/doc/index-of-whatsnew.rst b/pypy/doc/index-of-whatsnew.rst --- a/pypy/doc/index-of-whatsnew.rst +++ b/pypy/doc/index-of-whatsnew.rst @@ -7,6 +7,7 @@ .. toctree:: whatsnew-head.rst + whatsnew-5.0.0.rst whatsnew-4.0.1.rst whatsnew-4.0.0.rst whatsnew-2.6.1.rst diff --git a/pypy/doc/release-5.0.0.rst b/pypy/doc/release-5.0.0.rst new file mode 100644 --- /dev/null +++ b/pypy/doc/release-5.0.0.rst @@ -0,0 +1,226 @@ +========== +PyPy 5.0.0 +========== + +We have released PyPy 5.0.0, about three months after PyPy 4.0.0. +We encourage all users of PyPy to update to this version. Apart from the usual +bug fixes, there is an ongoing effort to improve the warmup time and memory +usage of JIT-related metadata. The exact effects depend vastly on the program +you're running and can range from insignificant to warmup being up to 30% +faster and memory dropping by about 30%. + +We also merged a major upgrade to our C-API layer (cpyext), simplifying the +interaction between c-level objects and PyPy interpreter level objects. As a +result, lxml with its cython compiled component `passes all tests`_ on PyPy + +Users who have gotten used to vmprof_ on Linux, and those on other platforms +who have not yet tried its awesomeness, will be happy to hear that vmprof +now just works on MacOS and Windows too, in both PyPy (built-in support) and +CPython (as an installed module). + +You can download the PyPy 5.0.0 release here: + + http://pypy.org/download.html + +We would like to thank our donors for the continued support of the PyPy +project. + +We would also like to thank our contributors and +encourage new people to join the project. PyPy has many +layers and we need help with all of them: `PyPy`_ and `RPython`_ documentation +improvements, tweaking popular `modules`_ to run on pypy, or general `help`_ +with making RPython's JIT even better. + +CFFI +==== + +While not applicable only to PyPy, `cffi`_ is arguably our most significant +contribution to the python ecosystem. PyPy 5.0.0 ships with +`cffi-1.5.2`_ which now allows embedding PyPy (or cpython) in a c program. + +.. _`PyPy`: http://doc.pypy.org +.. _`RPython`: https://rpython.readthedocs.org +.. _`cffi`: https://cffi.readthedocs.org +.. _`cffi-1.5.2`: http://cffi.readthedocs.org/en/latest/whatsnew.html#v1-5-2 +.. _`modules`: http://doc.pypy.org/en/latest/project-ideas.html#make-more-python-modules-pypy-friendly +.. _`help`: http://doc.pypy.org/en/latest/project-ideas.html +.. _`numpy`: https://bitbucket.org/pypy/numpy +.. _`passes all tests`: https://bitbucket.org/pypy/compatibility/wiki/lxml +.. _vmprof: http://vmprof.readthedocs.org + +What is PyPy? +============= + +PyPy is a very compliant Python interpreter, almost a drop-in replacement for +CPython 2.7. It's fast (`pypy and cpython 2.7.x`_ performance comparison) +due to its integrated tracing JIT compiler. + +We also welcome developers of other +`dynamic languages`_ to see what RPython can do for them. + +This release supports **x86** machines on most common operating systems +(Linux 32/64, Mac OS X 64, Windows 32, OpenBSD, freebsd), +newer **ARM** hardware (ARMv6 or ARMv7, with VFPv3) running Linux, and the +big- and little-endian variants of **ppc64** running Linux. + +.. _`pypy and cpython 2.7.x`: http://speed.pypy.org +.. _`dynamic languages`: http://pypyjs.org + +Other Highlights (since 4.0.1 released in November 2015) +========================================================= + +* New features: + + * Support embedding PyPy in a C-program via cffi and static callbacks in cffi. + This deprecates the old method of embedding PyPy + + * Refactor vmprof to work cross-operating-system, deprecate using buggy + libunwind on Linux platforms. Vmprof even works on Windows now. + + * Support more of the C-API type slots, like tp_getattro, and fix C-API + macros, functions, and structs such as _PyLong_FromByteArray(), + PyString_GET_SIZE, f_locals in PyFrameObject, Py_NAN, co_filename in + PyCodeObject + + * Use a more stable approach for allocating PyObjects in cpyext. (see + `blog post`_). Once the PyObject corresponding to a PyPy object is created, + it stays around at the same location until the death of the PyPy object. + Done with a little bit of custom GC support. It allows us to kill the + notion of "borrowing" inside cpyext, reduces 4 dictionaries down to 1, and + significantly simplifies the whole approach (which is why it is a new + feature while technically a refactoring) and allows PyPy to support the + populart lxml module (as of the *next* release) with no PyPy specific + patches needed + + * Make the default filesystem encoding ASCII, like CPython + + * Use `hypothesis`_ in test creation, which is great for randomizing tests + +* Bug Fixes + + * Backport always using os.urandom for uuid4 from cpython and fix the JIT as well + (issue #2202) + + * More completely support datetime, optimize timedelta creation + + * Fix for issue 2185 which caused an inconsistent list of operations to be + generated by the unroller, appeared in a complicated DJango app + + * Fix an elusive issue with stacklets on shadowstack which showed up when + forgetting stacklets without resuming them + + * Fix entrypoint() which now acquires the GIL + + * Fix direct_ffi_call() so failure does not bail out before setting CALL_MAY_FORCE + + * Fix (de)pickling long values by simplifying the implementation + + * Fix RPython rthread so that objects stored as threadlocal do not force minor + GC collection and are kept alive automatically. This improves perfomance of + short-running Python callbacks and prevents resetting such object between + calls + + * Support floats as parameters to itertools.isslice() + + * Check for the existence of CODESET, ignoring it should have prevented PyPy + from working on FreeBSD + + * Fix for corner case (likely shown by Krakatau) for consecutive guards with + interdependencies + + * Fix applevel bare class method comparisons which should fix pretty printing + in IPython + + * Issues reported with our previous release were resolved_ after reports from users on + our issue tracker at https://bitbucket.org/pypy/pypy/issues or on IRC at + #pypy + +* Numpy: + + * Updates to numpy 1.10.2 (incompatibilities and not-implemented features + still exist) + + * Support dtype=(('O', spec)) union while disallowing record arrays with + mixed object, non-object values + + * Remove all traces of micronumpy from cpyext if --withoutmod-micronumpy option used + + * Support indexing filtering with a boolean ndarray + + * Support partition() as an app-level function, together with a cffi wrapper + in pypy/numpy, this now provides partial support for partition() + +* Performance improvements and refactorings: + + * Refactor and improve exception analysis in the annotator + + * Remove unnecessary special handling of space.wrap(). + + * Improve the memory signature of numbering instances in the JIT. This should + massively decrease the amount of memory consumed by the JIT, which is + significant for most programs. Also compress the numberings using variable- + size encoding + + * Support list-resizing setslice operations in RPython + + * Tweak the trace-too-long heuristic for multiple jit drivers + + * Refactor bookkeeping (such a cool word - three double letters) in the + annotater + + * Refactor wrappers for OS functions from rtyper to rlib and simplify them + + * Simplify backend loading instructions to only use four variants + + * Optimize string concatination + + * Simplify GIL handling in non-jitted code + + * Use INT_LSHIFT instead of INT_MUL when possible + + * Improve struct.unpack by casting directly from the underlying buffer. + Unpacking floats and doubles is about 15 times faster, and integer types + about 50% faster (on 64 bit integers). This was then subsequently + improved further in optimizeopt.py. + + * Refactor naming in optimizeopt + + * Change GraphAnalyzer to use a more precise way to recognize external + functions and fix null pointer handling, generally clean up external + function handling + + * Optimize global lookups + + * Optimize two-tuple lookups in mapdict + + * Remove pure variants of ``getfield_gc_*`` operations from the JIT by + determining purity while tracing + + * Refactor databasing + + * Simplify bootstrapping in cpyext + + * Refactor rtyper debug code into python.rtyper.debug + + * Seperate structmember.h from Python.h Also enhance creating api functions + to specify which header file they appear in (previously only pypy_decl.h) + + * Reduce all guards from int_floordiv_ovf if one of the arguments is constant + + * Fix tokenizer to enforce universal newlines, needed for Python 3 support + + * Identify permutations of attributes at instance creation, reducing the + number of bridges created + + * Greatly improve re.sub() performance + +.. _resolved: http://doc.pypy.org/en/latest/whatsnew-5.0.0.html +.. _`hypothesis`: http://hypothesis.readthedocs.org +.. _`blog post`: http://morepypy.blogspot.com/2016/02/c-api-support-update.html + +Please update, and continue to help us make PyPy better. + +Cheers + +The PyPy Team + diff --git a/pypy/doc/whatsnew-head.rst b/pypy/doc/whatsnew-5.0.0.rst copy from pypy/doc/whatsnew-head.rst copy to pypy/doc/whatsnew-5.0.0.rst --- a/pypy/doc/whatsnew-head.rst +++ b/pypy/doc/whatsnew-5.0.0.rst @@ -1,6 +1,6 @@ -========================= -What's new in PyPy 4.1.+ -========================= +======================== +What's new in PyPy 5.0.0 +======================== .. this is a revision shortly after release-4.0.1 .. startrev: 4b5c840d0da2 @@ -183,4 +183,15 @@ .. branch: vlen-resume -Compress resume data, saving 10-20% of memory consumed by the JIT \ No newline at end of file +Compress resume data, saving 10-20% of memory consumed by the JIT + +.. branch: issue-2248 + +.. branch: ndarray-setitem-filtered + +Fix boolean-array indexing in micronumpy + +.. branch: numpy_partition +Support ndarray.partition() as an app-level function numpy.core._partition_use, +provided as a cffi wrapper to upstream's implementation in the pypy/numpy repo + diff --git a/pypy/doc/whatsnew-head.rst b/pypy/doc/whatsnew-head.rst --- a/pypy/doc/whatsnew-head.rst +++ b/pypy/doc/whatsnew-head.rst @@ -1,186 +1,7 @@ ========================= -What's new in PyPy 4.1.+ +What's new in PyPy 5.0.+ ========================= -.. this is a revision shortly after release-4.0.1 -.. startrev: 4b5c840d0da2 +.. this is a revision shortly after release-5.0.0 +.. startrev: 7bb6381d084c -Fixed ``_PyLong_FromByteArray()``, which was buggy. - -Fixed a crash with stacklets (or greenlets) on non-Linux machines -which showed up if you forget stacklets without resuming them. - -.. branch: numpy-1.10 - -Fix tests to run cleanly with -A and start to fix micronumpy for upstream numpy -which is now 1.10.2 - -.. branch: osx-flat-namespace - -Fix the cpyext tests on OSX by linking with -flat_namespace - -.. branch: anntype - -Refactor and improve exception analysis in the annotator. - -.. branch: posita/2193-datetime-timedelta-integrals - -Fix issue #2193. ``isinstance(..., int)`` => ``isinstance(..., numbers.Integral)`` -to allow for alternate ``int``-like implementations (e.g., ``future.types.newint``) - -.. branch: faster-rstruct - -Improve the performace of struct.unpack, which now directly reads inside the -string buffer and directly casts the bytes to the appropriate type, when -allowed. Unpacking of floats and doubles is about 15 times faster now, while -for integer types it's up to ~50% faster for 64bit integers. - -.. branch: wrap-specialisation - -Remove unnecessary special handling of space.wrap(). - -.. branch: compress-numbering - -Improve the memory signature of numbering instances in the JIT. This should massively -decrease the amount of memory consumed by the JIT, which is significant for most programs. - -.. branch: fix-trace-too-long-heuristic - -Improve the heuristic when disable trace-too-long - -.. branch: fix-setslice-can-resize - -Make rlist's ll_listsetslice() able to resize the target list to help -simplify objspace/std/listobject.py. Was issue #2196. - -.. branch: anntype2 - -A somewhat random bunch of changes and fixes following up on branch 'anntype'. Highlights: - -- Implement @doubledispatch decorator and use it for intersection() and difference(). - -- Turn isinstance into a SpaceOperation - -- Create a few direct tests of the fundamental annotation invariant in test_model.py - -- Remove bookkeeper attribute from DictDef and ListDef. - -.. branch: cffi-static-callback - -.. branch: vecopt-absvalue - -- Enhancement. Removed vector fields from AbstractValue. - -.. branch: memop-simplify2 - -Simplification. Backends implement too many loading instructions, only having a slightly different interface. -Four new operations (gc_load/gc_load_indexed, gc_store/gc_store_indexed) replace all the -commonly known loading operations - -.. branch: more-rposix - -Move wrappers for OS functions from `rpython/rtyper` to `rpython/rlib` and -turn them into regular RPython functions. Most RPython-compatible `os.*` -functions are now directly accessible as `rpython.rposix.*`. - -.. branch: always-enable-gil - -Simplify a bit the GIL handling in non-jitted code. Fixes issue #2205. - -.. branch: flowspace-cleanups - -Trivial cleanups in flowspace.operation : fix comment & duplicated method - -.. branch: test-AF_NETLINK - -Add a test for pre-existing AF_NETLINK support. Was part of issue #1942. - -.. branch: small-cleanups-misc - -Trivial misc cleanups: typo, whitespace, obsolete comments - -.. branch: cpyext-slotdefs -.. branch: fix-missing-canraise -.. branch: whatsnew - -.. branch: fix-2211 - -Fix the cryptic exception message when attempting to use extended slicing -in rpython. Was issue #2211. - -.. branch: ec-keepalive - -Optimize the case where, in a new C-created thread, we keep invoking -short-running Python callbacks. (CFFI on CPython has a hack to achieve -the same result.) This can also be seen as a bug fix: previously, -thread-local objects would be reset between two such calls. - -.. branch: globals-quasiimmut - -Optimize global lookups. - -.. branch: cffi-static-callback-embedding - -Updated to CFFI 1.5, which supports a new way to do embedding. -Deprecates http://pypy.readthedocs.org/en/latest/embedding.html. - -.. branch: fix-cpython-ssl-tests-2.7 - -Fix SSL tests by importing cpython's patch - - -.. branch: remove-getfield-pure - -Remove pure variants of ``getfield_gc_*`` operations from the JIT. Relevant -optimizations instead consult the field descriptor to determine the purity of -the operation. Additionally, pure ``getfield`` operations are now handled -entirely by `rpython/jit/metainterp/optimizeopt/heap.py` rather than -`rpython/jit/metainterp/optimizeopt/pure.py`, which can result in better codegen -for traces containing a large number of pure getfield operations. - -.. branch: exctrans - -Try to ensure that no new functions get annotated during the 'source_c' phase. -Refactor sandboxing to operate at a higher level. - -.. branch: cpyext-bootstrap - -.. branch: vmprof-newstack - -Refactor vmprof to work cross-operating-system. - -.. branch: seperate-strucmember_h - -Seperate structmember.h from Python.h Also enhance creating api functions -to specify which header file they appear in (previously only pypy_decl.h) - -.. branch: llimpl - -Refactor register_external(), remove running_on_llinterp mechanism and -apply sandbox transform on externals at the end of annotation. - -.. branch: cffi-embedding-win32 - -.. branch: windows-vmprof-support - -vmprof should work on Windows. - - -.. branch: reorder-map-attributes - -When creating instances and adding attributes in several different orders -depending on some condition, the JIT would create too much code. This is now -fixed. - -.. branch: cpyext-gc-support-2 - -Improve CPython C API support, which means lxml now runs unmodified -(after removing pypy hacks, pending pull request) - -.. branch: look-inside-tuple-hash - -Look inside tuple hash, improving mdp benchmark - -.. branch: vlen-resume - -Compress resume data, saving 10-20% of memory consumed by the JIT \ No newline at end of file diff --git a/pypy/interpreter/function.py b/pypy/interpreter/function.py --- a/pypy/interpreter/function.py +++ b/pypy/interpreter/function.py @@ -13,6 +13,9 @@ from pypy.interpreter.argument import Arguments from rpython.rlib import jit +from rpython.rlib.rarithmetic import LONG_BIT +from rpython.rlib.rbigint import rbigint + funccallunrolling = unrolling_iterable(range(4)) @@ -547,6 +550,24 @@ return space.w_False return space.eq(self.w_function, w_other.w_function) + def is_w(self, space, other): + if not isinstance(other, Method): + return False + return (self.w_instance is other.w_instance and + self.w_function is other.w_function) + + def immutable_unique_id(self, space): + from pypy.objspace.std.util import IDTAG_METHOD as tag + from pypy.objspace.std.util import IDTAG_SHIFT + if self.w_instance is not None: + id = space.bigint_w(space.id(self.w_instance)) + id = id.lshift(LONG_BIT) + else: + id = rbigint.fromint(0) + id = id.or_(space.bigint_w(space.id(self.w_function))) + id = id.lshift(IDTAG_SHIFT).int_or_(tag) + return space.newlong_from_rbigint(id) + def descr_method_hash(self): space = self.space w_result = space.hash(self.w_function) diff --git a/pypy/interpreter/test/test_app_main.py b/pypy/interpreter/test/test_app_main.py --- a/pypy/interpreter/test/test_app_main.py +++ b/pypy/interpreter/test/test_app_main.py @@ -133,7 +133,7 @@ self.check(['-S', '-O', '--info'], {}, output_contains='translation') self.check(['-S', '-O', '--version'], {}, output_contains='Python') self.check(['-S', '-OV'], {}, output_contains='Python') - self.check(['--jit', 'foobar', '-S'], {}, sys_argv=[''], + self.check(['--jit', 'off', '-S'], {}, sys_argv=[''], run_stdin=True, no_site=1) self.check(['-c', 'pass'], {}, sys_argv=['-c'], run_command='pass') self.check(['-cpass'], {}, sys_argv=['-c'], run_command='pass') diff --git a/pypy/interpreter/test/test_function.py b/pypy/interpreter/test/test_function.py --- a/pypy/interpreter/test/test_function.py +++ b/pypy/interpreter/test/test_function.py @@ -1,5 +1,5 @@ # encoding: utf-8 -import unittest +import pytest from pypy.interpreter import eval from pypy.interpreter.function import Function, Method, descr_function_get from pypy.interpreter.pycode import PyCode @@ -232,6 +232,7 @@ raises( TypeError, func, 42, {'arg1': 23}) + @pytest.mark.skipif("config.option.runappdirect") def test_kwargs_nondict_mapping(self): class Mapping: def keys(self): @@ -302,6 +303,14 @@ meth = func.__get__(obj, object) assert meth() == obj + def test_none_get_interaction(self): + skip("XXX issue #2083") + assert type(None).__repr__(None) == 'None' + + def test_none_get_interaction_2(self): + f = None.__repr__ + assert f() == 'None' + def test_no_get_builtin(self): assert not hasattr(dir, '__get__') class A(object): @@ -329,6 +338,7 @@ raises(TypeError, len, s, some_unknown_keyword=s) raises(TypeError, len, s, s, some_unknown_keyword=s) + @pytest.mark.skipif("config.option.runappdirect") def test_call_error_message(self): try: len() @@ -374,6 +384,7 @@ f = lambda: 42 assert f.__doc__ is None + @pytest.mark.skipif("config.option.runappdirect") def test_setstate_called_with_wrong_args(self): f = lambda: 42 # not sure what it should raise, since CPython doesn't have setstate @@ -536,6 +547,37 @@ assert A().m == X() assert X() == A().m + @pytest.mark.skipif("config.option.runappdirect") + def test_method_identity(self): + class A(object): + def m(self): + pass + def n(self): + pass + + class B(A): + pass + + class X(object): + def __eq__(self, other): + return True + + a = A() + a2 = A() + assert a.m is a.m + assert id(a.m) == id(a.m) + assert a.m is not a.n + assert id(a.m) != id(a.n) + assert a.m is not a2.m + assert id(a.m) != id(a2.m) + + assert A.m is A.m + assert id(A.m) == id(A.m) + assert A.m is not A.n + assert id(A.m) != id(A.n) + assert A.m is B.m + assert id(A.m) == id(B.m) + class TestMethod: def setup_method(self, method): diff --git a/pypy/module/__builtin__/descriptor.py b/pypy/module/__builtin__/descriptor.py --- a/pypy/module/__builtin__/descriptor.py +++ b/pypy/module/__builtin__/descriptor.py @@ -112,6 +112,7 @@ W_Super.typedef = TypeDef( 'super', __new__ = interp2app(descr_new_super), + __thisclass__ = interp_attrproperty_w("w_starttype", W_Super), __getattribute__ = interp2app(W_Super.getattribute), __get__ = interp2app(W_Super.get), __doc__ = """super(type) -> unbound super object diff --git a/pypy/module/__builtin__/test/test_descriptor.py b/pypy/module/__builtin__/test/test_descriptor.py --- a/pypy/module/__builtin__/test/test_descriptor.py +++ b/pypy/module/__builtin__/test/test_descriptor.py @@ -242,7 +242,7 @@ c = C() assert C.goo(1) == (C, 1) assert c.goo(1) == (C, 1) - + assert c.foo(1) == (c, 1) class D(C): pass @@ -266,6 +266,17 @@ meth = classmethod(1).__get__(1) raises(TypeError, meth) + def test_super_thisclass(self): + class A(object): + pass + + assert super(A, A()).__thisclass__ is A + + class B(A): + pass + + assert super(B, B()).__thisclass__ is B + assert super(A, B()).__thisclass__ is A def test_property_docstring(self): assert property.__doc__.startswith('property') diff --git a/pypy/module/_collections/app_defaultdict.py b/pypy/module/_collections/app_defaultdict.py --- a/pypy/module/_collections/app_defaultdict.py +++ b/pypy/module/_collections/app_defaultdict.py @@ -12,6 +12,7 @@ class defaultdict(dict): __slots__ = ['default_factory'] + __module__ = 'collections' def __init__(self, *args, **kwds): if len(args) > 0: diff --git a/pypy/module/_collections/test/test_defaultdict.py b/pypy/module/_collections/test/test_defaultdict.py --- a/pypy/module/_collections/test/test_defaultdict.py +++ b/pypy/module/_collections/test/test_defaultdict.py @@ -16,6 +16,12 @@ d[5].append(44) assert l == [42, 43] and l2 == [44] + def test_module(self): + from _collections import defaultdict + assert repr(defaultdict) in ( + "<class 'collections.defaultdict'>", # on PyPy + "<type 'collections.defaultdict'>") # on CPython + def test_keyerror_without_factory(self): from _collections import defaultdict for d1 in [defaultdict(), defaultdict(None)]: diff --git a/pypy/module/_sre/interp_sre.py b/pypy/module/_sre/interp_sre.py --- a/pypy/module/_sre/interp_sre.py +++ b/pypy/module/_sre/interp_sre.py @@ -7,6 +7,7 @@ from pypy.interpreter.error import OperationError from rpython.rlib.rarithmetic import intmask from rpython.rlib import jit +from rpython.rlib.rstring import StringBuilder, UnicodeBuilder # ____________________________________________________________ # @@ -37,6 +38,8 @@ if isinstance(ctx, rsre_core.BufMatchContext): return space.wrapbytes(ctx._buffer.getslice(start, end, 1, end-start)) + if isinstance(ctx, rsre_core.StrMatchContext): + return space.wrapbytes(ctx._string[start:end]) elif isinstance(ctx, rsre_core.UnicodeMatchContext): return space.wrap(ctx._unicodestr[start:end]) else: @@ -98,8 +101,8 @@ space.wrap("cannot copy this pattern object")) def make_ctx(self, w_string, pos=0, endpos=sys.maxint): - """Make a BufMatchContext or a UnicodeMatchContext for searching - in the given w_string object.""" + """Make a StrMatchContext, BufMatchContext or a UnicodeMatchContext for + searching in the given w_string object.""" space = self.space if pos < 0: pos = 0 @@ -117,6 +120,14 @@ endpos = len(unicodestr) return rsre_core.UnicodeMatchContext(self.code, unicodestr, pos, endpos, self.flags) + elif space.isinstance_w(w_string, space.w_str): + str = space.str_w(w_string) + if pos > len(str): + pos = len(str) + if endpos > len(str): + endpos = len(str) + return rsre_core.StrMatchContext(self.code, str, + pos, endpos, self.flags) else: buf = space.readbuf_w(w_string) if (not space.is_none(self.w_pattern) and @@ -224,6 +235,11 @@ def subx(self, w_ptemplate, w_string, count): space = self.space + # use a (much faster) string/unicode builder if w_ptemplate and + # w_string are both string or both unicode objects, and if w_ptemplate + # is a literal + use_builder = False + filter_as_unicode = filter_as_string = None if space.is_true(space.callable(w_ptemplate)): w_filter = w_ptemplate filter_is_callable = True @@ -231,6 +247,8 @@ if space.isinstance_w(w_ptemplate, space.w_unicode): filter_as_unicode = space.unicode_w(w_ptemplate) literal = u'\\' not in filter_as_unicode + use_builder = ( + space.isinstance_w(w_string, space.w_unicode) and literal) else: try: filter_as_string = space.bytes_w(w_ptemplate) @@ -240,6 +258,8 @@ literal = False else: literal = '\\' not in filter_as_string + use_builder = ( + space.isinstance_w(w_string, space.w_str) and literal) if literal: w_filter = w_ptemplate filter_is_callable = False @@ -250,19 +270,44 @@ space.wrap(self), w_ptemplate) filter_is_callable = space.is_true(space.callable(w_filter)) # + # XXX this is a bit of a mess, but it improves performance a lot ctx = self.make_ctx(w_string) - sublist_w = [] + sublist_w = strbuilder = unicodebuilder = None + if use_builder: + if filter_as_unicode is not None: + unicodebuilder = UnicodeBuilder(ctx.end) + else: + assert filter_as_string is not None + strbuilder = StringBuilder(ctx.end) + else: + sublist_w = [] n = last_pos = 0 while not count or n < count: + sub_jitdriver.jit_merge_point( + self=self, + use_builder=use_builder, + filter_is_callable=filter_is_callable, + filter_type=type(w_filter), + ctx=ctx, + w_filter=w_filter, + strbuilder=strbuilder, + unicodebuilder=unicodebuilder, + filter_as_string=filter_as_string, + filter_as_unicode=filter_as_unicode, + count=count, + w_string=w_string, + n=n, last_pos=last_pos, sublist_w=sublist_w + ) + space = self.space if not searchcontext(space, ctx): break if last_pos < ctx.match_start: - sublist_w.append(slice_w(space, ctx, last_pos, - ctx.match_start, space.w_None)) + _sub_append_slice( + ctx, space, use_builder, sublist_w, + strbuilder, unicodebuilder, last_pos, ctx.match_start) start = ctx.match_end if start == ctx.match_start: start += 1 - nextctx = ctx.fresh_copy(start) if not (last_pos == ctx.match_start == ctx.match_end and n > 0): # the above ignores empty matches on latest position @@ -270,28 +315,71 @@ w_match = self.getmatch(ctx, True) w_piece = space.call_function(w_filter, w_match) if not space.is_w(w_piece, space.w_None): + assert strbuilder is None and unicodebuilder is None + assert not use_builder sublist_w.append(w_piece) else: - sublist_w.append(w_filter) + if use_builder: + if strbuilder is not None: + assert filter_as_string is not None + strbuilder.append(filter_as_string) + else: + assert unicodebuilder is not None + assert filter_as_unicode is not None + unicodebuilder.append(filter_as_unicode) + else: + sublist_w.append(w_filter) last_pos = ctx.match_end n += 1 elif last_pos >= ctx.end: break # empty match at the end: finished - ctx = nextctx + ctx.reset(start) if last_pos < ctx.end: - sublist_w.append(slice_w(space, ctx, last_pos, ctx.end, - space.w_None)) + _sub_append_slice(ctx, space, use_builder, sublist_w, + strbuilder, unicodebuilder, last_pos, ctx.end) + if use_builder: + if strbuilder is not None: + return space.wrapbytes(strbuilder.build()), n + else: + assert unicodebuilder is not None + return space.wrap(unicodebuilder.build()), n + else: + if space.isinstance_w(w_string, space.w_unicode): + w_emptystr = space.wrap(u'') + else: + w_emptystr = space.wrapbytes('') + w_item = space.call_method(w_emptystr, 'join', + space.newlist(sublist_w)) + return w_item, n - if space.isinstance_w(w_string, space.w_unicode): - w_emptystr = space.wrap(u'') - else: - w_emptystr = space.wrapbytes('') - w_item = space.call_method(w_emptystr, 'join', - space.newlist(sublist_w)) - return w_item, n +sub_jitdriver = jit.JitDriver( + reds="""count n last_pos + ctx w_filter + strbuilder unicodebuilder + filter_as_string + filter_as_unicode + w_string sublist_w + self""".split(), + greens=["filter_is_callable", "use_builder", "filter_type", "ctx.pattern"]) +def _sub_append_slice(ctx, space, use_builder, sublist_w, + strbuilder, unicodebuilder, start, end): + if use_builder: + if isinstance(ctx, rsre_core.BufMatchContext): + assert strbuilder is not None + return strbuilder.append(ctx._buffer.getslice(start, end, 1, end-start)) + if isinstance(ctx, rsre_core.StrMatchContext): + assert strbuilder is not None + return strbuilder.append_slice(ctx._string, start, end) + elif isinstance(ctx, rsre_core.UnicodeMatchContext): + assert unicodebuilder is not None + return unicodebuilder.append_slice(ctx._unicodestr, start, end) + assert 0, "unreachable" + else: + sublist_w.append(slice_w(space, ctx, start, end, space.w_None)) + @unwrap_spec(flags=int, groups=int, w_groupindex=WrappedDefault(None), w_indexgroup=WrappedDefault(None)) def SRE_Pattern__new__(space, w_subtype, w_pattern, flags, w_code, @@ -494,6 +582,8 @@ ctx = self.ctx if isinstance(ctx, rsre_core.BufMatchContext): return space.wrapbytes(ctx._buffer.as_str()) + elif isinstance(ctx, rsre_core.StrMatchContext): + return space.wrapbytes(ctx._string) elif isinstance(ctx, rsre_core.UnicodeMatchContext): return space.wrap(ctx._unicodestr) else: diff --git a/pypy/module/cpyext/include/patchlevel.h b/pypy/module/cpyext/include/patchlevel.h --- a/pypy/module/cpyext/include/patchlevel.h +++ b/pypy/module/cpyext/include/patchlevel.h @@ -29,8 +29,8 @@ #define PY_VERSION "3.3.5" /* PyPy version as a string */ -#define PYPY_VERSION "4.1.0-alpha0" -#define PYPY_VERSION_NUM 0x04010000 +#define PYPY_VERSION "5.1.0-alpha0" +#define PYPY_VERSION_NUM 0x05010000 /* Defined to mean a PyPy where cpyext holds more regular references to PyObjects, e.g. staying alive as long as the internal PyPy object diff --git a/pypy/module/micronumpy/appbridge.py b/pypy/module/micronumpy/appbridge.py --- a/pypy/module/micronumpy/appbridge.py +++ b/pypy/module/micronumpy/appbridge.py @@ -9,6 +9,7 @@ w_array_repr = None w_array_str = None w__usefields = None + w_partition = None def __init__(self, space): pass diff --git a/pypy/module/micronumpy/concrete.py b/pypy/module/micronumpy/concrete.py --- a/pypy/module/micronumpy/concrete.py +++ b/pypy/module/micronumpy/concrete.py @@ -298,7 +298,14 @@ except IndexError: # not a single result chunks = self._prepare_slice_args(space, w_index) - return new_view(space, orig_arr, chunks) + copy = False + if isinstance(chunks[0], BooleanChunk): + # numpy compatibility + copy = True + w_ret = new_view(space, orig_arr, chunks) + if copy: + w_ret = w_ret.descr_copy(space, space.wrap(w_ret.get_order())) + return w_ret def descr_setitem(self, space, orig_arr, w_index, w_value): try: diff --git a/pypy/module/micronumpy/ndarray.py b/pypy/module/micronumpy/ndarray.py --- a/pypy/module/micronumpy/ndarray.py +++ b/pypy/module/micronumpy/ndarray.py @@ -22,7 +22,8 @@ from pypy.module.micronumpy.flagsobj import W_FlagsObject from pypy.module.micronumpy.strides import ( get_shape_from_iterable, shape_agreement, shape_agreement_multiple, - is_c_contiguous, is_f_contiguous, calc_strides, new_view) + is_c_contiguous, is_f_contiguous, calc_strides, new_view, BooleanChunk, + SliceChunk) from pypy.module.micronumpy.casting import can_cast_array from pypy.module.micronumpy.descriptor import get_dtype_cache @@ -204,7 +205,13 @@ if iter_shape is None: # w_index is a list of slices, return a view chunks = self.implementation._prepare_slice_args(space, w_index) - return new_view(space, self, chunks) + copy = False + if isinstance(chunks[0], BooleanChunk): + copy = True + w_ret = new_view(space, self, chunks) + if copy: + w_ret = w_ret.descr_copy(space, space.wrap(w_ret.get_order())) + return w_ret shape = res_shape + self.get_shape()[len(indexes):] w_res = W_NDimArray.from_shape(space, shape, self.get_dtype(), self.get_order(), w_instance=self) @@ -220,8 +227,24 @@ if iter_shape is None: # w_index is a list of slices chunks = self.implementation._prepare_slice_args(space, w_index) - view = new_view(space, self, chunks) - view.implementation.setslice(space, val_arr) + dim = -1 + view = self + for i, c in enumerate(chunks): + if isinstance(c, BooleanChunk): + dim = i + idx = c.w_idx + chunks.pop(i) + chunks.insert(0, SliceChunk(space.newslice(space.wrap(0), + space.w_None, space.w_None))) + break + if dim > 0: + view = self.implementation.swapaxes(space, self, 0, dim) + if dim >= 0: + view = new_view(space, self, chunks) + view.setitem_filter(space, idx, val_arr) + else: + view = new_view(space, self, chunks) + view.implementation.setslice(space, val_arr) return if support.product(iter_shape) == 0: return @@ -534,8 +557,12 @@ return self.get_scalar_value().item(space) l_w = [] for i in range(self.get_shape()[0]): - l_w.append(space.call_method(self.descr_getitem(space, - space.wrap(i)), "tolist")) + item_w = self.descr_getitem(space, space.wrap(i)) + if (isinstance(item_w, W_NDimArray) or + isinstance(item_w, boxes.W_GenericBox)): + l_w.append(space.call_method(item_w, "tolist")) + else: + l_w.append(item_w) return space.newlist(l_w) def descr_ravel(self, space, w_order=None): @@ -903,6 +930,10 @@ return return self.implementation.sort(space, w_axis, w_order) + def descr_partition(self, space, __args__): + return get_appbridge_cache(space).call_method( + space, 'numpy.core._partition_use', 'partition', __args__.prepend(self)) + def descr_squeeze(self, space, w_axis=None): cur_shape = self.get_shape() if not space.is_none(w_axis): @@ -1616,6 +1647,7 @@ argsort = interp2app(W_NDimArray.descr_argsort), sort = interp2app(W_NDimArray.descr_sort), + partition = interp2app(W_NDimArray.descr_partition), astype = interp2app(W_NDimArray.descr_astype), base = GetSetProperty(W_NDimArray.descr_get_base), byteswap = interp2app(W_NDimArray.descr_byteswap), diff --git a/pypy/module/micronumpy/strides.py b/pypy/module/micronumpy/strides.py --- a/pypy/module/micronumpy/strides.py +++ b/pypy/module/micronumpy/strides.py @@ -97,22 +97,19 @@ # filter by axis dim filtr = chunks[dim] assert isinstance(filtr, BooleanChunk) + # XXX this creates a new array, and fails in setitem w_arr = w_arr.getitem_filter(space, filtr.w_idx, axis=dim) arr = w_arr.implementation chunks[dim] = SliceChunk(space.newslice(space.wrap(0), - space.wrap(-1), space.w_None)) + space.w_None, space.w_None)) r = calculate_slice_strides(space, arr.shape, arr.start, arr.get_strides(), arr.get_backstrides(), chunks) else: r = calculate_slice_strides(space, arr.shape, arr.start, arr.get_strides(), arr.get_backstrides(), chunks) shape, start, strides, backstrides = r - w_ret = W_NDimArray.new_slice(space, start, strides[:], backstrides[:], + return W_NDimArray.new_slice(space, start, strides[:], backstrides[:], shape[:], arr, w_arr) - if dim == 0: - # Do not return a view - return w_ret.descr_copy(space, space.wrap(w_ret.get_order())) - return w_ret @jit.unroll_safe def _extend_shape(old_shape, chunks): diff --git a/pypy/module/micronumpy/test/test_ndarray.py b/pypy/module/micronumpy/test/test_ndarray.py --- a/pypy/module/micronumpy/test/test_ndarray.py +++ b/pypy/module/micronumpy/test/test_ndarray.py @@ -1936,6 +1936,22 @@ a = array([[1, 2], [3, 4]]) assert (a + a).tolist() == [[2, 4], [6, 8]] + def test_tolist_object(self): + from numpy import array + a = array([0], dtype=object) + assert a.tolist() == [0] + + def test_tolist_object_slice(self): + from numpy import array + list_expected = [slice(0, 1), 0] + a = array(list_expected, dtype=object) + assert a.tolist() == list_expected + + def test_tolist_object_slice_2d(self): + from numpy import array + a = array([(slice(0, 1), 1), (0, 1)], dtype=object) + assert a.tolist() == [[slice(0, 1, None), 1], [0, 1]] + def test_tolist_slice(self): from numpy import array a = array([[17.1, 27.2], [40.3, 50.3]]) @@ -2541,8 +2557,10 @@ assert b.base is None b = a[:, np.array([True, False, True])] assert b.base is not None + a[np.array([True, False]), 0] = 100 b = a[np.array([True, False]), 0] - assert (b ==[0]).all() + assert b.shape == (1,) + assert (b ==[100]).all() def test_scalar_indexing(self): import numpy as np diff --git a/pypy/module/pypyjit/test_pypy_c/test_00_model.py b/pypy/module/pypyjit/test_pypy_c/test_00_model.py --- a/pypy/module/pypyjit/test_pypy_c/test_00_model.py +++ b/pypy/module/pypyjit/test_pypy_c/test_00_model.py @@ -68,9 +68,12 @@ pipe.returncode,)) if stderr.startswith('SKIP:'): py.test.skip(stderr) - if stderr.startswith('debug_alloc.h:'): # lldebug builds - stderr = '' + #if stderr.startswith('debug_alloc.h:'): # lldebug builds + # stderr = '' #assert not stderr + if stderr: + print '*** stderr of the subprocess: ***' + print stderr # if discard_stdout_before_last_line: stdout = stdout.splitlines(True)[-1] diff --git a/pypy/module/sys/version.py b/pypy/module/sys/version.py --- a/pypy/module/sys/version.py +++ b/pypy/module/sys/version.py @@ -10,7 +10,7 @@ #XXX # sync CPYTHON_VERSION with patchlevel.h, package.py CPYTHON_API_VERSION = 1013 #XXX # sync with include/modsupport.h -PYPY_VERSION = (4, 1, 0, "alpha", 0) #XXX # sync patchlevel.h +PYPY_VERSION = (5, 1, 0, "alpha", 0) #XXX # sync patchlevel.h import pypy diff --git a/pypy/module/test_lib_pypy/cffi_tests/cffi0/test_zintegration.py b/pypy/module/test_lib_pypy/cffi_tests/cffi0/test_zintegration.py --- a/pypy/module/test_lib_pypy/cffi_tests/cffi0/test_zintegration.py +++ b/pypy/module/test_lib_pypy/cffi_tests/cffi0/test_zintegration.py @@ -12,7 +12,9 @@ def create_venv(name): tmpdir = udir.join(name) try: - subprocess.check_call(['virtualenv', '--distribute', + subprocess.check_call(['virtualenv', + #'--never-download', <= could be added, but causes failures + # in random cases on random machines '-p', os.path.abspath(sys.executable), str(tmpdir)]) except OSError as e: diff --git a/pypy/module/test_lib_pypy/cffi_tests/embedding/add1-test.c b/pypy/module/test_lib_pypy/cffi_tests/embedding/add1-test.c --- a/pypy/module/test_lib_pypy/cffi_tests/embedding/add1-test.c +++ b/pypy/module/test_lib_pypy/cffi_tests/embedding/add1-test.c @@ -1,3 +1,4 @@ +/* Generated by pypy/tool/import_cffi.py */ #include <stdio.h> extern int add1(int, int); diff --git a/pypy/module/test_lib_pypy/cffi_tests/embedding/add2-test.c b/pypy/module/test_lib_pypy/cffi_tests/embedding/add2-test.c --- a/pypy/module/test_lib_pypy/cffi_tests/embedding/add2-test.c +++ b/pypy/module/test_lib_pypy/cffi_tests/embedding/add2-test.c @@ -1,3 +1,4 @@ +/* Generated by pypy/tool/import_cffi.py */ #include <stdio.h> extern int add1(int, int); diff --git a/pypy/module/test_lib_pypy/cffi_tests/embedding/add_recursive-test.c b/pypy/module/test_lib_pypy/cffi_tests/embedding/add_recursive-test.c --- a/pypy/module/test_lib_pypy/cffi_tests/embedding/add_recursive-test.c +++ b/pypy/module/test_lib_pypy/cffi_tests/embedding/add_recursive-test.c @@ -1,3 +1,4 @@ +/* Generated by pypy/tool/import_cffi.py */ #include <stdio.h> #ifdef _MSC_VER diff --git a/pypy/module/test_lib_pypy/cffi_tests/embedding/perf-test.c b/pypy/module/test_lib_pypy/cffi_tests/embedding/perf-test.c --- a/pypy/module/test_lib_pypy/cffi_tests/embedding/perf-test.c +++ b/pypy/module/test_lib_pypy/cffi_tests/embedding/perf-test.c @@ -1,10 +1,12 @@ +/* Generated by pypy/tool/import_cffi.py */ #include <stdio.h> #include <assert.h> #include <sys/time.h> #ifdef PTEST_USE_THREAD # include <pthread.h> -# include <semaphore.h> -static sem_t done; +static pthread_mutex_t mutex1 = PTHREAD_MUTEX_INITIALIZER; +static pthread_cond_t cond1 = PTHREAD_COND_INITIALIZER; +static int remaining; #endif @@ -54,8 +56,11 @@ printf("time per call: %.3g\n", t); #ifdef PTEST_USE_THREAD - int status = sem_post(&done); - assert(status == 0); + pthread_mutex_lock(&mutex1); + remaining -= 1; + if (!remaining) + pthread_cond_signal(&cond1); + pthread_mutex_unlock(&mutex1); #endif return arg; @@ -68,19 +73,19 @@ start_routine(0); #else pthread_t th; - int i, status = sem_init(&done, 0, 0); - assert(status == 0); + int i, status; add1(0, 0); /* this is the main thread */ + remaining = PTEST_USE_THREAD; for (i = 0; i < PTEST_USE_THREAD; i++) { status = pthread_create(&th, NULL, start_routine, NULL); assert(status == 0); } - for (i = 0; i < PTEST_USE_THREAD; i++) { - status = sem_wait(&done); - assert(status == 0); - } + pthread_mutex_lock(&mutex1); + while (remaining) + pthread_cond_wait(&cond1, &mutex1); + pthread_mutex_unlock(&mutex1); #endif return 0; } diff --git a/pypy/module/test_lib_pypy/cffi_tests/embedding/test_basic.py b/pypy/module/test_lib_pypy/cffi_tests/embedding/test_basic.py --- a/pypy/module/test_lib_pypy/cffi_tests/embedding/test_basic.py +++ b/pypy/module/test_lib_pypy/cffi_tests/embedding/test_basic.py @@ -33,8 +33,12 @@ pythonpath.insert(0, cffi_base) return os.pathsep.join(pythonpath) -def setup_module(mod): - mod.org_env = os.environ.copy() +def copy_away_env(): + global org_env + try: + org_env + except NameError: + org_env = os.environ.copy() class EmbeddingTests: @@ -122,6 +126,7 @@ os.chdir(curdir) def patch_environment(self): + copy_away_env() path = self.get_path() # for libpypy-c.dll or Python27.dll path = os.path.split(sys.executable)[0] + os.path.pathsep + path diff --git a/pypy/module/test_lib_pypy/cffi_tests/embedding/thread-test.h b/pypy/module/test_lib_pypy/cffi_tests/embedding/thread-test.h --- a/pypy/module/test_lib_pypy/cffi_tests/embedding/thread-test.h +++ b/pypy/module/test_lib_pypy/cffi_tests/embedding/thread-test.h @@ -1,10 +1,45 @@ +/* Generated by pypy/tool/import_cffi.py */ /************************************************************/ #ifndef _MSC_VER /************************************************************/ #include <pthread.h> -#include <semaphore.h> + +/* don't include <semaphore.h>, it is not available on OS/X */ + +typedef struct { + pthread_mutex_t mutex1; + pthread_cond_t cond1; + unsigned int value; +} sem_t; + +static int sem_init(sem_t *sem, int pshared, unsigned int value) +{ + assert(pshared == 0); + sem->value = value; + return (pthread_mutex_init(&sem->mutex1, NULL) || + pthread_cond_init(&sem->cond1, NULL)); +} + +static int sem_post(sem_t *sem) +{ + pthread_mutex_lock(&sem->mutex1); + sem->value += 1; + pthread_cond_signal(&sem->cond1); + pthread_mutex_unlock(&sem->mutex1); + return 0; +} + +static int sem_wait(sem_t *sem) +{ + pthread_mutex_lock(&sem->mutex1); + while (sem->value == 0) + pthread_cond_wait(&sem->cond1, &sem->mutex1); + sem->value -= 1; + pthread_mutex_unlock(&sem->mutex1); + return 0; +} /************************************************************/ @@ -22,7 +57,7 @@ typedef HANDLE sem_t; typedef HANDLE pthread_t; -int sem_init(sem_t *sem, int pshared, unsigned int value) +static int sem_init(sem_t *sem, int pshared, unsigned int value) { assert(pshared == 0); assert(value == 0); @@ -30,26 +65,26 @@ return *sem ? 0 : -1; } -int sem_post(sem_t *sem) +static int sem_post(sem_t *sem) { return ReleaseSemaphore(*sem, 1, NULL) ? 0 : -1; } -int sem_wait(sem_t *sem) +static int sem_wait(sem_t *sem) { WaitForSingleObject(*sem, INFINITE); return 0; } -DWORD WINAPI myThreadProc(LPVOID lpParameter) +static DWORD WINAPI myThreadProc(LPVOID lpParameter) { void *(* start_routine)(void *) = (void *(*)(void *))lpParameter; start_routine(NULL); return 0; } -int pthread_create(pthread_t *thread, void *attr, - void *start_routine(void *), void *arg) +static int pthread_create(pthread_t *thread, void *attr, + void *start_routine(void *), void *arg) { assert(arg == NULL); *thread = CreateThread(NULL, 0, myThreadProc, start_routine, 0, NULL); diff --git a/pypy/module/test_lib_pypy/cffi_tests/embedding/thread1-test.c b/pypy/module/test_lib_pypy/cffi_tests/embedding/thread1-test.c --- a/pypy/module/test_lib_pypy/cffi_tests/embedding/thread1-test.c +++ b/pypy/module/test_lib_pypy/cffi_tests/embedding/thread1-test.c @@ -1,3 +1,4 @@ +/* Generated by pypy/tool/import_cffi.py */ #include <stdio.h> #include <assert.h> #include "thread-test.h" diff --git a/pypy/module/test_lib_pypy/cffi_tests/embedding/thread2-test.c b/pypy/module/test_lib_pypy/cffi_tests/embedding/thread2-test.c --- a/pypy/module/test_lib_pypy/cffi_tests/embedding/thread2-test.c +++ b/pypy/module/test_lib_pypy/cffi_tests/embedding/thread2-test.c @@ -1,3 +1,4 @@ +/* Generated by pypy/tool/import_cffi.py */ #include <stdio.h> #include <assert.h> #include "thread-test.h" diff --git a/pypy/module/test_lib_pypy/cffi_tests/embedding/thread3-test.c b/pypy/module/test_lib_pypy/cffi_tests/embedding/thread3-test.c --- a/pypy/module/test_lib_pypy/cffi_tests/embedding/thread3-test.c +++ b/pypy/module/test_lib_pypy/cffi_tests/embedding/thread3-test.c @@ -1,3 +1,4 @@ +/* Generated by pypy/tool/import_cffi.py */ #include <stdio.h> #include <assert.h> #include "thread-test.h" diff --git a/pypy/module/test_lib_pypy/cffi_tests/embedding/tlocal-test.c b/pypy/module/test_lib_pypy/cffi_tests/embedding/tlocal-test.c --- a/pypy/module/test_lib_pypy/cffi_tests/embedding/tlocal-test.c +++ b/pypy/module/test_lib_pypy/cffi_tests/embedding/tlocal-test.c @@ -1,3 +1,4 @@ +/* Generated by pypy/tool/import_cffi.py */ #include <stdio.h> #include <assert.h> #include "thread-test.h" diff --git a/pypy/objspace/std/celldict.py b/pypy/objspace/std/celldict.py --- a/pypy/objspace/std/celldict.py +++ b/pypy/objspace/std/celldict.py @@ -64,6 +64,9 @@ def setitem_str(self, w_dict, key, w_value): cell = self.getdictvalue_no_unwrapping(w_dict, key) + return self._setitem_str_cell_known(cell, w_dict, key, w_value) + + def _setitem_str_cell_known(self, cell, w_dict, key, w_value): w_value = write_cell(self.space, cell, w_value) if w_value is None: return @@ -74,10 +77,11 @@ space = self.space if space.is_w(space.type(w_key), space.w_unicode): key = space.str_w(w_key) - w_result = self.getitem_str(w_dict, key) + cell = self.getdictvalue_no_unwrapping(w_dict, key) + w_result = unwrap_cell(self.space, cell) if w_result is not None: return w_result - self.setitem_str(w_dict, key, w_default) + self._setitem_str_cell_known(cell, w_dict, key, w_default) return w_default else: self.switch_to_object_strategy(w_dict) diff --git a/pypy/objspace/std/complexobject.py b/pypy/objspace/std/complexobject.py --- a/pypy/objspace/std/complexobject.py +++ b/pypy/objspace/std/complexobject.py @@ -250,11 +250,12 @@ return None from rpython.rlib.longlong2float import float2longlong from pypy.objspace.std.util import IDTAG_COMPLEX as tag + from pypy.objspace.std.util import IDTAG_SHIFT real = space.float_w(space.getattr(self, space.wrap("real"))) imag = space.float_w(space.getattr(self, space.wrap("imag"))) real_b = rbigint.fromrarith_int(float2longlong(real)) imag_b = rbigint.fromrarith_int(r_ulonglong(float2longlong(imag))) - val = real_b.lshift(64).or_(imag_b).lshift(3).int_or_(tag) + val = real_b.lshift(64).or_(imag_b).lshift(IDTAG_SHIFT).int_or_(tag) return space.newlong_from_rbigint(val) def int(self, space): diff --git a/pypy/objspace/std/floatobject.py b/pypy/objspace/std/floatobject.py --- a/pypy/objspace/std/floatobject.py +++ b/pypy/objspace/std/floatobject.py @@ -170,15 +170,11 @@ return self.floatval def int(self, space): + # this is a speed-up only, for space.int(w_float). if (type(self) is not W_FloatObject and space.is_overloaded(self, space.w_float, '__int__')): return W_Root.int(self, space) - try: - value = ovfcheck_float_to_int(self.floatval) - except OverflowError: - return newlong_from_float(space, self.floatval) - else: - return space.newint(value) + return self.descr_trunc(space) def is_w(self, space, w_other): from rpython.rlib.longlong2float import float2longlong @@ -195,9 +191,10 @@ return None from rpython.rlib.longlong2float import float2longlong from pypy.objspace.std.util import IDTAG_FLOAT as tag + from pypy.objspace.std.util import IDTAG_SHIFT val = float2longlong(space.float_w(self)) b = rbigint.fromrarith_int(val) - b = b.lshift(3).int_or_(tag) + b = b.lshift(IDTAG_SHIFT).int_or_(tag) return space.newlong_from_rbigint(b) def __repr__(self): @@ -417,11 +414,10 @@ return W_FloatObject(a) def descr_trunc(self, space): - whole = math.modf(self.floatval)[1] try: - value = ovfcheck_float_to_int(whole) + value = ovfcheck_float_to_int(self.floatval) except OverflowError: - return newlong_from_float(space, whole) + return newlong_from_float(space, self.floatval) else: return space.newint(value) @@ -656,7 +652,7 @@ __hash__ = interp2app(W_FloatObject.descr_hash), __format__ = interp2app(W_FloatObject.descr_format), __bool__ = interp2app(W_FloatObject.descr_bool), - __int__ = interp2app(W_FloatObject.int), + __int__ = interp2app(W_FloatObject.descr_trunc), __float__ = interp2app(W_FloatObject.descr_float), __trunc__ = interp2app(W_FloatObject.descr_trunc), __neg__ = interp2app(W_FloatObject.descr_neg), diff --git a/pypy/objspace/std/intobject.py b/pypy/objspace/std/intobject.py --- a/pypy/objspace/std/intobject.py +++ b/pypy/objspace/std/intobject.py @@ -28,7 +28,7 @@ from pypy.interpreter.typedef import TypeDef from pypy.objspace.std import newformat from pypy.objspace.std.util import ( - BINARY_OPS, CMP_OPS, COMMUTATIVE_OPS, IDTAG_INT, wrap_parsestringerror) + BINARY_OPS, CMP_OPS, COMMUTATIVE_OPS, IDTAG_INT, IDTAG_SHIFT, wrap_parsestringerror) SENTINEL = object() @@ -54,7 +54,7 @@ if self.user_overridden_class: return None b = space.bigint_w(self) - b = b.lshift(3).int_or_(IDTAG_INT) + b = b.lshift(IDTAG_SHIFT).int_or_(IDTAG_INT) return space.newlong_from_rbigint(b) @staticmethod diff --git a/pypy/objspace/std/longobject.py b/pypy/objspace/std/longobject.py --- a/pypy/objspace/std/longobject.py +++ b/pypy/objspace/std/longobject.py @@ -13,7 +13,8 @@ from pypy.objspace.std import newformat from pypy.objspace.std.intobject import ( HASH_BITS, HASH_MODULUS, W_AbstractIntObject, W_IntObject) -from pypy.objspace.std.util import COMMUTATIVE_OPS +from pypy.objspace.std.util import ( + BINARY_OPS, CMP_OPS, COMMUTATIVE_OPS, IDTAG_LONG, IDTAG_SHIFT, wrap_parsestringerror) def delegate_other(func): diff --git a/pypy/objspace/std/setobject.py b/pypy/objspace/std/setobject.py --- a/pypy/objspace/std/setobject.py +++ b/pypy/objspace/std/setobject.py @@ -1056,7 +1056,7 @@ if self is w_other.strategy: strategy = self if w_set.length() > w_other.length(): - # swap operants + # swap operands storage = self._intersect_unwrapped(w_other, w_set) else: storage = self._intersect_unwrapped(w_set, w_other) @@ -1066,7 +1066,7 @@ else: strategy = self.space.fromcache(ObjectSetStrategy) if w_set.length() > w_other.length(): - # swap operants + # swap operands storage = w_other.strategy._intersect_wrapped(w_other, w_set) else: storage = self._intersect_wrapped(w_set, w_other) diff --git a/pypy/objspace/std/test/test_celldict.py b/pypy/objspace/std/test/test_celldict.py --- a/pypy/objspace/std/test/test_celldict.py +++ b/pypy/objspace/std/test/test_celldict.py @@ -114,22 +114,11 @@ class TestModuleDictImplementation(BaseTestRDictImplementation): StrategyClass = ModuleDictStrategy - -class TestModuleDictImplementationWithBuiltinNames(BaseTestRDictImplementation): - StrategyClass = ModuleDictStrategy - - string = "int" - string2 = "isinstance" - + setdefault_hash_count = 2 class TestDevolvedModuleDictImplementation(BaseTestDevolvedDictImplementation): StrategyClass = ModuleDictStrategy - -class TestDevolvedModuleDictImplementationWithBuiltinNames(BaseTestDevolvedDictImplementation): - StrategyClass = ModuleDictStrategy - - string = "int" - string2 = "isinstance" + setdefault_hash_count = 2 class AppTestCellDict(object): diff --git a/pypy/objspace/std/test/test_dictmultiobject.py b/pypy/objspace/std/test/test_dictmultiobject.py --- a/pypy/objspace/std/test/test_dictmultiobject.py +++ b/pypy/objspace/std/test/test_dictmultiobject.py @@ -1324,6 +1324,9 @@ impl.setitem(x, x) assert type(impl.get_strategy()) is ObjectDictStrategy + + setdefault_hash_count = 1 + def test_setdefault_fast(self): on_pypy = "__pypy__" in sys.builtin_module_names impl = self.impl @@ -1331,11 +1334,11 @@ x = impl.setdefault(key, 1) assert x == 1 if on_pypy and self.FakeString is FakeString: - assert key.hash_count == 1 + assert key.hash_count == self.setdefault_hash_count x = impl.setdefault(key, 2) assert x == 1 if on_pypy and self.FakeString is FakeString: - assert key.hash_count == 2 + assert key.hash_count == self.setdefault_hash_count + 1 def test_fallback_evil_key(self): class F(object): diff --git a/pypy/objspace/std/test/test_kwargsdict.py b/pypy/objspace/std/test/test_kwargsdict.py --- a/pypy/objspace/std/test/test_kwargsdict.py +++ b/pypy/objspace/std/test/test_kwargsdict.py @@ -119,10 +119,16 @@ def test_delitem(self): pass # delitem devolves for now + def test_setdefault_fast(self): + pass # not based on hashing at all + class TestDevolvedKwargsDictImplementation(BaseTestDevolvedDictImplementation): get_impl = get_impl StrategyClass = KwargsDictStrategy + def test_setdefault_fast(self): + pass # not based on hashing at all + class AppTestKwargsDictStrategy(object): def setup_class(cls): diff --git a/pypy/objspace/std/test/test_obj.py b/pypy/objspace/std/test/test_obj.py --- a/pypy/objspace/std/test/test_obj.py +++ b/pypy/objspace/std/test/test_obj.py @@ -170,14 +170,14 @@ def test_id_on_primitives(self): if self.cpython_apptest: skip("cpython behaves differently") - assert id(1) == (1 << 3) + 1 + assert id(1) == (1 << 4) + 1 class myint(int): pass assert id(myint(1)) != id(1) assert id(1.0) & 7 == 5 assert id(-0.0) != id(0.0) - assert hex(id(2.0)) == '0x20000000000000005' + assert hex(id(2.0)) == '0x40000000000000005' assert id(0.0) == 5 def test_id_on_strs(self): diff --git a/pypy/objspace/std/util.py b/pypy/objspace/std/util.py --- a/pypy/objspace/std/util.py +++ b/pypy/objspace/std/util.py @@ -3,11 +3,13 @@ from pypy.interpreter.error import OperationError from pypy.interpreter import gateway +IDTAG_SHIFT = 4 IDTAG_INT = 1 IDTAG_LONG = 3 IDTAG_FLOAT = 5 IDTAG_COMPLEX = 7 +IDTAG_METHOD = 9 CMP_OPS = dict(lt='<', le='<=', eq='==', ne='!=', gt='>', ge='>=') BINARY_BITWISE_OPS = {'and': '&', 'lshift': '<<', 'or': '|', 'rshift': '>>', diff --git a/pypy/tool/import_cffi.py b/pypy/tool/import_cffi.py --- a/pypy/tool/import_cffi.py +++ b/pypy/tool/import_cffi.py @@ -7,11 +7,18 @@ import sys, py -def mangle(lines): - yield "# Generated by pypy/tool/import_cffi.py\n" - for line in lines: - line = line.replace('from testing', 'from pypy.module.test_lib_pypy.cffi_tests') - yield line +def mangle(lines, ext): + if ext == '.py': + yield "# Generated by pypy/tool/import_cffi.py\n" + for line in lines: + line = line.replace('from testing', 'from pypy.module.test_lib_pypy.cffi_tests') + yield line + elif ext in ('.c', '.h'): + yield "/* Generated by pypy/tool/import_cffi.py */\n" + for line in lines: + yield line + else: + raise AssertionError(ext) def main(cffi_dir): cffi_dir = py.path.local(cffi_dir) @@ -23,10 +30,12 @@ for p in (list(cffi_dir.join('cffi').visit(fil='*.py')) + list(cffi_dir.join('cffi').visit(fil='*.h'))): cffi_dest.join('..', p.relto(cffi_dir)).write(p.read()) - for p in cffi_dir.join('testing').visit(fil='*.py'): + for p in (list(cffi_dir.join('testing').visit(fil='*.py')) + + list(cffi_dir.join('testing').visit(fil='*.h')) + + list(cffi_dir.join('testing').visit(fil='*.c'))): path = test_dest.join(p.relto(cffi_dir.join('testing'))) path.join('..').ensure(dir=1) - path.write(''.join(mangle(p.readlines()))) + path.write(''.join(mangle(p.readlines(), p.ext))) if __name__ == '__main__': if len(sys.argv) != 2: diff --git a/pypy/tool/release/force-builds.py b/pypy/tool/release/force-builds.py --- a/pypy/tool/release/force-builds.py +++ b/pypy/tool/release/force-builds.py @@ -9,7 +9,7 @@ modified by PyPy team """ -import os, sys, urllib +import os, sys, urllib, subprocess from twisted.internet import reactor, defer from twisted.python import log @@ -83,4 +83,9 @@ (options, args) = parser.parse_args() if not options.branch: parser.error("branch option required") + try: + subprocess.check_call(['hg','id','-r', options.branch]) + except subprocess.CalledProcessError: + print 'branch', options.branch, 'could not be found in local repository' + sys.exit(-1) main(options.branch, options.server, user=options.user) diff --git a/rpython/doc/translation.rst b/rpython/doc/translation.rst --- a/rpython/doc/translation.rst +++ b/rpython/doc/translation.rst @@ -33,7 +33,7 @@ The RPython translation toolchain never sees Python source code or syntax trees, but rather starts with the *code objects* that define the behaviour of the function objects one gives it as input. The -`bytecode evaluator`_ and the :ref:`flow graph builder<flow-graphs>` work through these +:ref:`flow graph builder<flow-graphs>` works through these code objects using `abstract interpretation`_ to produce a control flow graph (one per function): yet another representation of the source program, but one which is suitable for applying type inference @@ -85,7 +85,6 @@ .. _PDF color version: _static/translation.pdf -.. _bytecode evaluator: interpreter.html .. _abstract interpretation: http://en.wikipedia.org/wiki/Abstract_interpretation diff --git a/rpython/jit/backend/test/runner_test.py b/rpython/jit/backend/test/runner_test.py --- a/rpython/jit/backend/test/runner_test.py +++ b/rpython/jit/backend/test/runner_test.py @@ -548,7 +548,9 @@ if cpu.supports_floats: def func(f0, f1, f2, f3, f4, f5, f6, i0, f7, i1, f8, f9): + seen.append((f0, f1, f2, f3, f4, f5, f6, i0, f7, i1, f8, f9)) return f0 + f1 + f2 + f3 + f4 + f5 + f6 + float(i0 + i1) + f7 + f8 + f9 + seen = [] F = lltype.Float I = lltype.Signed FUNC = self.FuncType([F] * 7 + [I] + [F] + [I] + [F]* 2, F) @@ -557,13 +559,15 @@ calldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT, EffectInfo.MOST_GENERAL) funcbox = self.get_funcbox(cpu, func_ptr) - args = ([boxfloat(.1) for i in range(7)] + - [InputArgInt(1), boxfloat(.2), InputArgInt(2), boxfloat(.3), - boxfloat(.4)]) + args = ([boxfloat(.0), boxfloat(.1), boxfloat(.2), boxfloat(.3), + boxfloat(.4), boxfloat(.5), boxfloat(.6), + InputArgInt(1), boxfloat(.7), InputArgInt(2), boxfloat(.8), + boxfloat(.9)]) res = self.execute_operation(rop.CALL_F, [funcbox] + args, _______________________________________________ pypy-commit mailing list pypy-commit@python.org https://mail.python.org/mailman/listinfo/pypy-commit