Author: Ronan Lamy <ronan.l...@gmail.com>
Branch: py3.5
Changeset: r90683:afbf09453369
Date: 2017-03-14 14:26 +0000
http://bitbucket.org/pypy/pypy/changeset/afbf09453369/
Log: hg merge default
diff --git a/pypy/doc/index-of-release-notes.rst
b/pypy/doc/index-of-release-notes.rst
--- a/pypy/doc/index-of-release-notes.rst
+++ b/pypy/doc/index-of-release-notes.rst
@@ -6,6 +6,7 @@
.. toctree::
+ release-v5.7.0.rst
release-pypy2.7-v5.6.0.rst
release-pypy2.7-v5.4.1.rst
release-pypy2.7-v5.4.0.rst
@@ -53,6 +54,12 @@
release-0.7.0.rst
release-0.6
+CPython 3.5 compatible versions
+-------------------------------
+
+.. toctree::
+
+ release-v5.7.0.rst
CPython 3.3 compatible versions
-------------------------------
diff --git a/pypy/doc/index-of-whatsnew.rst b/pypy/doc/index-of-whatsnew.rst
--- a/pypy/doc/index-of-whatsnew.rst
+++ b/pypy/doc/index-of-whatsnew.rst
@@ -7,6 +7,7 @@
.. toctree::
whatsnew-head.rst
+ whatsnew-pypy2-5.7.0.rst
whatsnew-pypy2-5.6.0.rst
whatsnew-pypy2-5.4.0.rst
whatsnew-pypy2-5.3.1.rst
diff --git a/pypy/doc/whatsnew-head.rst b/pypy/doc/whatsnew-head.rst
--- a/pypy/doc/whatsnew-head.rst
+++ b/pypy/doc/whatsnew-head.rst
@@ -1,204 +1,8 @@
==========================
-What's new in PyPy2.7 5.6+
+What's new in PyPy2.7 5.8+
==========================
-.. this is a revision shortly after release-pypy2.7-v5.6
-.. startrev: 7e9787939641
+.. this is a revision shortly after release-pypy2.7-v5.7
+.. startrev: 44f31f6dd39f
-Since a while now, PyPy preserves the order of dictionaries and sets.
-However, the set literal syntax ``{x, y, z}`` would by mistake build a
-set with the opposite order: ``set([z, y, x])``. This has been fixed.
-Note that CPython is inconsistent too: in 2.7.12, ``{5, 5.0}`` would be
-``set([5.0])``, but in 2.7.trunk it is ``set([5])``. PyPy's behavior
-changed in exactly the same way because of this fix.
-
-
-.. branch: mappingproxy
-.. branch: py3k-finish_time
-.. branch: py3k-kwonly-builtin
-.. branch: py3k_add_terminal_size
-.. branch: testing-cleanup-py3k
-
-.. branch: rpython-resync
-Backport rpython changes made directly on the py3k and py3.5 branches.
-
-.. branch: rpython-error-to-systemerror
-
-Any uncaught RPython exception (from a PyPy bug) is turned into an
-app-level SystemError. This should improve the lot of users hitting an
-uncaught RPython error.
-
-.. branch: union-side-effects-2
-
-Try to improve the consistency of RPython annotation unions.
-
-.. branch: pytest-2.9.2
-
-.. branch: clean-exported-state
-
-Clean-ups in the jit optimizeopt
-
-.. branch: conditional_call_value_4
-
-Add jit.conditional_call_elidable(), a way to tell the JIT "conditonally
-call this function" returning a result.
-
-.. branch: desc-specialize
-
-Refactor FunctionDesc.specialize() and related code (RPython annotator).
-
-.. branch: raw-calloc
-
-.. branch: issue2446
-
-Assign ``tp_doc`` to the new TypeObject's type dictionary ``__doc__`` key
-so it will be picked up by app-level objects of that type
-
-.. branch: cling-support
-
-Module cppyy now uses cling as its backend (Reflex has been removed). The
-user-facing interface and main developer tools (genreflex, selection files,
-class loader, etc.) remain the same. A libcppyy_backend.so library is still
-needed but is now available through PyPI with pip: PyPy-cppyy-backend.
-
-The Cling-backend brings support for modern C++ (11, 14, etc.), dynamic
-template instantations, and improved integration with CFFI for better
-performance. It also provides interactive C++ (and bindings to that).
-
-.. branch: better-PyDict_Next
-
-Improve the performance of ``PyDict_Next``. When trying ``PyDict_Next`` on a
-typedef dict, the test exposed a problem converting a ``GetSetProperty`` to a
-``PyGetSetDescrObject``. The other direction seem to be fully implemented.
-This branch made a minimal effort to convert the basic fields to avoid
-segfaults, but trying to use the ``PyGetSetDescrObject`` will probably fail.
-
-.. branch: stdlib-2.7.13
-
-Updated the implementation to match CPython 2.7.13 instead of 2.7.13.
-
-.. branch: issue2444
-
-Fix ``PyObject_GetBuffer`` and ``PyMemoryView_GET_BUFFER``, which leaked
-memory and held references. Add a finalizer to CPyBuffer, add a
-PyMemoryViewObject with a PyBuffer attached so that the call to
-``PyMemoryView_GET_BUFFER`` does not leak a PyBuffer-sized piece of memory.
-Properly call ``bf_releasebuffer`` when not ``NULL``.
-
-.. branch: boehm-rawrefcount
-
-Support translations of cpyext with the Boehm GC (for special cases like
-revdb).
-
-.. branch: strbuf-as-buffer
-
-Implement StringBuffer.get_raw_address (missing feature for the buffer
protocol).
-More generally it is now possible to obtain the address of any object (if it
-is readonly) without pinning it.
-
-.. branch: cpyext-cleanup
-.. branch: api_func-refactor
-
-Refactor cpyext initialisation.
-
-.. branch: cpyext-from2
-
-Fix a test failure introduced by strbuf-as-buffer
-
-.. branch: cpyext-FromBuffer
-
-Do not recreate the object in PyMemoryView_FromBuffer, rather pass it to
-the returned PyMemoryViewObject, to take ownership of it. Fixes a ref leak.
-
-.. branch: issue2464
-
-Give (almost?) all GetSetProperties a valid __objclass__.
-
-.. branch: TreeStain/fixed-typo-line-29-mostly-to-most-1484469416419
-.. branch: TreeStain/main-lines-changed-in-l77-l83-made-para-1484471558033
-
-.. branch: missing-tp_new
-
-Improve mixing app-level classes in c-extensions, especially if the app-level
-class has a ``tp_new`` or ``tp_dealloc``. The issue is that c-extensions expect
-all the method slots to be filled with a function pointer, where app-level will
-search up the mro for an appropriate function at runtime. With this branch we
-now fill many more slots in the c-extenion type objects.
-Also fix for c-extension type that calls ``tp_hash`` during initialization
-(str, unicode types), and fix instantiating c-extension types from built-in
-classes by enforcing an order of instaniation.
-
-.. branch: rffi-parser-2
-
-rffi structures in cpyext can now be created by parsing simple C headers.
-Additionally, the cts object that holds the parsed information can act like
-cffi's ffi objects, with the methods cts.cast() and cts.gettype().
-
-.. branch: rpython-hash
-
-Don't freeze hashes in the translated pypy. In practice, that means
-that we can now translate PyPy with the option --hash=siphash24 and get
-the same hashes as CPython 3.5, which can be randomized (in a
-crypographically good way). It is the default in PyPy3. The default of
-PyPy2 remains unchanged: there are user programs out there that depend
-on constant hashes (or even sometimes on specific hash results).
-
-.. branch: dict-move-to-end
-
-Our dicts, which are always ordered, now have an extra "method" for
-Python 3.x which moves an item to first or last position. In PyPy 3.5
-it is the standard ``OrderedDict.move_to_end()`` method, but the
-behavior is also available on Python 2.x or for the ``dict`` type by
-calling ``__pypy__.move_to_end(dict, key, last=True)``.
-
-
-.. branch optinfo-into-bridges-3
-
-Improve the optimization of branchy Python code by retaining more information
-across failing guards.
-
-
-.. branch: space-newtext
-
-Internal refactoring of ``space.wrap()``, which is now replaced with
-explicitly-typed methods. Notably, there are now ``space.newbytes()``
-and ``space.newtext()``: these two methods are identical on PyPy 2.7 but
-not on PyPy 3.x. The latter is used to get an app-level unicode string
-by decoding the RPython string, assumed to be utf-8.
-
-.. branch: space-wrap
-
-.. branch: fix_bool_restype
-
-Fix for ``ctypes.c_bool``-returning ctypes functions
-
-.. branch: py3.5-text-utf8
-
-space.text_w now encodes to utf-8 not preserving surrogates.
-
-.. branch: fix-cpyext-releasebuffer
-
-Improve handling of the Py3-style buffer slots in cpyext: fix memoryviews
-keeping objects alive forever (missing decref), and make sure that
-bf_releasebuffer is called when it should, e.g. from PyBuffer_Release.
-
-.. branch: fix-global
-
-Fix bug (bad reported info) when asked to translate SyntaxWarning to
-SyntaxError.
-
-.. branch: optinfo-into-bridges-3
-
-Improve the optimization of branchy Python code by retaining more
-information across failing guards. This is done by appending some
-carefully encoded extra information into the resume code.
-
-.. branch: shadowstack-perf-2
-
-Two changes that together bring the performance of shadowstack close to
-asmgcc---close enough that we can now make shadowstack the default even
-on Linux. This should remove a whole class of rare bugs introduced by
-asmgcc.
-
-.. branch: fniephaus/fix-typo-1488123166752
diff --git a/pypy/doc/whatsnew-head.rst b/pypy/doc/whatsnew-pypy2-5.7.0.rst
copy from pypy/doc/whatsnew-head.rst
copy to pypy/doc/whatsnew-pypy2-5.7.0.rst
--- a/pypy/doc/whatsnew-head.rst
+++ b/pypy/doc/whatsnew-pypy2-5.7.0.rst
@@ -1,6 +1,6 @@
-==========================
-What's new in PyPy2.7 5.6+
-==========================
+=========================
+What's new in PyPy2.7 5.7
+=========================
.. this is a revision shortly after release-pypy2.7-v5.6
.. startrev: 7e9787939641
diff --git a/pypy/module/_cffi_backend/test/test_recompiler.py
b/pypy/module/_cffi_backend/test/test_recompiler.py
--- a/pypy/module/_cffi_backend/test/test_recompiler.py
+++ b/pypy/module/_cffi_backend/test/test_recompiler.py
@@ -1866,7 +1866,7 @@
def test_call_with_nested_anonymous_struct(self):
import sys
if sys.platform == 'win32':
- py.test.skip("needs a GCC extension")
+ skip("needs a GCC extension")
ffi, lib = self.prepare("""
struct foo { int a; union { int b, c; }; };
struct foo f(void);
@@ -1914,6 +1914,9 @@
"set_source() and not taking a final '...' argument)")
def test_call_with_zero_length_field(self):
+ import sys
+ if sys.platform == 'win32':
+ skip("zero-length field not supported by MSVC")
ffi, lib = self.prepare("""
struct foo { int a; int x[0]; };
struct foo f(void);
@@ -1959,7 +1962,7 @@
def test_call_with_packed_struct(self):
import sys
if sys.platform == 'win32':
- py.test.skip("needs a GCC extension")
+ skip("needs a GCC extension")
ffi, lib = self.prepare("""
struct foo { char y; int x; };
struct foo f(void);
diff --git a/pypy/module/_rawffi/alt/test/test_funcptr.py
b/pypy/module/_rawffi/alt/test/test_funcptr.py
--- a/pypy/module/_rawffi/alt/test/test_funcptr.py
+++ b/pypy/module/_rawffi/alt/test/test_funcptr.py
@@ -32,7 +32,10 @@
#
c_file.write(py.code.Source('\n'.join(snippets)))
eci = ExternalCompilationInfo(include_dirs=[cdir])
- return str(platform.compile([c_file], eci, 'x', standalone=False))
+ # Windows note: can't reuse the same file name 'x.dll', because
+ # the previous one is likely still opened
+ return str(platform.compile([c_file], eci, 'x' + cls.__name__,
+ standalone=False))
def setup_class(cls):
space = cls.space
diff --git a/pypy/module/cpyext/src/pymem.c b/pypy/module/cpyext/src/pymem.c
--- a/pypy/module/cpyext/src/pymem.c
+++ b/pypy/module/cpyext/src/pymem.c
@@ -1,5 +1,14 @@
+#ifdef _WIN32
+# define _WIN32_WINNT 0x0501
+#endif
+
#include <Python.h>
+#ifdef _WIN32
+# include <Windows.h>
+#endif
+
+
void *
PyMem_RawMalloc(size_t size)
{
@@ -120,10 +129,21 @@
_PyPyGC_AddMemoryPressure(report);
PyGILState_Release(state);
}
+
+ /* Should we return -2 or 0? In theory it should be -2, because
+ we're not using the info to really track the allocations.
+ But I'm sure someone is too clever somewhere and stops calling
+ _PyTraceMalloc_Track() if it returns -2. On the other hand,
+ returning 0 might lead to expectations that importing
+ 'tracemalloc' works on Python 3. Oh well, in that case we'll
+ just crash with ImportError during 'import tracemalloc'.
+ */
+ return 0;
}
int _PyTraceMalloc_Untrack(_PyTraceMalloc_domain_t domain,
uintptr_t ptr)
{
- /* nothing */
+ /* nothing to do */
+ return 0;
}
diff --git a/pypy/module/cpyext/test/test_pystate.py
b/pypy/module/cpyext/test/test_pystate.py
--- a/pypy/module/cpyext/test/test_pystate.py
+++ b/pypy/module/cpyext/test/test_pystate.py
@@ -71,6 +71,7 @@
("dance", "METH_NOARGS",
"""
PyThreadState *old_tstate, *new_tstate;
+ PyObject *d;
PyEval_InitThreads();
@@ -79,7 +80,7 @@
return PyLong_FromLong(0);
}
- PyObject* d = PyThreadState_GetDict(); /* fails on
cpython */
+ d = PyThreadState_GetDict(); /* fails on cpython */
if (d != NULL) {
return PyLong_FromLong(1);
}
@@ -142,6 +143,9 @@
("bounce", "METH_NOARGS",
"""
PyThreadState * tstate;
+ PyObject *dict;
+ PyGILState_STATE gilstate;
+
if (PyEval_ThreadsInitialized() == 0)
{
PyEval_InitThreads();
@@ -150,11 +154,11 @@
if (tstate == NULL) {
return PyLong_FromLong(0);
}
- PyObject* dict = PyThreadState_GetDict();
+ dict = PyThreadState_GetDict();
if (dict != NULL) {
return PyLong_FromLong(1);
}
- PyGILState_STATE gilstate = PyGILState_Ensure();
+ gilstate = PyGILState_Ensure();
dict = PyThreadState_GetDict();
if (dict == NULL) {
return PyLong_FromLong(2);
diff --git a/pypy/module/cpyext/test/test_tupleobject.py
b/pypy/module/cpyext/test/test_tupleobject.py
--- a/pypy/module/cpyext/test/test_tupleobject.py
+++ b/pypy/module/cpyext/test/test_tupleobject.py
@@ -154,5 +154,10 @@
def test_tuple_subclass(self):
module = self.import_module(name='foo')
- a = module.TupleLike([1, 2, 3])
+ a = module.TupleLike(range(100, 400, 100))
assert module.is_TupleLike(a) == 1
+ assert isinstance(a, tuple)
+ assert issubclass(type(a), tuple)
+ assert list(a) == range(100, 400, 100)
+ assert list(a) == range(100, 400, 100)
+ assert list(a) == range(100, 400, 100)
diff --git a/pypy/module/cpyext/test/test_typeobject.py
b/pypy/module/cpyext/test/test_typeobject.py
--- a/pypy/module/cpyext/test/test_typeobject.py
+++ b/pypy/module/cpyext/test/test_typeobject.py
@@ -947,12 +947,15 @@
pass
class bar(f1, f2):
pass
+ class foo(f2, f1):
+ pass
assert bar.__base__ is f2
# On cpython, the size changes.
if '__pypy__' in sys.builtin_module_names:
assert module.size_of_instances(bar) == size
else:
assert module.size_of_instances(bar) >= size
+ assert module.size_of_instances(foo) == module.size_of_instances(bar)
def test_app_cant_subclass_two_types(self):
import sys
@@ -1176,4 +1179,21 @@
assert type(obj).__doc__ == "The Base12 type or object"
assert obj.__doc__ == "The Base12 type or object"
-
+ def test_multiple_inheritance_fetch_tp_bases(self):
+ module = self.import_extension('foo', [
+ ("foo", "METH_O",
+ '''
+ PyTypeObject *tp;
+ tp = (PyTypeObject*)args;
+ Py_INCREF(tp->tp_bases);
+ return tp->tp_bases;
+ '''
+ )])
+ class A(object):
+ pass
+ class B(object):
+ pass
+ class C(A, B):
+ pass
+ bases = module.foo(C)
+ assert bases == (A, B)
diff --git a/pypy/module/cpyext/test/test_userslots.py
b/pypy/module/cpyext/test/test_userslots.py
--- a/pypy/module/cpyext/test/test_userslots.py
+++ b/pypy/module/cpyext/test/test_userslots.py
@@ -160,8 +160,9 @@
};
''', more_init='''
PyObject * mod1 = PyImport_ImportModule("datetime");
+ PyObject * dt;
if (mod1 == NULL) INITERROR;
- PyObject * dt = PyUnicode_FromString("datetime");
+ dt = PyUnicode_FromString("datetime");
datetime_cls = (PyTypeObject*)PyObject_GetAttr(mod1, dt);
if (datetime_cls == NULL) INITERROR;
_Timestamp.tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE;
diff --git a/pypy/module/cpyext/typeobject.py b/pypy/module/cpyext/typeobject.py
--- a/pypy/module/cpyext/typeobject.py
+++ b/pypy/module/cpyext/typeobject.py
@@ -417,8 +417,8 @@
def inherit_special(space, pto, base_pto):
# XXX missing: copy basicsize and flags in a magical way
- # (minimally, if tp_basicsize is zero we copy it from the base)
- if not pto.c_tp_basicsize:
+ # (minimally, if tp_basicsize is zero or too low, we copy it from the base)
+ if pto.c_tp_basicsize < base_pto.c_tp_basicsize:
pto.c_tp_basicsize = base_pto.c_tp_basicsize
if pto.c_tp_itemsize < base_pto.c_tp_itemsize:
pto.c_tp_itemsize = base_pto.c_tp_itemsize
@@ -672,7 +672,7 @@
if builder.cpyext_type_init is not None:
builder.cpyext_type_init.append((pto, w_type))
else:
- finish_type_1(space, pto)
+ finish_type_1(space, pto, w_type.bases_w)
finish_type_2(space, pto, w_type)
pto.c_tp_basicsize = rffi.sizeof(typedescr.basestruct)
@@ -804,7 +804,7 @@
return w_obj
-def finish_type_1(space, pto):
+def finish_type_1(space, pto, bases_w=None):
"""
Sets up tp_bases, necessary before creating the interpreter type.
"""
@@ -816,11 +816,12 @@
if base and not pto.c_ob_type: # will be filled later
pto.c_ob_type = base.c_ob_type
if not pto.c_tp_bases:
- if not base:
- bases = space.newtuple([])
- else:
- bases = space.newtuple([from_ref(space, base_pyo)])
- pto.c_tp_bases = make_ref(space, bases)
+ if bases_w is None:
+ if not base:
+ bases_w = []
+ else:
+ bases_w = [from_ref(space, base_pyo)]
+ pto.c_tp_bases = make_ref(space, space.newtuple(bases_w))
def finish_type_2(space, pto, w_obj):
"""
diff --git a/pypy/module/test_lib_pypy/cffi_tests/cffi0/backend_tests.py
b/pypy/module/test_lib_pypy/cffi_tests/cffi0/backend_tests.py
--- a/pypy/module/test_lib_pypy/cffi_tests/cffi0/backend_tests.py
+++ b/pypy/module/test_lib_pypy/cffi_tests/cffi0/backend_tests.py
@@ -1231,7 +1231,8 @@
def test_ffi_buffer_comparisons(self):
ffi = FFI(backend=self.Backend())
ba = bytearray(range(100, 110))
- assert ba == memoryview(ba) # justification for the following
+ if sys.version_info >= (2, 7):
+ assert ba == memoryview(ba) # justification for the following
a = ffi.new("uint8_t[]", list(ba))
c = ffi.new("uint8_t[]", [99] + list(ba))
try:
diff --git a/pypy/module/test_lib_pypy/cffi_tests/cffi1/test_recompiler.py
b/pypy/module/test_lib_pypy/cffi_tests/cffi1/test_recompiler.py
--- a/pypy/module/test_lib_pypy/cffi_tests/cffi1/test_recompiler.py
+++ b/pypy/module/test_lib_pypy/cffi_tests/cffi1/test_recompiler.py
@@ -2040,7 +2040,7 @@
struct foo s = { 40, 200 };
return s;
}
- struct foo g(int a, ...) { }
+ struct foo g(int a, ...) { return f(); }
""")
assert lib.f().x == 200
e = py.test.raises(NotImplementedError, lib.g, 0)
@@ -2069,7 +2069,7 @@
s.b = 200;
return s;
}
- struct foo g(int a, ...) { }
+ struct foo g(int a, ...) { return f(); }
""")
assert lib.f().b == 200
e = py.test.raises(NotImplementedError, lib.g, 0)
@@ -2095,7 +2095,7 @@
struct foo s = { 11 };
return s;
}
- struct foo g(int a, ...) { }
+ struct foo g(int a, ...) { return f(); }
""")
assert lib.f().x == 11
e = py.test.raises(NotImplementedError, lib.g, 0)
@@ -2107,6 +2107,8 @@
"set_source() and not taking a final '...' argument)")
def test_call_with_zero_length_field():
+ if sys.platform == 'win32':
+ py.test.skip("zero-length field not supported by MSVC")
ffi = FFI()
ffi.cdef("""
struct foo { int a; int x[0]; };
@@ -2119,7 +2121,7 @@
struct foo s = { 42 };
return s;
}
- struct foo g(int a, ...) { }
+ struct foo g(int a, ...) { return f(); }
""")
assert lib.f().a == 42
e = py.test.raises(NotImplementedError, lib.g, 0)
@@ -2143,7 +2145,7 @@
union foo s = { 42 };
return s;
}
- union foo g(int a, ...) { }
+ union foo g(int a, ...) { return f(); }
""")
assert lib.f().a == 42
e = py.test.raises(NotImplementedError, lib.g, 0)
diff --git a/rpython/config/translationoption.py
b/rpython/config/translationoption.py
--- a/rpython/config/translationoption.py
+++ b/rpython/config/translationoption.py
@@ -26,10 +26,15 @@
SUPPORT__THREAD = ( # whether the particular C compiler supports __thread
sys.platform.startswith("linux") or # Linux works
- sys.platform.startswith("darwin")) # OS/X >= 10.7 works
+ #sys.platform.startswith("darwin") or # OS/X >= 10.7 works (*)
+ False)
# Windows doesn't work. Please
# add other platforms here if it works on them.
+# (*) NOTE: __thread on OS/X does not work together with
+# pthread_key_create(): when the destructor is called, the __thread is
+# already freed!
+
MAINDIR = os.path.dirname(os.path.dirname(__file__))
CACHE_DIR = os.path.realpath(os.path.join(MAINDIR, '_cache'))
diff --git a/rpython/jit/backend/x86/assembler.py
b/rpython/jit/backend/x86/assembler.py
--- a/rpython/jit/backend/x86/assembler.py
+++ b/rpython/jit/backend/x86/assembler.py
@@ -349,15 +349,12 @@
#
mc.MOV(eax, heap(self.cpu.pos_exception()))
mc.TEST_rr(eax.value, eax.value)
- mc.J_il8(rx86.Conditions['NZ'], 0)
- jnz_location = mc.get_relative_pos()
+ jnz_location = mc.emit_forward_jump('NZ')
#
mc.RET()
#
# patch the JNZ above
- offset = mc.get_relative_pos() - jnz_location
- assert 0 < offset <= 127
- mc.overwrite(jnz_location-1, chr(offset))
+ mc.patch_forward_jump(jnz_location)
# From now on this function is basically "merged" with
# its caller and so contains DEFAULT_FRAME_BYTES bytes
# plus my own return address, which we'll ignore next
@@ -834,16 +831,13 @@
ofs = self.cpu.unpack_fielddescr(descrs.arraydescr.lendescr)
mc.CMP_bi(ofs, 0xffffff) # force writing 32 bit
stack_check_cmp_ofs = mc.get_relative_pos() - 4
- mc.J_il8(rx86.Conditions['GE'], 0)
- jg_location = mc.get_relative_pos()
+ jg_location = mc.emit_forward_jump('GE')
mc.MOV_si(WORD, 0xffffff) # force writing 32 bit
ofs2 = mc.get_relative_pos() - 4
self.push_gcmap(mc, gcmap, store=True)
mc.CALL(imm(self._frame_realloc_slowpath))
# patch the JG above
- offset = mc.get_relative_pos() - jg_location
- assert 0 < offset <= 127
- mc.overwrite(jg_location-1, chr(offset))
+ mc.patch_forward_jump(jg_location)
self.frame_depth_to_patch.append(stack_check_cmp_ofs)
self.frame_depth_to_patch.append(ofs2)
@@ -857,16 +851,13 @@
ofs = self.cpu.unpack_fielddescr(descrs.arraydescr.lendescr)
mc.CMP_bi(ofs, 0xffffff)
stack_check_cmp_ofs = mc.get_relative_pos() - 4
- mc.J_il8(rx86.Conditions['GE'], 0)
- jg_location = mc.get_relative_pos()
+ jg_location = mc.emit_forward_jump('GE')
mc.MOV_rr(edi.value, ebp.value)
mc.MOV_ri(esi.value, 0xffffff)
ofs2 = mc.get_relative_pos() - 4
mc.CALL(imm(self.cpu.realloc_frame_crash))
# patch the JG above
- offset = mc.get_relative_pos() - jg_location
- assert 0 < offset <= 127
- mc.overwrite(jg_location-1, chr(offset))
+ mc.patch_forward_jump(jg_location)
self.frame_depth_to_patch.append(stack_check_cmp_ofs)
self.frame_depth_to_patch.append(ofs2)
@@ -1002,13 +993,10 @@
self.mc.MOV(eax, heap(endaddr)) # MOV eax, [start]
self.mc.SUB(eax, esp) # SUB eax, current
self.mc.CMP(eax, heap(lengthaddr)) # CMP eax, [length]
- self.mc.J_il8(rx86.Conditions['BE'], 0) # JBE .skip
- jb_location = self.mc.get_relative_pos()
+ jb_location = self.mc.emit_forward_jump('BE')#JBE .skip
self.mc.CALL(imm(self.stack_check_slowpath))# CALL slowpath
# patch the JB above # .skip:
- offset = self.mc.get_relative_pos() - jb_location
- assert 0 < offset <= 127
- self.mc.overwrite(jb_location-1, chr(offset))
+ self.mc.patch_forward_jump(jb_location)
#
def _call_footer(self):
@@ -1242,15 +1230,12 @@
return genop_cmp
def _if_parity_clear_zero_and_carry(self):
- self.mc.J_il8(rx86.Conditions['NP'], 0)
- jnp_location = self.mc.get_relative_pos()
+ jnp_location = self.mc.emit_forward_jump('NP')
# CMP EBP, 0: as EBP cannot be null here, that operation should
# always clear zero and carry
self.mc.CMP_ri(ebp.value, 0)
# patch the JNP above
- offset = self.mc.get_relative_pos() - jnp_location
- assert 0 < offset <= 127
- self.mc.overwrite(jnp_location-1, chr(offset))
+ self.mc.patch_forward_jump(jnp_location)
def _cmpop_float(cond, rev_cond):
is_ne = cond == 'NE'
@@ -1728,10 +1713,11 @@
# jump to jump over this GUARD_NO_EXCEPTION as well, if we can
if self._find_nearby_operation(-1).getopnum() in (
rop.COND_CALL, rop.COND_CALL_VALUE_I, rop.COND_CALL_VALUE_R):
- jmp_adr = self.previous_cond_call_jcond
- offset = self.mc.get_relative_pos() - jmp_adr
- if offset <= 127:
- self.mc.overwrite(jmp_adr-1, chr(offset))
+ j_location = self.previous_cond_call_jcond
+ try:
+ self.mc.patch_forward_jump(j_location)
+ except codebuf.ShortJumpTooFar:
+ pass # ignore this case
def genop_guard_guard_not_invalidated(self, guard_op, guard_token,
locs, ign):
@@ -1842,13 +1828,10 @@
def genop_guard_guard_nonnull_class(self, guard_op, guard_token, locs,
ign):
self.mc.CMP(locs[0], imm1)
# Patched below
- self.mc.J_il8(rx86.Conditions['B'], 0)
- jb_location = self.mc.get_relative_pos()
+ jb_location = self.mc.emit_forward_jump('B')
self._cmp_guard_class(locs)
# patch the JB above
- offset = self.mc.get_relative_pos() - jb_location
- assert 0 < offset <= 127
- self.mc.overwrite(jb_location-1, chr(offset))
+ self.mc.patch_forward_jump(jb_location)
#
self.guard_success_cc = rx86.Conditions['E']
self.implement_guard(guard_token)
@@ -2223,19 +2206,15 @@
ofs = self.cpu.get_ofs_of_frame_field('jf_descr')
self.mc.CMP(mem(eax, ofs), imm(value))
# patched later
- self.mc.J_il8(rx86.Conditions['E'], 0) # goto B if we get
'done_with_this_frame'
- return self.mc.get_relative_pos()
+ return self.mc.emit_forward_jump('E') # goto B if we get
'done_with_this_frame'
def _call_assembler_patch_je(self, result_loc, je_location):
if (IS_X86_32 and isinstance(result_loc, FrameLoc) and
result_loc.type == FLOAT):
self.mc.FSTPL_b(result_loc.value)
- self.mc.JMP_l8(0) # jump to done, patched later
- jmp_location = self.mc.get_relative_pos()
+ jmp_location = self.mc.emit_forward_jump_uncond() # jump to the end
#
- offset = jmp_location - je_location
- assert 0 < offset <= 127
- self.mc.overwrite(je_location - 1, chr(offset))
+ self.mc.patch_forward_jump(je_location)
self.mc.force_frame_size(DEFAULT_FRAME_BYTES)
#
return jmp_location
@@ -2255,9 +2234,7 @@
self.mc.MOV_rm(eax.value, (eax.value, ofs))
def _call_assembler_patch_jmp(self, jmp_location):
- offset = self.mc.get_relative_pos() - jmp_location
- assert 0 <= offset <= 127
- self.mc.overwrite(jmp_location - 1, chr(offset))
+ self.mc.patch_forward_jump(jmp_location)
# ------------------- END CALL ASSEMBLER -----------------------
@@ -2288,16 +2265,14 @@
else:
loc = addr_add_const(loc_base, descr.jit_wb_if_flag_byteofs)
mc.TEST8(loc, imm(mask))
- mc.J_il8(rx86.Conditions['Z'], 0) # patched later
- jz_location = mc.get_relative_pos()
+ jz_location = mc.emit_forward_jump('Z') # patched later
# for cond_call_gc_wb_array, also add another fast path:
# if GCFLAG_CARDS_SET, then we can just set one bit and be done
if card_marking:
# GCFLAG_CARDS_SET is in this byte at 0x80, so this fact can
- # been checked by the status flags of the previous TEST8
- mc.J_il8(rx86.Conditions['S'], 0) # patched later
- js_location = mc.get_relative_pos()
+ # been checked by the sign flags of the previous TEST8
+ js_location = mc.emit_forward_jump('S') # patched later
else:
js_location = 0
@@ -2326,13 +2301,10 @@
# The helper ends again with a check of the flag in the object.
# So here, we can simply write again a 'JNS', which will be
# taken if GCFLAG_CARDS_SET is still not set.
- mc.J_il8(rx86.Conditions['NS'], 0) # patched later
- jns_location = mc.get_relative_pos()
+ jns_location = mc.emit_forward_jump('NS') # patched later
#
# patch the JS above
- offset = mc.get_relative_pos() - js_location
- assert 0 < offset <= 127
- mc.overwrite(js_location-1, chr(offset))
+ mc.patch_forward_jump(js_location)
#
# case GCFLAG_CARDS_SET: emit a few instructions to do
# directly the card flag setting
@@ -2367,14 +2339,10 @@
raise AssertionError("index is neither RegLoc nor ImmedLoc")
#
# patch the JNS above
- offset = mc.get_relative_pos() - jns_location
- assert 0 < offset <= 127
- mc.overwrite(jns_location-1, chr(offset))
+ mc.patch_forward_jump(jns_location)
# patch the JZ above
- offset = mc.get_relative_pos() - jz_location
- assert 0 < offset <= 127
- mc.overwrite(jz_location-1, chr(offset))
+ mc.patch_forward_jump(jz_location)
def genop_discard_cond_call_gc_wb(self, op, arglocs):
self._write_barrier_fastpath(self.mc, op.getdescr(), arglocs)
@@ -2407,9 +2375,8 @@
def cond_call(self, gcmap, imm_func, arglocs, resloc=None):
assert self.guard_success_cc >= 0
- self.mc.J_il8(rx86.invert_condition(self.guard_success_cc), 0)
- # patched later
- jmp_adr = self.mc.get_relative_pos()
+ j_location = self.mc.emit_forward_jump_cond(
+ rx86.invert_condition(self.guard_success_cc))
self.guard_success_cc = rx86.cond_none
#
self.push_gcmap(self.mc, gcmap, store=True)
@@ -2459,26 +2426,21 @@
v = gpr_reg_mgr_cls.all_reg_indexes[eax.value]
self.mc.MOV_rb(eax.value, v * WORD + base_ofs)
#
- offset = self.mc.get_relative_pos() - jmp_adr
- assert 0 < offset <= 127
- self.mc.overwrite(jmp_adr-1, chr(offset))
+ self.mc.patch_forward_jump(j_location)
# might be overridden again to skip over the following
# guard_no_exception too
- self.previous_cond_call_jcond = jmp_adr
+ self.previous_cond_call_jcond = j_location
def malloc_cond(self, nursery_free_adr, nursery_top_adr, size, gcmap):
assert size & (WORD-1) == 0 # must be correctly aligned
self.mc.MOV(ecx, heap(nursery_free_adr))
self.mc.LEA_rm(edx.value, (ecx.value, size))
self.mc.CMP(edx, heap(nursery_top_adr))
- self.mc.J_il8(rx86.Conditions['NA'], 0) # patched later
- jmp_adr = self.mc.get_relative_pos()
+ jna_location = self.mc.emit_forward_jump('NA') # patched later
# save the gcmap
self.push_gcmap(self.mc, gcmap, store=True)
self.mc.CALL(imm(follow_jump(self.malloc_slowpath)))
- offset = self.mc.get_relative_pos() - jmp_adr
- assert 0 < offset <= 127
- self.mc.overwrite(jmp_adr-1, chr(offset))
+ self.mc.patch_forward_jump(jna_location)
self.mc.MOV(heap(nursery_free_adr), edx)
def malloc_cond_varsize_frame(self, nursery_free_adr, nursery_top_adr,
@@ -2492,14 +2454,11 @@
else:
self.mc.LEA_ra(edx.value, (ecx.value, sizeloc.value, 0, 0))
self.mc.CMP(edx, heap(nursery_top_adr))
- self.mc.J_il8(rx86.Conditions['NA'], 0) # patched later
- jmp_adr = self.mc.get_relative_pos()
+ jna_location = self.mc.emit_forward_jump('NA') # patched later
# save the gcmap
self.push_gcmap(self.mc, gcmap, store=True)
self.mc.CALL(imm(follow_jump(self.malloc_slowpath)))
- offset = self.mc.get_relative_pos() - jmp_adr
- assert 0 < offset <= 127
- self.mc.overwrite(jmp_adr-1, chr(offset))
+ self.mc.patch_forward_jump(jna_location)
self.mc.MOV(heap(nursery_free_adr), edx)
def malloc_cond_varsize(self, kind, nursery_free_adr, nursery_top_adr,
@@ -2517,8 +2476,7 @@
varsizeloc = edx
self.mc.CMP(varsizeloc, imm(maxlength))
- self.mc.J_il8(rx86.Conditions['A'], 0) # patched later
- jmp_adr0 = self.mc.get_relative_pos()
+ ja_location = self.mc.emit_forward_jump('A') # patched later
self.mc.MOV(ecx, heap(nursery_free_adr))
if valid_addressing_size(itemsize):
@@ -2542,12 +2500,9 @@
# now edx contains the total size in bytes, rounded up to a multiple
# of WORD, plus nursery_free_adr
self.mc.CMP(edx, heap(nursery_top_adr))
- self.mc.J_il8(rx86.Conditions['NA'], 0) # patched later
- jmp_adr1 = self.mc.get_relative_pos()
+ jna_location = self.mc.emit_forward_jump('NA') # patched later
#
- offset = self.mc.get_relative_pos() - jmp_adr0
- assert 0 < offset <= 127
- self.mc.overwrite(jmp_adr0-1, chr(offset))
+ self.mc.patch_forward_jump(ja_location)
# save the gcmap
self.push_gcmap(self.mc, gcmap, store=True)
if kind == rewrite.FLAG_ARRAY:
@@ -2563,21 +2518,16 @@
addr = self.malloc_slowpath_unicode
self.mc.MOV(edx, lengthloc)
self.mc.CALL(imm(follow_jump(addr)))
- self.mc.JMP_l8(0) # jump to done, patched later
- jmp_location = self.mc.get_relative_pos()
+ jmp_location = self.mc.emit_forward_jump_uncond() # jump to later
#
- offset = self.mc.get_relative_pos() - jmp_adr1
- assert 0 < offset <= 127
- self.mc.overwrite(jmp_adr1-1, chr(offset))
+ self.mc.patch_forward_jump(jna_location)
self.mc.force_frame_size(DEFAULT_FRAME_BYTES)
# write down the tid, but not if it's the result of the CALL
self.mc.MOV(mem(ecx, 0), imm(arraydescr.tid))
# while we're at it, this line is not needed if we've done the CALL
self.mc.MOV(heap(nursery_free_adr), edx)
#
- offset = self.mc.get_relative_pos() - jmp_location
- assert 0 < offset <= 127
- self.mc.overwrite(jmp_location - 1, chr(offset))
+ self.mc.patch_forward_jump(jmp_location)
def store_force_descr(self, op, fail_locs, frame_depth):
guard_token = self.implement_guard_recovery(op.opnum,
diff --git a/rpython/jit/backend/x86/callbuilder.py
b/rpython/jit/backend/x86/callbuilder.py
--- a/rpython/jit/backend/x86/callbuilder.py
+++ b/rpython/jit/backend/x86/callbuilder.py
@@ -341,25 +341,20 @@
# thread. So here we check if the shadowstack pointer
# is still the same as before we released the GIL (saved
# in 'ebx'), and if not, we fall back to 'reacqgil_addr'.
- mc.J_il8(rx86.Conditions['NE'], 0)
- jne_location = mc.get_relative_pos()
+ jne_location = mc.emit_forward_jump('NE')
# here, ecx (=old_value) is zero (so rpy_fastgil was in 'released'
# state before the XCHG, but the XCHG acquired it by writing 1)
rst = gcrootmap.get_root_stack_top_addr()
mc = self.mc
mc.CMP(ebx, heap(rst))
- mc.J_il8(rx86.Conditions['E'], 0)
- je_location = mc.get_relative_pos()
+ je_location = mc.emit_forward_jump('E')
# revert the rpy_fastgil acquired above, so that the
# general 'reacqgil_addr' below can acquire it again...
mc.MOV(heap(fastgil), ecx)
# patch the JNE above
- offset = mc.get_relative_pos() - jne_location
- assert 0 < offset <= 127
- mc.overwrite(jne_location-1, chr(offset))
+ mc.patch_forward_jump(jne_location)
else:
- mc.J_il8(rx86.Conditions['E'], 0)
- je_location = mc.get_relative_pos()
+ je_location = mc.emit_forward_jump('E')
#
# Yes, we need to call the reacqgil() function
if not self.result_value_saved_early:
@@ -374,9 +369,7 @@
self.restore_result_value(save_edx=False)
#
# patch the JE above
- offset = mc.get_relative_pos() - je_location
- assert 0 < offset <= 127
- mc.overwrite(je_location-1, chr(offset))
+ mc.patch_forward_jump(je_location)
#
if restore_edx:
mc.MOV_rs(edx.value, 12) # restore this
diff --git a/rpython/jit/backend/x86/codebuf.py
b/rpython/jit/backend/x86/codebuf.py
--- a/rpython/jit/backend/x86/codebuf.py
+++ b/rpython/jit/backend/x86/codebuf.py
@@ -1,12 +1,13 @@
from rpython.rtyper.lltypesystem import lltype, rffi
from rpython.rlib.rarithmetic import intmask
+from rpython.rlib.objectmodel import specialize
from rpython.rlib.debug import debug_start, debug_print, debug_stop
from rpython.rlib.debug import have_debug_prints
from rpython.jit.backend.llsupport.asmmemmgr import BlockBuilderMixin
from rpython.jit.backend.x86.rx86 import X86_32_CodeBuilder, X86_64_CodeBuilder
from rpython.jit.backend.x86.regloc import LocationCodeBuilder
from rpython.jit.backend.x86.arch import IS_X86_32, IS_X86_64, WORD
-from rpython.jit.backend.x86 import valgrind
+from rpython.jit.backend.x86 import rx86, valgrind
# XXX: Seems nasty to change the superclass of MachineCodeBlockWrapper
# like this
@@ -17,6 +18,9 @@
codebuilder_cls = X86_64_CodeBuilder
backend_name = 'x86_64'
+class ShortJumpTooFar(Exception):
+ pass
+
class MachineCodeBlockWrapper(BlockBuilderMixin,
LocationCodeBuilder,
@@ -53,3 +57,22 @@
adr[0] = rffi.cast(rffi.INT, intmask(adr[0]) - p)
valgrind.discard_translations(addr, self.get_relative_pos())
self._dump(addr, "jit-backend-dump", backend_name)
+
+ @specialize.arg(1)
+ def emit_forward_jump(self, condition_string):
+ return self.emit_forward_jump_cond(rx86.Conditions[condition_string])
+
+ def emit_forward_jump_cond(self, cond):
+ self.J_il8(cond, 0)
+ return self.get_relative_pos()
+
+ def emit_forward_jump_uncond(self):
+ self.JMP_l8(0)
+ return self.get_relative_pos()
+
+ def patch_forward_jump(self, jcond_location):
+ offset = self.get_relative_pos() - jcond_location
+ assert offset >= 0
+ if offset > 127:
+ raise ShortJumpTooFar
+ self.overwrite(jcond_location-1, chr(offset))
diff --git a/rpython/rlib/rposix.py b/rpython/rlib/rposix.py
--- a/rpython/rlib/rposix.py
+++ b/rpython/rlib/rposix.py
@@ -464,28 +464,29 @@
how = SEEK_END
return handle_posix_error('lseek', c_lseek(fd, pos, how))
-c_pread = external('pread',
- [rffi.INT, rffi.VOIDP, rffi.SIZE_T , OFF_T], rffi.SSIZE_T,
- save_err=rffi.RFFI_SAVE_ERRNO)
-c_pwrite = external('pwrite',
- [rffi.INT, rffi.VOIDP, rffi.SIZE_T, OFF_T], rffi.SSIZE_T,
- save_err=rffi.RFFI_SAVE_ERRNO)
+if not _WIN32:
+ c_pread = external('pread',
+ [rffi.INT, rffi.VOIDP, rffi.SIZE_T , OFF_T],
rffi.SSIZE_T,
+ save_err=rffi.RFFI_SAVE_ERRNO)
+ c_pwrite = external('pwrite',
+ [rffi.INT, rffi.VOIDP, rffi.SIZE_T, OFF_T],
rffi.SSIZE_T,
+ save_err=rffi.RFFI_SAVE_ERRNO)
-@enforceargs(int, int, None)
-def pread(fd, count, offset):
- if count < 0:
- raise OSError(errno.EINVAL, None)
- validate_fd(fd)
- with rffi.scoped_alloc_buffer(count) as buf:
- void_buf = rffi.cast(rffi.VOIDP, buf.raw)
- return buf.str(handle_posix_error('pread', c_pread(fd, void_buf,
count, offset)))
-
-@enforceargs(int, None, None)
-def pwrite(fd, data, offset):
- count = len(data)
- validate_fd(fd)
- with rffi.scoped_nonmovingbuffer(data) as buf:
- return handle_posix_error('pwrite', c_pwrite(fd, buf, count, offset))
+ @enforceargs(int, int, None)
+ def pread(fd, count, offset):
+ if count < 0:
+ raise OSError(errno.EINVAL, None)
+ validate_fd(fd)
+ with rffi.scoped_alloc_buffer(count) as buf:
+ void_buf = rffi.cast(rffi.VOIDP, buf.raw)
+ return buf.str(handle_posix_error('pread', c_pread(fd, void_buf,
count, offset)))
+
+ @enforceargs(int, None, None)
+ def pwrite(fd, data, offset):
+ count = len(data)
+ validate_fd(fd)
+ with rffi.scoped_nonmovingbuffer(data) as buf:
+ return handle_posix_error('pwrite', c_pwrite(fd, buf, count,
offset))
c_ftruncate = external('ftruncate', [rffi.INT, rffi.LONGLONG], rffi.INT,
macro=_MACRO_ON_POSIX, save_err=rffi.RFFI_SAVE_ERRNO)
_______________________________________________
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit
