date:20150511

[pypy-commit] cffi cffi-1.0: Start reviewing the docs

2015-05-11 Thread arigo

Author: Armin Rigo ar...@tunes.org
Branch: cffi-1.0
Changeset: r1965:0292ad4234e1
Date: 2015-05-11 09:26 +0200
http://bitbucket.org/cffi/cffi/changeset/0292ad4234e1/

Log:Start reviewing the docs

diff --git a/doc/source/index.rst b/doc/source/index.rst
--- a/doc/source/index.rst
+++ b/doc/source/index.rst
@@ -58,43 +58,53 @@
 
 In more details:
 
-This code has been developed on Linux but should work on any POSIX
-platform as well as on Win32.  There are some Windows-specific issues
-left.
+This code has been developed on Linux, but should work on any POSIX
+platform as well as on Windows 32 and 64.  (It relies occasionally on
+libffi, so it depends on libffi being bug-free; this may not be fully
+the case on some of the more exotic platforms.)
 
-It supports CPython 2.6; 2.7; 3.x (tested with 3.2 and 3.3);
-and is distributed with PyPy 2.0 beta2 or later.
+CFFI supports CPython 2.6, 2.7, 3.x (tested with 3.2 to 3.4); and is
+distributed with PyPy 2.0 beta2 or later.  CFFI 1.0 is distributed
+with (and requires) PyPy 2.6.
 
-Its speed is comparable to ctypes on CPython (a bit faster but a higher
-warm-up time).  It is already faster than ctypes on PyPy (1.5x-2x), but not yet
-*much* faster; stay tuned.
+The core speed of CFFI is better than ctypes, with import times being
+either lower if you use the post-1.0 features, or much higher if you
+don't.  The wrapper Python code you typically need to write around the
+raw CFFI interface slows things down on CPython, but not unreasonably
+so.  On PyPy, this wrapper code has a minimal impact thanks to the JIT
+compiler.  This makes CFFI the recommended way to interface with C
+libraries on PyPy.
 
 Requirements:
 
-* CPython 2.6 or 2.7 or 3.x, or PyPy 2.0 beta2
+* CPython 2.6 or 2.7 or 3.x, or PyPy (PyPy 2.0 for the earliest
+  versions of CFFI; or PyPy 2.6 for CFFI 1.0).
 
-* on CPython you need to build the C extension module, so you need
-  ``python-dev`` and ``libffi-dev`` (for Windows, libffi is included
-  with CFFI).
+* in some cases you need to be able to compile C extension modules;
+  refer to the appropriate docs for your OS.  This includes installing
+  CFFI from sources (CPython only, as it is already included with
+  PyPy); or developing code based on ``ffi.set_source()`` or
+  ``ffi.verify()``; or installing such 3rd-party modules from sources.
 
-* pycparser = 2.06: https://github.com/eliben/pycparser
+* on CPython, on non-Windows platforms, you also need to install
+  ``libffi-dev`` in order to compile CFFI itself.
 
-* a C compiler is required to use CFFI during development, but not to run
-  correctly-installed programs that use CFFI.
+* pycparser = 2.06: https://github.com/eliben/pycparser (automatically
+  tracked by ``pip install cffi``).
 
-* `py.test`_ is needed to run the tests of CFFI.
+* `py.test`_ is needed to run the tests of CFFI itself.
 
 .. _`py.test`: http://pypi.python.org/pypi/pytest
 
 Download and Installation:
 
-* http://pypi.python.org/packages/source/c/cffi/cffi-0.9.2.tar.gz
+* http://pypi.python.org/packages/source/c/cffi/cffi-1.0.0.tar.gz
 
- Or grab the most current version by following the instructions below.
 
-   - MD5: b1bf4625ae07a8a932f2f1a2eb200c54
+   - MD5: ...
 
-   - SHA: 7cfc992699ef8b65d6300c04f3efad00bd2a6cba
+   - SHA: ...
 
 * Or get it from the `Bitbucket page`_:
   ``hg clone https://bitbucket.org/cffi/cffi``
@@ -103,13 +113,9 @@
   (should work out of the box on Linux or Windows; see below for
   `MacOS X`_ or `Windows 64`_.)
 
-* or you can directly import and use ``cffi``, but if you don't
-  compile the ``_cffi_backend`` extension module, it will fall back
-  to using internally ``ctypes`` (much slower; we recommend not to use it).
-
-* running the tests: ``py.test c/ testing/`` (if you didn't
-  install cffi yet, you may need ``python setup_base.py build``
-  and ``PYTHONPATH=build/lib.xyz.../``)
+* running the tests: ``py.test c/ _cffi1/ testing/`` (if you didn't
+  install cffi yet, you may need ``python setup_base.py build_ext -f
+  -i``)
 
 .. _`Bitbucket page`: https://bitbucket.org/cffi/cffi
 
@@ -118,13 +124,13 @@
 * The `demo`_ directory contains a number of small and large demos
   of using ``cffi``.
 
-* The documentation below is sketchy on the details; for now the
+* The documentation below might be sketchy on details; for now the
   ultimate reference is given by the tests, notably
-  `testing/test_verify.py`_ and `testing/backend_tests.py`_.
+  `_cffi1/test_verify1.py`_ and `_cffi1/test_new_ffi_1.py`_.
 
 .. _`demo`: https://bitbucket.org/cffi/cffi/src/default/demo
-.. _`testing/backend_tests.py`: 
https://bitbucket.org/cffi/cffi/src/default/testing/backend_tests.py
-.. _`testing/test_verify.py`: 
https://bitbucket.org/cffi/cffi/src/default/testing/test_verify.py
+.. _`cffi1/test_verify1.py`: 
https://bitbucket.org/cffi/cffi/src/default/_cffi1/test_verify1.py
+.. _`testing/test_verify.py`: 
https://bitbucket.org/cffi/cffi/src/default/_cffi1/test_new_ffi_1.py

[pypy-commit] cffi cffi-1.0: Tweak tweak tweak: use an initialization sequence on CPython that is closer

2015-05-11 Thread arigo

Author: Armin Rigo ar...@tunes.org
Branch: cffi-1.0
Changeset: r1966:897b96412b7a
Date: 2015-05-11 11:15 +0200
http://bitbucket.org/cffi/cffi/changeset/897b96412b7a/

Log:Tweak tweak tweak: use an initialization sequence on CPython that is
closer to the PyPy one. Should make future version checking more
uniform in CPython and PyPy.

diff --git a/_cffi1/_cffi_include.h b/_cffi1/_cffi_include.h
--- a/_cffi1/_cffi_include.h
+++ b/_cffi1/_cffi_include.h
@@ -51,14 +51,6 @@
 #ifndef PYPY_VERSION
 
 
-#if PY_MAJOR_VERSION  3
-# undef PyCapsule_CheckExact
-# undef PyCapsule_GetPointer
-# define PyCapsule_CheckExact(capsule) (PyCObject_Check(capsule))
-# define PyCapsule_GetPointer(capsule, name) \
-(PyCObject_AsVoidPtr(capsule))
-#endif
-
 #if PY_MAJOR_VERSION = 3
 # define PyInt_FromLong PyLong_FromLong
 #endif
@@ -143,10 +135,7 @@
 ((Py_ssize_t(*)(CTypeDescrObject *, PyObject *, char **))_cffi_exports[23])
 #define _cffi_convert_array_from_object  \
 ((int(*)(char *, CTypeDescrObject *, PyObject *))_cffi_exports[24])
-#define _cffi_init_module\
-((PyObject *(*)(char *, const struct _cffi_type_context_s *))\
-  _cffi_exports[25])
-#define _CFFI_NUM_EXPORTS 26
+#define _CFFI_NUM_EXPORTS 25
 
 typedef struct _ctypedescr CTypeDescrObject;
 
@@ -156,41 +145,37 @@
 assertuintptr_t)_cffi_types[index])  1) == 0), \
 (CTypeDescrObject *)_cffi_types[index])
 
-static int _cffi_init(void)
+static PyObject *_cffi_init(char *module_name, Py_ssize_t version,
+const struct _cffi_type_context_s *ctx)
 {
-PyObject *module, *c_api_object = NULL;
-void *src;
+PyObject *module, *o_arg, *new_module;
+void *raw[] = {
+(void *)module_name,
+(void *)version,
+(void *)_cffi_exports,
+(void *)ctx,
+};
 
 module = PyImport_ImportModule(_cffi_backend);
 if (module == NULL)
 goto failure;
 
-c_api_object = PyObject_GetAttrString(module, _C_API);
-if (c_api_object == NULL)
+o_arg = PyLong_FromVoidPtr((void *)raw);
+if (o_arg == NULL)
 goto failure;
-if (!PyCapsule_CheckExact(c_api_object)) {
-PyErr_SetNone(PyExc_ImportError);
-goto failure;
-}
-src = PyCapsule_GetPointer(c_api_object, cffi);
-if ((uintptr_t)(((void **)src)[0])  _CFFI_NUM_EXPORTS) {
-PyErr_SetString(PyExc_ImportError,
-the _cffi_backend module is an outdated version);
-goto failure;
-}
-memcpy(_cffi_exports, src, _CFFI_NUM_EXPORTS * sizeof(void *));
 
+new_module = PyObject_CallMethod(
+module, _init_cffi_1_0_external_module, O, o_arg);
+
+Py_DECREF(o_arg);
 Py_DECREF(module);
-Py_DECREF(c_api_object);
-return 0;
+return new_module;
 
   failure:
 Py_XDECREF(module);
-Py_XDECREF(c_api_object);
-return -1;
+return NULL;
 }
 
-
 #endif
 /**  end CPython-specific section  **/
 
diff --git a/_cffi1/cffi1_module.c b/_cffi1/cffi1_module.c
--- a/_cffi1/cffi1_module.c
+++ b/_cffi1/cffi1_module.c
@@ -103,16 +103,9 @@
 return -1;
 }
 
-static PyObject *_cffi_init_module(char *module_name,
-   const struct _cffi_type_context_s *ctx)
+static PyObject *_my_Py_InitModule(char *module_name)
 {
-PyObject *m;
-FFIObject *ffi;
-LibObject *lib;
-
 #if PY_MAJOR_VERSION = 3
-/* note: the module_def leaks, but anyway the C extension module cannot
-   be unloaded */
 struct PyModuleDef *module_def, local_module_def = {
 PyModuleDef_HEAD_INIT,
 module_name,
@@ -120,17 +113,57 @@
 -1,
 NULL, NULL, NULL, NULL, NULL
 };
+/* note: the 'module_def' is allocated dynamically and leaks,
+   but anyway the C extension module can never be unloaded */
 module_def = PyMem_Malloc(sizeof(struct PyModuleDef));
 if (module_def == NULL)
 return PyErr_NoMemory();
 *module_def = local_module_def;
-m = PyModule_Create(module_def);
+return PyModule_Create(module_def);
 #else
-m = Py_InitModule(module_name, NULL);
+return Py_InitModule(module_name, NULL);
 #endif
+}
+
+#define CFFI_VERSION_MIN0x2600
+#define CFFI_VERSION_MAX0x260F
+
+static PyObject *b_init_cffi_1_0_external_module(PyObject *self, PyObject *arg)
+{
+PyObject *m;
+FFIObject *ffi;
+LibObject *lib;
+Py_ssize_t version;
+char *module_name, *exports;
+void **raw;
+const struct _cffi_type_context_s *ctx;
+
+raw = (void **)PyLong_AsVoidPtr(arg);
+if (raw == NULL)
+return NULL;
+
+module_name = (char *)raw[0];
+version = (Py_ssize_t)raw[1];
+exports = (char *)raw[2];
+ctx = (const struct _cffi_type_context_s *)raw[3];
+
+if (version  CFFI_VERSION_MIN || version  CFFI_VERSION_MAX) {
+if

[pypy-commit] pypy cffi-1.0: Passing test

2015-05-11 Thread arigo

Author: Armin Rigo ar...@tunes.org
Branch: cffi-1.0
Changeset: r77289:99520e806061
Date: 2015-05-11 11:22 +0200
http://bitbucket.org/pypy/pypy/changeset/99520e806061/

Log:Passing test

diff --git a/pypy/module/_cffi_backend/test/test_recompiler.py 
b/pypy/module/_cffi_backend/test/test_recompiler.py
--- a/pypy/module/_cffi_backend/test/test_recompiler.py
+++ b/pypy/module/_cffi_backend/test/test_recompiler.py
@@ -58,6 +58,7 @@
 w_res = space.appexec(args_w, (modulename, filename):
 import imp
 mod = imp.load_dynamic(modulename, filename)
+assert mod.__name__ == modulename
 return (mod.ffi, mod.lib)
 )
 ffiobject = space.getitem(w_res, space.wrap(0))
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] cffi cffi-1.0: Make ffi-types_builder an inlined struct; add a ref from the lib to the ffi

2015-05-11 Thread arigo

Author: Armin Rigo ar...@tunes.org
Branch: cffi-1.0
Changeset: r1970:03470b52f5a3
Date: 2015-05-11 12:10 +0200
http://bitbucket.org/cffi/cffi/changeset/03470b52f5a3/

Log:Make ffi-types_builder an inlined struct; add a ref from the lib to
the ffi

diff --git a/_cffi1/cffi1_module.c b/_cffi1/cffi1_module.c
--- a/_cffi1/cffi1_module.c
+++ b/_cffi1/cffi1_module.c
@@ -169,12 +169,12 @@
 if (ffi == NULL || PyModule_AddObject(m, ffi, (PyObject *)ffi)  0)
 return NULL;
 
-lib = lib_internal_new(ffi-types_builder, module_name);
+lib = lib_internal_new(ffi, module_name);
 if (lib == NULL || PyModule_AddObject(m, lib, (PyObject *)lib)  0)
 return NULL;
 
 if (make_included_tuples(module_name, ctx-includes,
- ffi-types_builder-included_ffis,
+ ffi-types_builder.included_ffis,
  lib-l_includes)  0)
 return NULL;
 
diff --git a/_cffi1/ffi_obj.c b/_cffi1/ffi_obj.c
--- a/_cffi1/ffi_obj.c
+++ b/_cffi1/ffi_obj.c
@@ -25,7 +25,7 @@
 PyObject *gc_wrefs;
 struct _cffi_parse_info_s info;
 int ctx_is_static;
-builder_c_t *types_builder;
+builder_c_t types_builder;
 };
 
 static FFIObject *ffi_internal_new(PyTypeObject *ffitype,
@@ -45,13 +45,12 @@
 if (ffi == NULL)
 return NULL;
 
-ffi-types_builder = new_builder_c(static_ctx);
-if (ffi-types_builder == NULL) {
+if (init_builder_c(ffi-types_builder, static_ctx)  0) {
 Py_DECREF(ffi);
 return NULL;
 }
 ffi-gc_wrefs = NULL;
-ffi-info.ctx = ffi-types_builder-ctx;
+ffi-info.ctx = ffi-types_builder.ctx;
 ffi-info.output = internal_output;
 ffi-info.output_size = FFI_COMPLEXITY_OUTPUT;
 ffi-ctx_is_static = (static_ctx != NULL);
@@ -70,15 +69,15 @@
 #endif
 
 if (!ffi-ctx_is_static)
-free_builder_c(ffi-types_builder);
+free_dynamic_builder_c(ffi-types_builder);
 
 Py_TYPE(ffi)-tp_free((PyObject *)ffi);
 }
 
 static int ffi_traverse(FFIObject *ffi, visitproc visit, void *arg)
 {
-Py_VISIT(ffi-types_builder-types_dict);
-Py_VISIT(ffi-types_builder-included_ffis);
+Py_VISIT(ffi-types_builder.types_dict);
+Py_VISIT(ffi-types_builder.included_ffis);
 Py_VISIT(ffi-gc_wrefs);
 return 0;
 }
@@ -110,7 +109,7 @@
Does not return a new reference!
 */
 if ((accept  ACCEPT_STRING)  PyText_Check(arg)) {
-PyObject *types_dict = ffi-types_builder-types_dict;
+PyObject *types_dict = ffi-types_builder.types_dict;
 PyObject *x = PyDict_GetItem(types_dict, arg);
 
 if (x == NULL) {
@@ -125,7 +124,7 @@
  input_text, spaces);
 return NULL;
 }
-x = realize_c_type_or_func(ffi-types_builder,
+x = realize_c_type_or_func(ffi-types_builder,
ffi-info.output, index);
 if (x == NULL)
 return NULL;
@@ -749,12 +748,12 @@
 Py_INCREF(args); /* to keep alive the strings in '.name' */
 Py_XDECREF(self-dynamic_types);
 self-dynamic_types = args;
-self-types_builder-ctx.types = types;
-self-types_builder-num_types_imported = lst1_length + lst2_length;
-self-types_builder-ctx.struct_unions = struct_unions;
-self-types_builder-ctx.num_struct_unions = lst1_length;
-self-types_builder-ctx.typenames = typenames;
-self-types_builder-ctx.num_typenames = lst2_length;
+self-types_builder.ctx.types = types;
+self-types_builder.num_types_imported = lst1_length + lst2_length;
+self-types_builder.ctx.struct_unions = struct_unions;
+self-types_builder.ctx.num_struct_unions = lst1_length;
+self-types_builder.ctx.typenames = typenames;
+self-types_builder.ctx.num_typenames = lst2_length;
 
 Py_INCREF(Py_None);
 return Py_None;
@@ -855,19 +854,19 @@
 PyObject *x;
 
 ffi1 = (FFIObject *)PyTuple_GET_ITEM(included_ffis, i);
-sindex = search_in_struct_unions(ffi1-types_builder-ctx, s-name,
+sindex = search_in_struct_unions(ffi1-types_builder.ctx, s-name,
  strlen(s-name));
 if (sindex  0)  /* not found at all */
 continue;
-s1 = ffi1-types_builder-ctx.struct_unions[sindex];
+s1 = ffi1-types_builder.ctx.struct_unions[sindex];
 if ((s1-flags  (_CFFI_F_EXTERNAL | _CFFI_F_UNION))
 == (s-flags  _CFFI_F_UNION)) {
 /* s1 is not external, and the same kind (struct or union) as s */
-return _realize_c_struct_or_union(ffi1-types_builder, sindex);
+return _realize_c_struct_or_union(ffi1-types_builder, sindex);
 }
 /* not found, look more recursively */
 x = _fetch_external_struct_or_union(
-s, ffi1-types_builder-included_ffis, recursion + 1);
+s, ffi1-types_builder.included_ffis, recursion + 1);

[pypy-commit] cffi cffi-1.0: Passing test

2015-05-11 Thread arigo

Author: Armin Rigo ar...@tunes.org
Branch: cffi-1.0
Changeset: r1967:a6d2d25ad174
Date: 2015-05-11 11:20 +0200
http://bitbucket.org/cffi/cffi/changeset/a6d2d25ad174/

Log:Passing test

diff --git a/_cffi1/test_recompiler.py b/_cffi1/test_recompiler.py
--- a/_cffi1/test_recompiler.py
+++ b/_cffi1/test_recompiler.py
@@ -467,6 +467,8 @@
 sys.path.insert(0, str(udir))
 import test_module_name_in_package.mymod
 assert test_module_name_in_package.mymod.lib.foo(10) == 42
+assert test_module_name_in_package.mymod.__name__ == (
+'test_module_name_in_package.mymod')
 finally:
 sys.path[:] = old_sys_path
 
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] cffi cffi-1.0: A case where raising NotImplementedError is more appropriate than just

2015-05-11 Thread arigo

Author: Armin Rigo ar...@tunes.org
Branch: cffi-1.0
Changeset: r1969:861f9ef06179
Date: 2015-05-11 11:53 +0200
http://bitbucket.org/cffi/cffi/changeset/861f9ef06179/

Log:A case where raising NotImplementedError is more appropriate than
just AttributeError

diff --git a/cffi/api.py b/cffi/api.py
--- a/cffi/api.py
+++ b/cffi/api.py
@@ -583,6 +583,11 @@
 if name in library.__dict__:
 return
 #
+key = 'constant ' + name
+if key in ffi._parser._declarations:
+raise NotImplementedError(fetching a non-integer constant 
+  after dlopen())
+#
 raise AttributeError(name)
 #
 def make_accessor(name):
diff --git a/testing/test_function.py b/testing/test_function.py
--- a/testing/test_function.py
+++ b/testing/test_function.py
@@ -101,6 +101,17 @@
 x = m.cos(1.23)
 assert x == math.cos(1.23)
 
+def test_dlopen_constant(self):
+ffi = FFI(backend=self.Backend())
+ffi.cdef(
+#define FOOBAR 42
+static const float baz = 42.5;   /* not visible */
+double sin(double x);
+)
+m = ffi.dlopen(lib_m)
+assert m.FOOBAR == 42
+py.test.raises(NotImplementedError, m.baz)
+
 def test_tlsalloc(self):
 if sys.platform != 'win32':
 py.test.skip(win32 only)
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] cffi cffi-1.0: Python 3 compat

2015-05-11 Thread arigo

Author: Armin Rigo ar...@tunes.org
Branch: cffi-1.0
Changeset: r1968:11bceeb4ecfa
Date: 2015-05-11 11:24 +0200
http://bitbucket.org/cffi/cffi/changeset/11bceeb4ecfa/

Log:Python 3 compat

diff --git a/_cffi1/test_ffi_obj.py b/_cffi1/test_ffi_obj.py
--- a/_cffi1/test_ffi_obj.py
+++ b/_cffi1/test_ffi_obj.py
@@ -161,7 +161,7 @@
 def test_ffi_buffer():
 ffi = _cffi1_backend.FFI()
 a = ffi.new(signed char[], [5, 6, 7])
-assert ffi.buffer(a)[:] == '\x05\x06\x07'
+assert ffi.buffer(a)[:] == b'\x05\x06\x07'
 
 def test_ffi_from_buffer():
 import array
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] pypy vecopt: added some missing vector x86 instructions to mc

2015-05-11 Thread plan_rich

Author: Richard Plangger r...@pasra.at
Branch: vecopt
Changeset: r77292:7d60c4409027
Date: 2015-05-11 15:46 +0200
http://bitbucket.org/pypy/pypy/changeset/7d60c4409027/

Log:added some missing vector x86 instructions to mc started to
implement the new instructions (pack/unpack/expand/...) not yet
finished. i need to find the suitable instructions for those

diff --git a/pypy/module/micronumpy/test/test_zjit.py 
b/pypy/module/micronumpy/test/test_zjit.py
--- a/pypy/module/micronumpy/test/test_zjit.py
+++ b/pypy/module/micronumpy/test/test_zjit.py
@@ -4,15 +4,20 @@
 
 import py
 from rpython.jit.metainterp.test.support import LLJitMixin
+from rpython.jit.backend.x86.test.test_basic import Jit386Mixin
 from rpython.jit.metainterp.warmspot import reset_jit, get_stats
 from pypy.module.micronumpy import boxes
 from pypy.module.micronumpy.compile import FakeSpace, Parser, InterpreterState
 from pypy.module.micronumpy.base import W_NDimArray
 
-class TestNumpyJit(LLJitMixin):
+class TestNumpyJit(Jit386Mixin):
 graph = None
 interp = None
 
+def setup_method(self, method):
+if not self.CPUClass.vector_extension:
+py.test.skip(needs vector extension to run (for now))
+
 def setup_class(cls):
 default = 
 a = [1,2,3,4]
@@ -128,7 +133,6 @@
 
 
 def test_sum(self):
-py.test.skip('TODO')
 result = self.run(sum)
 assert result == sum(range(30))
 self.check_trace_count(1)
@@ -150,7 +154,6 @@
 
 
 def test_cumsum(self):
-py.test.skip('TODO')
 result = self.run(cumsum)
 assert result == 15
 self.check_trace_count(1)
@@ -220,7 +223,6 @@
 })
 
 def define_reduce():
-py.test.skip('TODO')
 return 
 a = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
 sum(a)
diff --git a/rpython/jit/backend/llgraph/runner.py 
b/rpython/jit/backend/llgraph/runner.py
--- a/rpython/jit/backend/llgraph/runner.py
+++ b/rpython/jit/backend/llgraph/runner.py
@@ -242,7 +242,10 @@
 translate_support_code = False
 is_llgraph = True
 
-vector_register_size = 16
+vector_extension = True
+vector_register_size = 16 # in bytes
+vector_horizontal_operations = True
+vector_pack_slots = True
 
 def __init__(self, rtyper, stats=None, *ignored_args, **kwds):
 model.AbstractCPU.__init__(self)
@@ -794,8 +797,6 @@
 _type = longlong.FLOATSTORAGE
 else:
 raise AssertionError(box)
-#for a in arg:
-#assert lltype.typeOf(a) == _type
 else:
 raise AssertionError(box)
 #
diff --git a/rpython/jit/backend/llsupport/llmodel.py 
b/rpython/jit/backend/llsupport/llmodel.py
--- a/rpython/jit/backend/llsupport/llmodel.py
+++ b/rpython/jit/backend/llsupport/llmodel.py
@@ -25,6 +25,11 @@
 
 HAS_CODEMAP = False
 
+vector_extension = False
+vector_register_size = 0 # in bytes
+vector_horizontal_operations = False
+vector_pack_slots = False
+
 def __init__(self, rtyper, stats, opts, translate_support_code=False,
  gcdescr=None):
 assert type(opts) is not bool
diff --git a/rpython/jit/backend/x86/assembler.py 
b/rpython/jit/backend/x86/assembler.py
--- a/rpython/jit/backend/x86/assembler.py
+++ b/rpython/jit/backend/x86/assembler.py
@@ -1,5 +1,6 @@
 import sys
 import os
+import py
 
 from rpython.jit.backend.llsupport import symbolic, jitframe, rewrite
 from rpython.jit.backend.llsupport.assembler import (GuardToken, BaseAssembler,
@@ -2517,6 +2518,45 @@
 else:
 raise NotImplementedError
 
+def genop_vec_int_sub(self, op, arglocs, resloc):
+loc0, loc1, itemsize_loc = arglocs
+itemsize = itemsize_loc.value
+if itemsize == 1:
+self.mc.PSUBB(loc0, loc1)
+elif itemsize == 2:
+self.mc.PSUBW(loc0, loc1)
+elif itemsize == 4:
+self.mc.PSUBD(loc0, loc1)
+elif itemsize == 8:
+self.mc.PSUBQ(loc0, loc1)
+else:
+raise NotImplementedError
+
+genop_vec_float_arith = 
+def genop_vec_float_{type}(self, op, arglocs, resloc):
+loc0, loc1, itemsize_loc = arglocs
+itemsize = itemsize_loc.value
+if itemsize == 4:
+self.mc.{p_op_s}(loc0, loc1)
+elif itemsize == 8:
+self.mc.{p_op_d}(loc0, loc1)
+else:
+raise NotImplementedError
+
+for op in ['add','mul','sub','div']:
+OP = op.upper()
+_source = genop_vec_float_arith.format(type=op, 
p_op_s=OP+'PS',p_op_d=OP+'PD')
+exec py.code.Source(_source).compile()
+del genop_vec_float_arith
+
+def genop_vec_unpack(self, op, arglocs, resloc):
+loc0, indexloc, sizeloc = arglocs
+size = sizeloc.value
+if size == 4:
+pass
+elif size == 8:
+self.mc.CMPPD(
+
 def genop_vec_int_signext(self,

[pypy-commit] pypy vecopt: renamed detect_sse2.py to detect_feature.py

2015-05-11 Thread plan_rich

Author: Richard Plangger r...@pasra.at
Branch: vecopt
Changeset: r77290:ddd27b50021e
Date: 2015-05-11 10:48 +0200
http://bitbucket.org/pypy/pypy/changeset/ddd27b50021e/

Log:renamed detect_sse2.py to detect_feature.py added checks for sse
4.1, 4.2 and 4a

diff --git a/rpython/jit/backend/detect_cpu.py 
b/rpython/jit/backend/detect_cpu.py
--- a/rpython/jit/backend/detect_cpu.py
+++ b/rpython/jit/backend/detect_cpu.py
@@ -73,12 +73,12 @@
 result = MODEL_X86_64
 else:
 assert sys.maxint == 2**31-1
-from rpython.jit.backend.x86 import detect_sse2
-if detect_sse2.detect_sse2():
+from rpython.jit.backend.x86 import detect_feature
+if detect_feature.detect_sse2():
 result = MODEL_X86
 else:
 result = MODEL_X86_NO_SSE2
-if detect_sse2.detect_x32_mode():
+if detect_feature.detect_x32_mode():
 raise ProcessorAutodetectError(
 'JITting in x32 mode is not implemented')
 #
diff --git a/rpython/jit/backend/x86/detect_feature.py 
b/rpython/jit/backend/x86/detect_feature.py
new file mode 100644
--- /dev/null
+++ b/rpython/jit/backend/x86/detect_feature.py
@@ -0,0 +1,74 @@
+import sys
+import struct
+from rpython.rtyper.lltypesystem import lltype, rffi
+from rpython.rlib.rmmap import alloc, free
+
+def cpu_info(instr):
+data = alloc(4096)
+pos = 0
+for c in instr:
+data[pos] = c
+pos += 1
+fnptr = rffi.cast(lltype.Ptr(lltype.FuncType([], lltype.Signed)), data)
+code = fnptr()
+free(data, 4096)
+return code
+
+def detect_sse2():
+code = cpu_id(eax=1)
+return bool(code  (125)) and bool(code  (126))
+
+def cpu_id(eax = 1, ret_edx=True, ret_ecx = False):
+asm = \xB8 + struct.pack('I', eax) # MOV EAX, $eax
+asm += \x53 # PUSH EBX
+   \x0F\xA2 # CPUID
+   \x5B # POP EBX
+if ret_edx:
+asm += \x92 # XCHG EAX, EDX
+elif ret_ecx:
+asm += \x91 # XCHG EAX, ECX
+asm += \xC3 # RET
+#code = cpu_info(\xB8\x01\x00\x00\x00 # MOV EAX, 1
+#\x53 # PUSH EBX
+#\x0F\xA2 # CPUID
+#\x5B # POP EBX
+#\x92 # XCHG EAX, EDX
+#\xC3 # RET
+#   )
+return cpu_info(asm)
+
+def detect_sse4_1(code=-1):
+ use cpu_id_eax_1_ecx() to get code parameter 
+if code == -1:
+code = cpu_id(eax=1, ret_edx=False, ret_ecx=False)
+return bool(code  (119))
+
+def detect_sse4_2(code=-1):
+ use cpu_id_eax_1_ecx() to get code parameter 
+if code == -1:
+code = cpu_id(eax=1, ret_edx=False, ret_ecx=False)
+return bool(code  (120))
+
+def detect_sse4a(code=-1):
+ use cpu_id_eax_1_ecx() to get code parameter 
+if code == -1:
+code = feature.cpu_id(eax=0x8001, ret_edx=False, ret_ecx=True)
+return bool(code  (120))
+
+def detect_x32_mode():
+# 32-bit 64-bit / x32
+code = cpu_info(\x48# DEC EAX
+\xB8\xC8\x00\x00\x00# MOV EAX, 200   MOV RAX, 
0x4040404000C8
+\x40\x40\x40\x40# 4x INC EAX
+\xC3)   # RETRET
+assert code in (200, 204, 0x4040404000C8)
+return code == 200
+
+
+if __name__ == '__main__':
+if detect_sse2():
+print 'Processor supports sse2.'
+else:
+print 'Missing processor support for sse2.'
+if detect_x32_mode():
+print 'Process is running in x32 mode.'
diff --git a/rpython/jit/backend/x86/detect_sse2.py 
b/rpython/jit/backend/x86/detect_sse2.py
deleted file mode 100644
--- a/rpython/jit/backend/x86/detect_sse2.py
+++ /dev/null
@@ -1,51 +0,0 @@
-import sys
-from rpython.rtyper.lltypesystem import lltype, rffi
-from rpython.rlib.rmmap import alloc, free
-
-def cpu_info(instr):
-data = alloc(4096)
-pos = 0
-for c in instr:
-data[pos] = c
-pos += 1
-fnptr = rffi.cast(lltype.Ptr(lltype.FuncType([], lltype.Signed)), data)
-code = fnptr()
-free(data, 4096)
-return code
-
-def detect_sse2():
-code = cpu_info(\xB8\x01\x00\x00\x00 # MOV EAX, 1
-\x53 # PUSH EBX
-\x0F\xA2 # CPUID
-\x5B # POP EBX
-\x92 # XCHG EAX, EDX
-\xC3 # RET
-   )
-return bool(code  (125)) and bool(code  (126))
-
-def byte_size_for_vector_registers(sse2, avx, avxbw):
-if avx:
-if avxbw:
-return 64
-return 32
-if sse2:
-return 16
-assert False, No

[pypy-commit] pypy vecopt: simplifications, added a x86_64 sse4 cpu

2015-05-11 Thread plan_rich

Author: Richard Plangger r...@pasra.at
Branch: vecopt
Changeset: r77291:9363d09e85dc
Date: 2015-05-11 11:08 +0200
http://bitbucket.org/pypy/pypy/changeset/9363d09e85dc/

Log:simplifications, added a x86_64 sse4 cpu

diff --git a/rpython/jit/backend/detect_cpu.py 
b/rpython/jit/backend/detect_cpu.py
--- a/rpython/jit/backend/detect_cpu.py
+++ b/rpython/jit/backend/detect_cpu.py
@@ -13,6 +13,7 @@
 MODEL_X86 = 'x86'
 MODEL_X86_NO_SSE2 = 'x86-without-sse2'
 MODEL_X86_64  = 'x86-64'
+MODEL_X86_64_SSE4 = 'x86-64-sse4'
 MODEL_ARM = 'arm'
 MODEL_PPC_64  = 'ppc-64'
 # don't use '_' in the model strings; they are replaced by '-'
@@ -69,18 +70,22 @@
 raise ProcessorAutodetectError, unknown machine name %s % mach
 #
 if result.startswith('x86'):
+from rpython.jit.backend.x86 import detect_feature as feature
 if sys.maxint == 2**63-1:
 result = MODEL_X86_64
+# has sse 2 at least
+if feature.detect_sse4_1():
+result = MODEL_X86_64_SSE4
 else:
 assert sys.maxint == 2**31-1
-from rpython.jit.backend.x86 import detect_feature
-if detect_feature.detect_sse2():
+if feature.detect_sse2():
 result = MODEL_X86
 else:
 result = MODEL_X86_NO_SSE2
 if detect_feature.detect_x32_mode():
 raise ProcessorAutodetectError(
 'JITting in x32 mode is not implemented')
+
 #
 if result.startswith('arm'):
 from rpython.jit.backend.arm.detect import detect_float
@@ -108,6 +113,8 @@
 return rpython.jit.backend.x86.runner, CPU386_NO_SSE2
 elif backend_name == MODEL_X86_64:
 return rpython.jit.backend.x86.runner, CPU_X86_64
+elif backend_name == MODEL_X86_64_SSE4:
+return rpython.jit.backend.x86.runner, CPU_X86_64_SSE4
 elif backend_name == MODEL_ARM:
 return rpython.jit.backend.arm.runner, CPU_ARM
 else:
diff --git a/rpython/jit/backend/x86/detect_feature.py 
b/rpython/jit/backend/x86/detect_feature.py
--- a/rpython/jit/backend/x86/detect_feature.py
+++ b/rpython/jit/backend/x86/detect_feature.py
@@ -18,41 +18,32 @@
 code = cpu_id(eax=1)
 return bool(code  (125)) and bool(code  (126))
 
-def cpu_id(eax = 1, ret_edx=True, ret_ecx = False):
+def cpu_id(eax = 1, ret_edx = True, ret_ecx = False):
 asm = \xB8 + struct.pack('I', eax) # MOV EAX, $eax
-asm += \x53 # PUSH EBX
-   \x0F\xA2 # CPUID
-   \x5B # POP EBX
+asm += (\x53 # PUSH EBX
+\x0F\xA2 # CPUID
+\x5B # POP EBX
+   )
 if ret_edx:
 asm += \x92 # XCHG EAX, EDX
 elif ret_ecx:
 asm += \x91 # XCHG EAX, ECX
 asm += \xC3 # RET
-#code = cpu_info(\xB8\x01\x00\x00\x00 # MOV EAX, 1
-#\x53 # PUSH EBX
-#\x0F\xA2 # CPUID
-#\x5B # POP EBX
-#\x92 # XCHG EAX, EDX
-#\xC3 # RET
-#   )
 return cpu_info(asm)
 
 def detect_sse4_1(code=-1):
- use cpu_id_eax_1_ecx() to get code parameter 
 if code == -1:
-code = cpu_id(eax=1, ret_edx=False, ret_ecx=False)
+code = cpu_id(eax=1, ret_edx=False, ret_ecx=True)
 return bool(code  (119))
 
 def detect_sse4_2(code=-1):
- use cpu_id_eax_1_ecx() to get code parameter 
 if code == -1:
-code = cpu_id(eax=1, ret_edx=False, ret_ecx=False)
+code = cpu_id(eax=1, ret_edx=False, ret_ecx=True)
 return bool(code  (120))
 
 def detect_sse4a(code=-1):
- use cpu_id_eax_1_ecx() to get code parameter 
 if code == -1:
-code = feature.cpu_id(eax=0x8001, ret_edx=False, ret_ecx=True)
+code = cpu_id(eax=0x8001, ret_edx=False, ret_ecx=True)
 return bool(code  (120))
 
 def detect_x32_mode():
@@ -67,8 +58,13 @@
 
 if __name__ == '__main__':
 if detect_sse2():
-print 'Processor supports sse2.'
-else:
-print 'Missing processor support for sse2.'
+print 'Processor supports sse2'
+if detect_sse4_1():
+print 'Processor supports sse4.1'
+if detect_sse4_2():
+print 'Processor supports sse4.2'
+if detect_sse4a():
+print 'Processor supports sse4a'
+
 if detect_x32_mode():
 print 'Process is running in x32 mode.'
diff --git a/rpython/jit/backend/x86/runner.py 
b/rpython/jit/backend/x86/runner.py
--- a/rpython/jit/backend/x86/runner.py
+++ b/rpython/jit/backend/x86/runner.py
@@ -51,16 +51,6 @@
 
 self.profile_agent = profile_agent
 
-if self.supports_floats and self.supports_longlong:
-# has sse 2 at least
-

[pypy-commit] pypy cffi-1.0: ffi.addressof(lib, var)

2015-05-11 Thread arigo

Author: Armin Rigo ar...@tunes.org
Branch: cffi-1.0
Changeset: r77293:abb305ea8684
Date: 2015-05-11 15:55 +0200
http://bitbucket.org/pypy/pypy/changeset/abb305ea8684/

Log:ffi.addressof(lib, var)

diff --git a/pypy/module/_cffi_backend/cglob.py 
b/pypy/module/_cffi_backend/cglob.py
--- a/pypy/module/_cffi_backend/cglob.py
+++ b/pypy/module/_cffi_backend/cglob.py
@@ -1,6 +1,7 @@
 from pypy.interpreter.baseobjspace import W_Root
 from pypy.interpreter.typedef import TypeDef
 from pypy.module._cffi_backend.cdataobj import W_CData
+from pypy.module._cffi_backend import newtype
 
 
 class W_GlobSupport(W_Root):
@@ -15,5 +16,9 @@
 def write_global_var(self, w_newvalue):
 self.w_ctype.convert_from_object(self.ptr, w_newvalue)
 
+def address(self):
+w_ctypeptr = newtype.new_pointer_type(self.space, self.w_ctype)
+return W_CData(self.space, self.ptr, w_ctypeptr)
+
 W_GlobSupport.typedef = TypeDef(FFIGlobSupport)
 W_GlobSupport.typedef.acceptable_as_base_class = False
diff --git a/pypy/module/_cffi_backend/ffi_obj.py 
b/pypy/module/_cffi_backend/ffi_obj.py
--- a/pypy/module/_cffi_backend/ffi_obj.py
+++ b/pypy/module/_cffi_backend/ffi_obj.py
@@ -145,20 +145,34 @@
 
 def descr_addressof(self, w_arg, args_w):
 \
-With a single arg, return the address of a cdata 'struct-or-union'.
-If 'fields_or_indexes' are given, returns the address of that field or
-array item in the structure or array, recursively in case of nested
-structures.
+Limited equivalent to the '' operator in C:
+
+1. ffi.addressof(cdata 'struct-or-union') returns a cdata that is a
+pointer to this struct or union.
+
+2. ffi.addressof(cdata, field-or-index...) returns the address of a
+field or array item inside the given structure or array, recursively
+in case of nested structures or arrays.
+
+3. ffi.addressof(library, name) returns the address of the named
+global variable.
+#
+from pypy.module._cffi_backend.lib_obj import W_LibObject
+space = self.space
+if isinstance(w_arg, W_LibObject) and len(args_w) == 1:
+# case 3 in the docstring
+return w_arg.address_of_global_var(space.str_w(args_w[0]))
 #
 w_ctype = self.ffi_type(w_arg, ACCEPT_CDATA)
-space = self.space
 if len(args_w) == 0:
+# case 1 in the docstring
 if (not isinstance(w_ctype, ctypestruct.W_CTypeStructOrUnion) and
 not isinstance(w_ctype, ctypearray.W_CTypeArray)):
 raise oefmt(space.w_TypeError,
 expected a cdata struct/union/array object)
 offset = 0
 else:
+# case 2 in the docstring
 if (not isinstance(w_ctype, ctypestruct.W_CTypeStructOrUnion) and
 not isinstance(w_ctype, ctypearray.W_CTypeArray) and
 not isinstance(w_ctype, ctypeptr.W_CTypePointer)):
diff --git a/pypy/module/_cffi_backend/lib_obj.py 
b/pypy/module/_cffi_backend/lib_obj.py
--- a/pypy/module/_cffi_backend/lib_obj.py
+++ b/pypy/module/_cffi_backend/lib_obj.py
@@ -10,6 +10,7 @@
 from pypy.module._cffi_backend import cffi_opcode, cglob
 from pypy.module._cffi_backend.realize_c_type import getop, getarg
 from pypy.module._cffi_backend.cdataobj import W_CData
+from pypy.module._cffi_backend.ctypefunc import W_CTypeFunc
 from pypy.module._cffi_backend.structwrapper import W_StructWrapper
 
 
@@ -173,6 +174,24 @@
for i in range(total)]
 return space.newlist(names_w)
 
+def address_of_global_var(self, varname):
+# rebuild a string object from 'varname', to do typechecks and
+# to force a unicode back to a plain string
+space = self.space
+w_value = self._get_attr(space.wrap(varname))
+if isinstance(w_value, cglob.W_GlobSupport):
+# regular case: a global variable
+return w_value.address()
+#
+if ((isinstance(w_value, W_CData) and
+isinstance(w_value.ctype, W_CTypeFunc))
+or isinstance(w_value, W_StructWrapper)):
+# 'func' is 'func' in C, for a constant function 'func'
+return w_value
+#
+raise oefmt(space.w_AttributeError,
+cannot take the address of the constant '%s', varname)
+
 
 W_LibObject.typedef = TypeDef(
 'CompiledLib',
diff --git a/pypy/module/_cffi_backend/test/test_recompiler.py 
b/pypy/module/_cffi_backend/test/test_recompiler.py
--- a/pypy/module/_cffi_backend/test/test_recompiler.py
+++ b/pypy/module/_cffi_backend/test/test_recompiler.py
@@ -695,3 +695,37 @@
 assert repr(ffi.typeof(foo_t)) == ctype 'foo_t'
 assert repr(ffi.typeof(bar_p)) == ctype 'struct $1 *'
 assert repr(ffi.typeof(baz_pp)) == ctype 'struct $2 * *'
+
+def test_address_of_global_var(self):
+ffi, lib = self.prepare(
+long bottom, bottoms[2];
+long FetchRectBottom(void);
+long

[pypy-commit] cffi cffi-1.0: Reasonably messy to implement: ffi.addressof(lib, var)

2015-05-11 Thread arigo

Author: Armin Rigo ar...@tunes.org
Branch: cffi-1.0
Changeset: r1971:758808b32477
Date: 2015-05-11 15:36 +0200
http://bitbucket.org/cffi/cffi/changeset/758808b32477/

Log:Reasonably messy to implement: ffi.addressof(lib, var)

diff --git a/_cffi1/cglob.c b/_cffi1/cglob.c
--- a/_cffi1/cglob.c
+++ b/_cffi1/cglob.c
@@ -60,9 +60,13 @@
 return convert_from_object(gs-gs_data, gs-gs_type, obj);
 }
 
-#if 0
-static PyObject *addressof_global_var(GlobSupportObject *gs)
+static PyObject *cg_addressof_global_var(GlobSupportObject *gs)
 {
-return new_simple_cdata(gs-gs_data, gs-gs_type);
+PyObject *x, *ptrtype = new_pointer_type(gs-gs_type);
+if (ptrtype == NULL)
+return NULL;
+
+x = new_simple_cdata(gs-gs_data, (CTypeDescrObject *)ptrtype);
+Py_DECREF(ptrtype);
+return x;
 }
-#endif
diff --git a/_cffi1/ffi_obj.c b/_cffi1/ffi_obj.c
--- a/_cffi1/ffi_obj.c
+++ b/_cffi1/ffi_obj.c
@@ -19,6 +19,7 @@
 #define FFI_COMPLEXITY_OUTPUT   1200 /* xxx should grow as needed */
 
 #define FFIObject_Check(op) PyObject_TypeCheck(op, FFI_Type)
+#define LibObject_Check(ob)  ((Py_TYPE(ob) == Lib_Type))
 
 struct FFIObject_s {
 PyObject_HEAD
@@ -368,10 +369,19 @@
 }
 
 PyDoc_STRVAR(ffi_addressof_doc,
-With a single arg, return the address of a cdata 'struct-or-union'.\n
-If 'fields_or_indexes' are given, returns the address of that field or\n
-array item in the structure or array, recursively in case of nested\n
-structures.);
+Limited equivalent to the '' operator in C:\n
+\n
+1. ffi.addressof(cdata 'struct-or-union') returns a cdata that is a\n
+pointer to this struct or union.\n
+\n
+2. ffi.addressof(cdata, field-or-index...) returns the address of a\n
+field or array item inside the given structure or array, recursively\n
+in case of nested structures or arrays.\n
+\n
+3. ffi.addressof(library, \name\) returns the address of the named\n
+global variable.);
+
+static PyObject *address_of_global_var(PyObject *args);  /* forward */
 
 static PyObject *ffi_addressof(FFIObject *self, PyObject *args)
 {
@@ -387,11 +397,17 @@
 }
 
 arg = PyTuple_GET_ITEM(args, 0);
+if (LibObject_Check(arg)) {
+/* case 3 in the docstring */
+return address_of_global_var(args);
+}
+
 ct = _ffi_type(self, arg, ACCEPT_CDATA);
 if (ct == NULL)
 return NULL;
 
 if (PyTuple_GET_SIZE(args) == 1) {
+/* case 1 in the docstring */
 accepted_flags = CT_STRUCT | CT_UNION | CT_ARRAY;
 if ((ct-ct_flags  accepted_flags) == 0) {
 PyErr_SetString(PyExc_TypeError,
@@ -400,6 +416,7 @@
 }
 }
 else {
+/* case 2 in the docstring */
 accepted_flags = CT_STRUCT | CT_UNION | CT_ARRAY | CT_POINTER;
 if ((ct-ct_flags  accepted_flags) == 0) {
 PyErr_SetString(PyExc_TypeError,
diff --git a/_cffi1/lib_obj.c b/_cffi1/lib_obj.c
--- a/_cffi1/lib_obj.c
+++ b/_cffi1/lib_obj.c
@@ -27,15 +27,9 @@
 FFIObject *l_ffi;   /* reference back to the ffi object */
 };
 
-#define LibObject_Check(ob)  ((Py_TYPE(ob) == Lib_Type))
-
-static PyObject *_cpyextfunc_type_index(PyObject *x)
+static struct CPyExtFunc_s *_cpyextfunc_get(PyObject *x)
 {
 struct CPyExtFunc_s *exf;
-LibObject *lib;
-PyObject *tuple, *result;
-
-assert(PyErr_Occurred());
 
 if (!PyCFunction_Check(x))
 return NULL;
@@ -46,6 +40,20 @@
 if (exf-md.ml_doc != cpyextfunc_doc)
 return NULL;
 
+return exf;
+}
+
+static PyObject *_cpyextfunc_type_index(PyObject *x)
+{
+struct CPyExtFunc_s *exf;
+LibObject *lib;
+PyObject *tuple, *result;
+
+assert(PyErr_Occurred());
+exf = _cpyextfunc_get(x);
+if (exf == NULL)
+return NULL;/* still the same exception is set */
+
 PyErr_Clear();
 
 lib = (LibObject *)PyCFunction_GET_SELF(x);
@@ -269,14 +277,21 @@
 return x;
 }
 
+#define LIB_GET_OR_CACHE_ADDR(x, lib, name, error)  \
+do {\
+x = PyDict_GetItem(lib-l_dict, name);  \
+if (x == NULL) {\
+x = lib_build_and_cache_attr(lib, name, 0); \
+if (x == NULL) {\
+error;  \
+}   \
+}   \
+} while (0)
+
 static PyObject *lib_getattr(LibObject *lib, PyObject *name)
 {
-PyObject *x = PyDict_GetItem(lib-l_dict, name);
-if (x == NULL) {
-x = lib_build_and_cache_attr(lib, name, 0);
-if (x == NULL)
-return NULL;
-}
+PyObject *x;
+LIB_GET_OR_CACHE_ADDR(x, lib, name, return NULL);
 
 if (GlobSupport_Check(x)) {
 return read_global_var((GlobSupportObject *)x);
@@ -287,12 +302,8 @@
 
 static int lib_setattr(LibObject *lib, PyObject *name, PyObject *val)
 {
-PyObject *x = PyDict_GetItem(lib-l_dict, name);
-

[pypy-commit] cffi cffi-1.0: in-progress: ffi.dlopen()

2015-05-11 Thread arigo

Author: Armin Rigo ar...@tunes.org
Branch: cffi-1.0
Changeset: r1972:2148820bd1cb
Date: 2015-05-11 13:36 +0200
http://bitbucket.org/cffi/cffi/changeset/2148820bd1cb/

Log:in-progress: ffi.dlopen()

diff --git a/_cffi1/cdlopen.c b/_cffi1/cdlopen.c
new file mode 100644
--- /dev/null
+++ b/_cffi1/cdlopen.c
@@ -0,0 +1,57 @@
+/* ffi.dlopen() interface with dlopen()/dlsym()/dlclose() */
+
+static void *cdlopen_fetch(PyObject *libname, void *libhandle, char *symbol)
+{
+void *address;
+
+if (libhandle == NULL) {
+PyErr_Format(FFIError, library '%s' has been closed,
+ PyText_AS_UTF8(libname));
+return NULL;
+}
+
+address = dlsym(libhandle, symbol);
+if (address == NULL) {
+const char *error = dlerror();
+PyErr_Format(FFIError, symbol '%s' not found in library '%s': %s,
+ symbol, PyText_AS_UTF8(libname), error);
+}
+return address;
+}
+
+static int cdlopen_close(PyObject *libname, void *libhandle)
+{
+if (libhandle != NULL  dlclose(libhandle) != 0) {
+const char *error = dlerror();
+PyErr_Format(FFIError, closing library '%s': %s,
+ PyText_AS_UTF8(libname), error);
+return -1;
+}
+return 0;
+}
+
+
+
+static PyObject *ffi_dlclose(PyObject *self, PyObject *args)
+{
+LibObject *lib;
+if (!PyArg_ParseTuple(args, O!, Lib_Type, lib))
+return NULL;
+
+if (lib-l_libhandle == NULL) {
+PyErr_Format(FFIError, library '%s' is already closed 
+ or was not created with ffi.dlopen(),
+ PyText_AS_UTF8(lib-l_libhandle));
+return NULL;
+}
+
+if (cdlopen_close(lib-l_libname, lib-l_libhandle)  0)
+return NULL;
+
+/* Clear the dict to force further accesses to do cdlopen_fetch()
+   again, and fail because the library was closed. */
+PyDict_Clear(lib-l_dict);
+
+Py_INCREF(Py_None);
+return Py_None;
+}
diff --git a/_cffi1/cffi1_module.c b/_cffi1/cffi1_module.c
--- a/_cffi1/cffi1_module.c
+++ b/_cffi1/cffi1_module.c
@@ -12,6 +12,7 @@
 #include cglob.c
 #include cgc.c
 #include lib_obj.c
+#include cdlopen.c
 
 
 static int init_ffi_lib(PyObject *m)
@@ -169,7 +170,7 @@
 if (ffi == NULL || PyModule_AddObject(m, ffi, (PyObject *)ffi)  0)
 return NULL;
 
-lib = lib_internal_new(ffi, module_name);
+lib = lib_internal_new(ffi, module_name, NULL);
 if (lib == NULL || PyModule_AddObject(m, lib, (PyObject *)lib)  0)
 return NULL;
 
diff --git a/_cffi1/cffi_opcode.py b/_cffi1/cffi_opcode.py
--- a/_cffi1/cffi_opcode.py
+++ b/_cffi1/cffi_opcode.py
@@ -30,6 +30,7 @@
 OP_CONSTANT= 29
 OP_CONSTANT_INT= 31
 OP_GLOBAL_VAR  = 33
+OP_DLOPEN  = 35
 
 PRIM_VOID  = 0
 PRIM_BOOL  = 1
diff --git a/_cffi1/ffi_obj.c b/_cffi1/ffi_obj.c
--- a/_cffi1/ffi_obj.c
+++ b/_cffi1/ffi_obj.c
@@ -24,7 +24,7 @@
 PyObject_HEAD
 PyObject *gc_wrefs;
 struct _cffi_parse_info_s info;
-int ctx_is_static;
+char ctx_is_static, ctx_is_nonempty;
 builder_c_t types_builder;
 };
 
@@ -54,9 +54,7 @@
 ffi-info.output = internal_output;
 ffi-info.output_size = FFI_COMPLEXITY_OUTPUT;
 ffi-ctx_is_static = (static_ctx != NULL);
-#if 0
-ffi-dynamic_types = NULL;
-#endif
+ffi-ctx_is_nonempty = (static_ctx != NULL);
 return ffi;
 }
 
@@ -64,12 +62,8 @@
 {
 PyObject_GC_UnTrack(ffi);
 Py_XDECREF(ffi-gc_wrefs);
-#if 0
-Py_XDECREF(ffi-dynamic_types);
-#endif
 
-if (!ffi-ctx_is_static)
-free_dynamic_builder_c(ffi-types_builder);
+free_builder_c(ffi-types_builder, ffi-ctx_is_static);
 
 Py_TYPE(ffi)-tp_free((PyObject *)ffi);
 }
@@ -90,9 +84,31 @@
 
 static int ffiobj_init(PyObject *self, PyObject *args, PyObject *kwds)
 {
-static char *keywords[] = {NULL};
-if (!PyArg_ParseTupleAndKeywords(args, kwds, :FFI, keywords))
+FFIObject *ffi;
+static char *keywords[] = {module_name, _version, _types,
+   _globals, _struct_unions, _enums,
+   _typenames, _consts, NULL};
+char *ffiname = NULL, *types = NULL;
+Py_ssize_t version = -1;
+Py_ssize_t types_len = 0;
+PyObject *globals = NULL, *struct_unions = NULL, *enums = NULL;
+PyObject *typenames = NULL, *consts = NULL;
+
+if (!PyArg_ParseTupleAndKeywords(args, kwds, |sns#O:FFI, keywords,
+ ffiname, version, types, types_len,
+ globals, struct_unions, enums,
+ typenames, consts))
 return -1;
+
+ffi = (FFIObject *)self;
+if (ffi-ctx_is_nonempty) {
+PyErr_SetString(PyExc_ValueError,
+cannot call FFI.__init__() more than once);
+return -1;
+}
+
+//...;
+ffi-ctx_is_nonempty = 1;
 return 0;
 }
 
@@ -671,6 +687,23 @@
 return 0;
 }
 
+PyDoc_STRVAR(ffi_dlopen_doc,
+Load and

[pypy-commit] cffi cffi-1.0: Just enough to pass this small demo

2015-05-11 Thread arigo

Author: Armin Rigo ar...@tunes.org
Branch: cffi-1.0
Changeset: r1975:dbbbc44e0a15
Date: 2015-05-11 17:24 +0200
http://bitbucket.org/cffi/cffi/changeset/dbbbc44e0a15/

Log:Just enough to pass this small demo

diff --git a/_cffi1/cdlopen.c b/_cffi1/cdlopen.c
--- a/_cffi1/cdlopen.c
+++ b/_cffi1/cdlopen.c
@@ -20,6 +20,12 @@
 return address;
 }
 
+static void cdlopen_close_ignore_errors(void *libhandle)
+{
+if (libhandle != NULL)
+dlclose(libhandle);
+}
+
 static int cdlopen_close(PyObject *libname, void *libhandle)
 {
 if (libhandle != NULL  dlclose(libhandle) != 0) {
@@ -31,7 +37,38 @@
 return 0;
 }
 
+static PyObject *ffi_dlopen(PyObject *self, PyObject *args)
+{
+char *filename_or_null, *printable_filename;
+void *handle;
+int flags = 0;
 
+if (PyTuple_GET_SIZE(args) == 0 || PyTuple_GET_ITEM(args, 0) == Py_None) {
+PyObject *dummy;
+if (!PyArg_ParseTuple(args, |Oi:load_library,
+  dummy, flags))
+return NULL;
+filename_or_null = NULL;
+}
+else if (!PyArg_ParseTuple(args, et|i:load_library,
+  Py_FileSystemDefaultEncoding, filename_or_null,
+  flags))
+return NULL;
+
+if ((flags  (RTLD_NOW | RTLD_LAZY)) == 0)
+flags |= RTLD_NOW;
+printable_filename = filename_or_null ? filename_or_null : None;
+
+handle = dlopen(filename_or_null, flags);
+if (handle == NULL) {
+const char *error = dlerror();
+PyErr_Format(PyExc_OSError, cannot load library '%s': %s,
+ printable_filename, error);
+return NULL;
+}
+return (PyObject *)lib_internal_new((FFIObject *)self,
+printable_filename, handle);
+}
 
 static PyObject *ffi_dlclose(PyObject *self, PyObject *args)
 {
@@ -56,3 +93,101 @@
 Py_INCREF(Py_None);
 return Py_None;
 }
+
+
+static int cdl_int(char *src)
+{
+unsigned char *usrc = (unsigned char *)src;
+return (usrc[0]  24) | (usrc[1]  16) | (usrc[2]  8) | usrc[3];
+}
+
+static _cffi_opcode_t cdl_opcode(char *src)
+{
+return (_cffi_opcode_t)(Py_ssize_t)cdl_int(src);
+}
+
+static int ffiobj_init(PyObject *self, PyObject *args, PyObject *kwds)
+{
+FFIObject *ffi;
+static char *keywords[] = {module_name, _version, _types,
+   _globals, _struct_unions, _enums,
+   _typenames, _consts, NULL};
+char *ffiname = NULL, *types = NULL, *building = NULL;
+Py_ssize_t version = -1;
+Py_ssize_t types_len = 0;
+PyObject *globals = NULL, *struct_unions = NULL, *enums = NULL;
+PyObject *typenames = NULL, *consts = NULL;
+
+if (!PyArg_ParseTupleAndKeywords(args, kwds, |sns#O!:FFI, keywords,
+ ffiname, version, types, types_len,
+ PyTuple_Type, globals,
+ struct_unions, enums,
+ typenames, consts))
+return -1;
+
+ffi = (FFIObject *)self;
+if (ffi-ctx_is_nonempty) {
+PyErr_SetString(PyExc_ValueError,
+cannot call FFI.__init__() more than once);
+return -1;
+}
+ffi-ctx_is_nonempty = 1;
+
+if (types_len  0) {
+_cffi_opcode_t *ntypes;
+Py_ssize_t i, n = types_len / 4;/* 4 bytes entries */
+
+building = PyMem_Malloc(n * sizeof(_cffi_opcode_t));
+if (building == NULL)
+goto error;
+ntypes = (_cffi_opcode_t *)building;
+
+for (i = 0; i  n; i++) {
+ntypes[i] = cdl_opcode(types);
+types += 4;
+}
+ffi-types_builder.ctx.types = ntypes;
+building = NULL;
+}
+
+if (globals != NULL) {
+struct _cffi_global_s *nglob;
+Py_ssize_t i, n = PyTuple_GET_SIZE(globals);
+
+building = PyMem_Malloc(n * sizeof(struct _cffi_global_s));
+if (building == NULL)
+goto error;
+memset(building, 0, n * sizeof(struct _cffi_global_s));
+nglob = (struct _cffi_global_s *)building;
+
+for (i = 0; i  n; i++) {
+char *g = PyString_AS_STRING(PyTuple_GET_ITEM(globals, i));
+nglob[i].type_op = cdl_opcode(g);
+nglob[i].name = g + 4;
+}
+ffi-types_builder.ctx.globals = nglob;
+ffi-types_builder.ctx.num_globals = n;
+building = NULL;
+}
+
+if (consts != NULL) {
+Py_INCREF(consts);
+ffi-types_builder.known_constants = consts;
+}
+
+/* Above, we took directly some char * strings out of the strings,
+   typically from somewhere inside tuples.  Keep them alive by
+   incref'ing the whole input arguments. */
+Py_INCREF(args);
+Py_XINCREF(kwds);
+ffi-types_builder._keepalive1 = args;
+ffi-types_builder._keepalive2 = kwds;
+return 0;
+
+ error:
+if (building != NULL)
+

[pypy-commit] cffi cffi-1.0: in-progress

2015-05-11 Thread arigo

Author: Armin Rigo ar...@tunes.org
Branch: cffi-1.0
Changeset: r1973:45d8b5f1fbd0
Date: 2015-05-11 15:00 +0200
http://bitbucket.org/cffi/cffi/changeset/45d8b5f1fbd0/

Log:in-progress

diff --git a/TODO b/TODO
--- a/TODO
+++ b/TODO
@@ -13,3 +13,6 @@
 * mention todo: ffi.new(xyz) makes {xyz: ctype} always immortal
 
 * mention todo: dlopen(), by compiling a cdef()-only FFI into a .py module
+
+* ffi.set_source() produces a C file that is entirely independent on
+  the OS, what is installed, and the current Python version
diff --git a/_cffi1/cdlopen.c b/_cffi1/cdlopen.c
--- a/_cffi1/cdlopen.c
+++ b/_cffi1/cdlopen.c
@@ -10,6 +10,7 @@
 return NULL;
 }
 
+dlerror();   /* clear error condition */
 address = dlsym(libhandle, symbol);
 if (address == NULL) {
 const char *error = dlerror();
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] cffi cffi-1.0: merge heads

2015-05-11 Thread arigo

Author: Armin Rigo ar...@tunes.org
Branch: cffi-1.0
Changeset: r1974:56fc30b8d7e1
Date: 2015-05-11 15:59 +0200
http://bitbucket.org/cffi/cffi/changeset/56fc30b8d7e1/

Log:merge heads

diff --git a/TODO b/TODO
--- a/TODO
+++ b/TODO
@@ -13,3 +13,6 @@
 * mention todo: ffi.new(xyz) makes {xyz: ctype} always immortal
 
 * mention todo: dlopen(), by compiling a cdef()-only FFI into a .py module
+
+* ffi.set_source() produces a C file that is entirely independent on
+  the OS, what is installed, and the current Python version
diff --git a/_cffi1/cdlopen.c b/_cffi1/cdlopen.c
new file mode 100644
--- /dev/null
+++ b/_cffi1/cdlopen.c
@@ -0,0 +1,58 @@
+/* ffi.dlopen() interface with dlopen()/dlsym()/dlclose() */
+
+static void *cdlopen_fetch(PyObject *libname, void *libhandle, char *symbol)
+{
+void *address;
+
+if (libhandle == NULL) {
+PyErr_Format(FFIError, library '%s' has been closed,
+ PyText_AS_UTF8(libname));
+return NULL;
+}
+
+dlerror();   /* clear error condition */
+address = dlsym(libhandle, symbol);
+if (address == NULL) {
+const char *error = dlerror();
+PyErr_Format(FFIError, symbol '%s' not found in library '%s': %s,
+ symbol, PyText_AS_UTF8(libname), error);
+}
+return address;
+}
+
+static int cdlopen_close(PyObject *libname, void *libhandle)
+{
+if (libhandle != NULL  dlclose(libhandle) != 0) {
+const char *error = dlerror();
+PyErr_Format(FFIError, closing library '%s': %s,
+ PyText_AS_UTF8(libname), error);
+return -1;
+}
+return 0;
+}
+
+
+
+static PyObject *ffi_dlclose(PyObject *self, PyObject *args)
+{
+LibObject *lib;
+if (!PyArg_ParseTuple(args, O!, Lib_Type, lib))
+return NULL;
+
+if (lib-l_libhandle == NULL) {
+PyErr_Format(FFIError, library '%s' is already closed 
+ or was not created with ffi.dlopen(),
+ PyText_AS_UTF8(lib-l_libhandle));
+return NULL;
+}
+
+if (cdlopen_close(lib-l_libname, lib-l_libhandle)  0)
+return NULL;
+
+/* Clear the dict to force further accesses to do cdlopen_fetch()
+   again, and fail because the library was closed. */
+PyDict_Clear(lib-l_dict);
+
+Py_INCREF(Py_None);
+return Py_None;
+}
diff --git a/_cffi1/cffi1_module.c b/_cffi1/cffi1_module.c
--- a/_cffi1/cffi1_module.c
+++ b/_cffi1/cffi1_module.c
@@ -12,6 +12,7 @@
 #include cglob.c
 #include cgc.c
 #include lib_obj.c
+#include cdlopen.c
 
 
 static int init_ffi_lib(PyObject *m)
@@ -169,7 +170,7 @@
 if (ffi == NULL || PyModule_AddObject(m, ffi, (PyObject *)ffi)  0)
 return NULL;
 
-lib = lib_internal_new(ffi, module_name);
+lib = lib_internal_new(ffi, module_name, NULL);
 if (lib == NULL || PyModule_AddObject(m, lib, (PyObject *)lib)  0)
 return NULL;
 
diff --git a/_cffi1/cffi_opcode.py b/_cffi1/cffi_opcode.py
--- a/_cffi1/cffi_opcode.py
+++ b/_cffi1/cffi_opcode.py
@@ -30,6 +30,7 @@
 OP_CONSTANT= 29
 OP_CONSTANT_INT= 31
 OP_GLOBAL_VAR  = 33
+OP_DLOPEN  = 35
 
 PRIM_VOID  = 0
 PRIM_BOOL  = 1
diff --git a/_cffi1/ffi_obj.c b/_cffi1/ffi_obj.c
--- a/_cffi1/ffi_obj.c
+++ b/_cffi1/ffi_obj.c
@@ -25,7 +25,7 @@
 PyObject_HEAD
 PyObject *gc_wrefs;
 struct _cffi_parse_info_s info;
-int ctx_is_static;
+char ctx_is_static, ctx_is_nonempty;
 builder_c_t types_builder;
 };
 
@@ -55,9 +55,7 @@
 ffi-info.output = internal_output;
 ffi-info.output_size = FFI_COMPLEXITY_OUTPUT;
 ffi-ctx_is_static = (static_ctx != NULL);
-#if 0
-ffi-dynamic_types = NULL;
-#endif
+ffi-ctx_is_nonempty = (static_ctx != NULL);
 return ffi;
 }
 
@@ -65,12 +63,8 @@
 {
 PyObject_GC_UnTrack(ffi);
 Py_XDECREF(ffi-gc_wrefs);
-#if 0
-Py_XDECREF(ffi-dynamic_types);
-#endif
 
-if (!ffi-ctx_is_static)
-free_dynamic_builder_c(ffi-types_builder);
+free_builder_c(ffi-types_builder, ffi-ctx_is_static);
 
 Py_TYPE(ffi)-tp_free((PyObject *)ffi);
 }
@@ -91,9 +85,31 @@
 
 static int ffiobj_init(PyObject *self, PyObject *args, PyObject *kwds)
 {
-static char *keywords[] = {NULL};
-if (!PyArg_ParseTupleAndKeywords(args, kwds, :FFI, keywords))
+FFIObject *ffi;
+static char *keywords[] = {module_name, _version, _types,
+   _globals, _struct_unions, _enums,
+   _typenames, _consts, NULL};
+char *ffiname = NULL, *types = NULL;
+Py_ssize_t version = -1;
+Py_ssize_t types_len = 0;
+PyObject *globals = NULL, *struct_unions = NULL, *enums = NULL;
+PyObject *typenames = NULL, *consts = NULL;
+
+if (!PyArg_ParseTupleAndKeywords(args, kwds, |sns#O:FFI, keywords,
+ ffiname, version, types, types_len,
+ globals, struct_unions, enums,
+ typenames, consts))

[pypy-commit] pypy fix-result-types: extract W_Ufunc1.find_specialization()

2015-05-11 Thread rlamy

Author: Ronan Lamy ronan.l...@gmail.com
Branch: fix-result-types
Changeset: r77294:3ba5bdc3be93
Date: 2015-05-11 18:56 +0100
http://bitbucket.org/pypy/pypy/changeset/3ba5bdc3be93/

Log:extract W_Ufunc1.find_specialization()

diff --git a/pypy/module/micronumpy/ufuncs.py b/pypy/module/micronumpy/ufuncs.py
--- a/pypy/module/micronumpy/ufuncs.py
+++ b/pypy/module/micronumpy/ufuncs.py
@@ -337,18 +337,7 @@
 out = None
 w_obj = numpify(space, w_obj)
 dtype = w_obj.get_dtype(space)
-if dtype.is_flexible():
-raise OperationError(space.w_TypeError,
-  space.wrap('Not implemented for this type'))
-if (self.int_only and not (dtype.is_int() or dtype.is_object()) or
-not self.allow_bool and dtype.is_bool() or
-not self.allow_complex and dtype.is_complex()):
-raise oefmt(space.w_TypeError,
-ufunc %s not supported for the input type, self.name)
-calc_dtype = find_unaryop_result_dtype(space,
-  dtype,
-  promote_to_float=self.promote_to_float,
-  promote_bools=self.promote_bools)
+calc_dtype, func = self.find_specialization(space, dtype)
 if out is not None:
 if not isinstance(out, W_NDimArray):
 raise oefmt(space.w_TypeError, 'output must be an array')
@@ -371,7 +360,7 @@
 assert isinstance(w_obj, W_NDimArray)
 shape = shape_agreement(space, w_obj.get_shape(), out,
 broadcast_down=False)
-return loop.call1(space, shape, self.func, calc_dtype, res_dtype,
+return loop.call1(space, shape, func, calc_dtype, res_dtype,
   w_obj, out)
 
 def call_scalar(self, space, w_arg, in_dtype, out_dtype, out):
@@ -387,6 +376,20 @@
 out.fill(space, w_val)
 return out
 
+def find_specialization(self, space, dtype):
+if dtype.is_flexible():
+raise oefmt(space.w_TypeError, 'Not implemented for this type')
+if (self.int_only and not (dtype.is_int() or dtype.is_object()) or
+not self.allow_bool and dtype.is_bool() or
+not self.allow_complex and dtype.is_complex()):
+raise oefmt(space.w_TypeError,
+ufunc %s not supported for the input type, self.name)
+calc_dtype = find_unaryop_result_dtype(space,
+  dtype,
+  promote_to_float=self.promote_to_float,
+  promote_bools=self.promote_bools)
+return calc_dtype, self.func
+
 
 class W_Ufunc2(W_Ufunc):
 _immutable_fields_ = [func, comparison_func, done_func]
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] pypy fix-result-types: Use the same logic as cnumpy in W_Ufunc1.find_specialization()

2015-05-11 Thread rlamy

Author: Ronan Lamy ronan.l...@gmail.com
Branch: fix-result-types
Changeset: r77296:2fc8c1b68f07
Date: 2015-05-12 06:16 +0100
http://bitbucket.org/pypy/pypy/changeset/2fc8c1b68f07/

Log:Use the same logic as cnumpy in W_Ufunc1.find_specialization()

diff --git a/pypy/module/micronumpy/descriptor.py 
b/pypy/module/micronumpy/descriptor.py
--- a/pypy/module/micronumpy/descriptor.py
+++ b/pypy/module/micronumpy/descriptor.py
@@ -900,17 +900,20 @@
 NPY.CDOUBLE: self.w_float64dtype,
 NPY.CLONGDOUBLE: self.w_floatlongdtype,
 }
-self.builtin_dtypes = [
-self.w_booldtype,
+integer_dtypes = [
 self.w_int8dtype, self.w_uint8dtype,
 self.w_int16dtype, self.w_uint16dtype,
+self.w_int32dtype, self.w_uint32dtype,
 self.w_longdtype, self.w_ulongdtype,
-self.w_int32dtype, self.w_uint32dtype,
-self.w_int64dtype, self.w_uint64dtype,
-] + float_dtypes + complex_dtypes + [
-self.w_stringdtype, self.w_unicodedtype, self.w_voiddtype,
-self.w_objectdtype,
-]
+self.w_int64dtype, self.w_uint64dtype]
+self.builtin_dtypes = ([self.w_booldtype] + integer_dtypes +
+float_dtypes + complex_dtypes + [
+self.w_stringdtype, self.w_unicodedtype, self.w_voiddtype,
+self.w_objectdtype,
+])
+self.integer_dtypes = integer_dtypes
+self.float_dtypes = float_dtypes
+self.complex_dtypes = complex_dtypes
 self.float_dtypes_by_num_bytes = sorted(
 (dtype.elsize, dtype)
 for dtype in float_dtypes
diff --git a/pypy/module/micronumpy/test/test_ufuncs.py 
b/pypy/module/micronumpy/test/test_ufuncs.py
--- a/pypy/module/micronumpy/test/test_ufuncs.py
+++ b/pypy/module/micronumpy/test/test_ufuncs.py
@@ -1,5 +1,5 @@
 from pypy.module.micronumpy.test.test_base import BaseNumpyAppTest
-from pypy.module.micronumpy.ufuncs import W_UfuncGeneric
+from pypy.module.micronumpy.ufuncs import W_UfuncGeneric, W_Ufunc1
 from pypy.module.micronumpy.support import _parse_signature
 from pypy.module.micronumpy.descriptor import get_dtype_cache
 from pypy.module.micronumpy.base import W_NDimArray
@@ -54,6 +54,20 @@
 exc = raises(OperationError, ufunc.type_resolver, space, [f32_array], 
[None],
 'i-i', ufunc.dtypes)
 
+def test_allowed_types(self, space):
+dt_bool = get_dtype_cache(space).w_booldtype
+dt_float16 = get_dtype_cache(space).w_float16dtype
+dt_int32 = get_dtype_cache(space).w_int32dtype
+ufunc = W_Ufunc1(None, 'x', int_only=True)
+assert ufunc._calc_dtype(space, dt_bool) == dt_bool
+assert ufunc.allowed_types(space)  # XXX: shouldn't contain too much 
stuff
+
+ufunc = W_Ufunc1(None, 'x', promote_to_float=True)
+assert ufunc._calc_dtype(space, dt_bool) == dt_float16
+
+ufunc = W_Ufunc1(None, 'x')
+assert ufunc._calc_dtype(space, dt_int32) == dt_int32
+
 class AppTestUfuncs(BaseNumpyAppTest):
 def test_constants(self):
 import numpy as np
diff --git a/pypy/module/micronumpy/ufuncs.py b/pypy/module/micronumpy/ufuncs.py
--- a/pypy/module/micronumpy/ufuncs.py
+++ b/pypy/module/micronumpy/ufuncs.py
@@ -18,7 +18,8 @@
 from pypy.module.micronumpy.nditer import W_NDIter, coalesce_iter
 from pypy.module.micronumpy.strides import shape_agreement
 from pypy.module.micronumpy.support import _parse_signature, product, 
get_storage_as_int
-from .casting import find_unaryop_result_dtype, find_binop_result_dtype
+from .casting import (
+find_unaryop_result_dtype, find_binop_result_dtype, can_cast_type)
 
 def done_if_true(dtype, val):
 return dtype.itemtype.bool(val)
@@ -384,12 +385,36 @@
 not self.allow_complex and dtype.is_complex()):
 raise oefmt(space.w_TypeError,
 ufunc %s not supported for the input type, self.name)
-calc_dtype = find_unaryop_result_dtype(space,
-  dtype,
-  promote_to_float=self.promote_to_float,
-  promote_bools=self.promote_bools)
+calc_dtype = self._calc_dtype(space, dtype)
 return calc_dtype, self.func
 
+def _calc_dtype(self, space, arg_dtype):
+use_min_scalar=False
+if arg_dtype.is_object():
+return arg_dtype
+for dtype in self.allowed_types(space):
+if use_min_scalar:
+if can_cast_array(space, w_arg, dtype, casting='safe'):
+return dtype
+else:
+if can_cast_type(space, arg_dtype, dtype, casting='safe'):
+return dtype
+else:
+raise oefmt(space.w_TypeError,
+No loop matching the specified signature was found 
+for ufunc %s, self.name)
+
+def

[pypy-commit] pypy fix-result-types: Fix casting table

2015-05-11 Thread rlamy

Author: Ronan Lamy ronan.l...@gmail.com
Branch: fix-result-types
Changeset: r77295:5436f4d8b598
Date: 2015-05-12 05:45 +0100
http://bitbucket.org/pypy/pypy/changeset/5436f4d8b598/

Log:Fix casting table

diff --git a/pypy/module/micronumpy/casting.py 
b/pypy/module/micronumpy/casting.py
--- a/pypy/module/micronumpy/casting.py
+++ b/pypy/module/micronumpy/casting.py
@@ -90,7 +90,7 @@
 if origin.kind in kind_ordering and target.kind in kind_ordering:
 return kind_ordering[origin.kind] = kind_ordering[target.kind]
 return False
-else:
+else:  # 'safe'
 return origin.can_cast_to(target)
 
 def can_cast_array(space, w_from, target, casting):
diff --git a/pypy/module/micronumpy/test/test_casting.py 
b/pypy/module/micronumpy/test/test_casting.py
--- a/pypy/module/micronumpy/test/test_casting.py
+++ b/pypy/module/micronumpy/test/test_casting.py
@@ -1,7 +1,7 @@
 from pypy.module.micronumpy.test.test_base import BaseNumpyAppTest
 from pypy.module.micronumpy.descriptor import get_dtype_cache
 from pypy.module.micronumpy.casting import (
-find_unaryop_result_dtype, find_binop_result_dtype)
+find_unaryop_result_dtype, find_binop_result_dtype, can_cast_type)
 
 
 class AppTestNumSupport(BaseNumpyAppTest):
@@ -27,6 +27,7 @@
 assert np.can_cast(np.int32, np.int64)
 assert np.can_cast(np.float64, complex)
 assert not np.can_cast(np.complex64, float)
+assert np.can_cast(np.bool_, np.bool_)
 
 assert np.can_cast('i8', 'f8')
 assert not np.can_cast('i8', 'f4')
@@ -123,6 +124,15 @@
 # XXX: np.asarray(2**64) fails with OverflowError
 # assert np.min_scalar_type(2**64) == np.dtype('O')
 
+def test_can_cast_same_type(space):
+dt_bool = get_dtype_cache(space).w_booldtype
+assert can_cast_type(space, dt_bool, dt_bool, 'no')
+assert can_cast_type(space, dt_bool, dt_bool, 'equiv')
+assert can_cast_type(space, dt_bool, dt_bool, 'safe')
+assert can_cast_type(space, dt_bool, dt_bool, 'same_kind')
+assert can_cast_type(space, dt_bool, dt_bool, 'unsafe')
+
+
 class TestCoercion(object):
 def test_binops(self, space):
 bool_dtype = get_dtype_cache(space).w_booldtype
diff --git a/pypy/module/micronumpy/types.py b/pypy/module/micronumpy/types.py
--- a/pypy/module/micronumpy/types.py
+++ b/pypy/module/micronumpy/types.py
@@ -2439,7 +2439,7 @@
 
 casting_table = [[False] * NPY.NTYPES for _ in range(NPY.NTYPES)]
 number_types = int_types + float_types + complex_types
-all_types = number_types + [ObjectType, StringType, UnicodeType, VoidType]
+all_types = [Bool] + number_types + [ObjectType, StringType, UnicodeType, 
VoidType]
 
 def enable_cast(type1, type2):
 casting_table[type1.num][type2.num] = True
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] cffi cffi-1.0: struct/unions

2015-05-11 Thread arigo

Author: Armin Rigo ar...@tunes.org
Branch: cffi-1.0
Changeset: r1976:73400152d574
Date: 2015-05-11 17:55 +0200
http://bitbucket.org/cffi/cffi/changeset/73400152d574/

Log:struct/unions

diff --git a/_cffi1/cdlopen.c b/_cffi1/cdlopen.c
--- a/_cffi1/cdlopen.c
+++ b/_cffi1/cdlopen.c
@@ -134,8 +134,9 @@
 ffi-ctx_is_nonempty = 1;
 
 if (types_len  0) {
+/* unpack a string of 4-byte entries into an array of _cffi_opcode_t */
 _cffi_opcode_t *ntypes;
-Py_ssize_t i, n = types_len / 4;/* 4 bytes entries */
+Py_ssize_t i, n = types_len / 4;
 
 building = PyMem_Malloc(n * sizeof(_cffi_opcode_t));
 if (building == NULL)
@@ -147,29 +148,95 @@
 types += 4;
 }
 ffi-types_builder.ctx.types = ntypes;
+ffi-types_builder.ctx.num_types = n;
 building = NULL;
 }
 
 if (globals != NULL) {
-struct _cffi_global_s *nglob;
+/* unpack a tuple of strings, each of which describes one global_s
+   entry with no specified address or size */
+struct _cffi_global_s *nglobs;
 Py_ssize_t i, n = PyTuple_GET_SIZE(globals);
 
-building = PyMem_Malloc(n * sizeof(struct _cffi_global_s));
+i = n * sizeof(struct _cffi_global_s);
+building = PyMem_Malloc(i);
 if (building == NULL)
 goto error;
-memset(building, 0, n * sizeof(struct _cffi_global_s));
-nglob = (struct _cffi_global_s *)building;
+memset(building, 0, i);
+nglobs = (struct _cffi_global_s *)building;
 
 for (i = 0; i  n; i++) {
 char *g = PyString_AS_STRING(PyTuple_GET_ITEM(globals, i));
-nglob[i].type_op = cdl_opcode(g);
-nglob[i].name = g + 4;
+nglobs[i].type_op = cdl_opcode(g);
+nglobs[i].name = g + 4;
 }
-ffi-types_builder.ctx.globals = nglob;
+ffi-types_builder.ctx.globals = nglobs;
 ffi-types_builder.ctx.num_globals = n;
 building = NULL;
 }
 
+if (struct_unions != NULL) {
+/* unpack a tuple of struct/unions, each described as a sub-tuple;
+   the item 0 of each sub-tuple describes the struct/union, and
+   the items 1..N-1 describe the fields, if any */
+struct _cffi_struct_union_s *nstructs;
+struct _cffi_field_s *nfields;
+Py_ssize_t i, n = PyTuple_GET_SIZE(struct_unions);
+Py_ssize_t nf = 0;   /* total number of fields */
+
+for (i = 0; i  n; i++) {
+nf += PyTuple_GET_SIZE(PyTuple_GET_ITEM(struct_unions, i)) - 1;
+}
+i = (n * sizeof(struct _cffi_struct_union_s) +
+ nf * sizeof(struct _cffi_field_s));
+building = PyMem_Malloc(i);
+if (building == NULL)
+goto error;
+memset(building, 0, i);
+nstructs = (struct _cffi_struct_union_s *)building;
+nfields = (struct _cffi_field_s *)(nstructs + n);
+nf = 0;
+
+for (i = 0; i  n; i++) {
+/* 'desc' is the tuple of strings (desc_struct, desc_field_1, ..) 
*/
+PyObject *desc = PyTuple_GET_ITEM(struct_unions, i);
+Py_ssize_t j, nf1 = PyTuple_GET_SIZE(desc) - 1;
+char *s = PyString_AS_STRING(PyTuple_GET_ITEM(desc, 0));
+/* 's' is the first string, describing the struct/union */
+nstructs[i].type_index = cdl_int(s);
+nstructs[i].flags = cdl_int(s + 4);
+nstructs[i].name = s + 8;
+if (nstructs[i].flags  _CFFI_F_OPAQUE) {
+nstructs[i].size = (size_t)-1;
+nstructs[i].alignment = -1;
+nstructs[i].first_field_index = -1;
+nstructs[i].num_fields = 0;
+assert(nf1 == 0);
+}
+else {
+nstructs[i].size = (size_t)-2;
+nstructs[i].alignment = -2;
+nstructs[i].first_field_index = nf;
+nstructs[i].num_fields = nf1;
+}
+for (j = 0; j  nf1; j++) {
+char *f = PyString_AS_STRING(PyTuple_GET_ITEM(desc, j + 1));
+/* 'f' is one of the other strings beyond the first one,
+   describing one field each */
+nfields[nf].field_type_op = cdl_opcode(f);
+nfields[nf].name = f + 4;
+nfields[nf].field_offset = (size_t)-1;
+nfields[nf].field_size = (size_t)-1;
+/* XXX BITFIELD MISSING  */
+nf++;
+}
+}
+ffi-types_builder.ctx.struct_unions = nstructs;
+ffi-types_builder.ctx.fields = nfields;
+ffi-types_builder.ctx.num_struct_unions = n;
+building = NULL;
+}
+
 if (consts != NULL) {
 Py_INCREF(consts);
 ffi-types_builder.known_constants = consts;
diff --git a/_cffi1/ffi_obj.c b/_cffi1/ffi_obj.c
--- a/_cffi1/ffi_obj.c
+++

[pypy-commit] cffi cffi-1.0: enums, integer constants

2015-05-11 Thread arigo

Author: Armin Rigo ar...@tunes.org
Branch: cffi-1.0
Changeset: r1977:f3745cfa00a0
Date: 2015-05-11 19:30 +0200
http://bitbucket.org/cffi/cffi/changeset/f3745cfa00a0/

Log:enums, integer constants

diff --git a/_cffi1/cdlopen.c b/_cffi1/cdlopen.c
--- a/_cffi1/cdlopen.c
+++ b/_cffi1/cdlopen.c
@@ -95,15 +95,38 @@
 }
 
 
-static int cdl_int(char *src)
+static Py_ssize_t cdl_4bytes(char *src)
 {
+/* read 4 bytes in little-endian order; return it as a signed integer */
+signed char *ssrc = (signed char *)src;
 unsigned char *usrc = (unsigned char *)src;
-return (usrc[0]  24) | (usrc[1]  16) | (usrc[2]  8) | usrc[3];
+return (ssrc[0]  24) | (usrc[1]  16) | (usrc[2]  8) | usrc[3];
 }
 
 static _cffi_opcode_t cdl_opcode(char *src)
 {
-return (_cffi_opcode_t)(Py_ssize_t)cdl_int(src);
+return (_cffi_opcode_t)cdl_4bytes(src);
+}
+
+typedef struct {
+unsigned long long value;
+int neg;
+} cdl_intconst_t;
+
+int _cdl_realize_global_int(struct _cffi_getconst_s *gc)
+{
+/* The 'address' field of 'struct _cffi_global_s' is set to point
+   to this function in case ffiobj_init() sees constant integers.
+   This fishes around after the 'ctx-globals' array, which is
+   initialized to contain another array, this time of
+   'cdl_intconst_t' structures.  We get the nth one and it tells
+   us what to return.
+*/
+cdl_intconst_t *ic;
+ic = (cdl_intconst_t *)(gc-ctx-globals + gc-ctx-num_globals);
+ic += gc-gindex;
+gc-value = ic-value;
+return ic-neg;
 }
 
 static int ffiobj_init(PyObject *self, PyObject *args, PyObject *kwds)
@@ -111,18 +134,19 @@
 FFIObject *ffi;
 static char *keywords[] = {module_name, _version, _types,
_globals, _struct_unions, _enums,
-   _typenames, _consts, NULL};
+   _typenames, NULL};
 char *ffiname = NULL, *types = NULL, *building = NULL;
 Py_ssize_t version = -1;
 Py_ssize_t types_len = 0;
 PyObject *globals = NULL, *struct_unions = NULL, *enums = NULL;
-PyObject *typenames = NULL, *consts = NULL;
+PyObject *typenames = NULL;
 
-if (!PyArg_ParseTupleAndKeywords(args, kwds, |sns#O!:FFI, keywords,
+if (!PyArg_ParseTupleAndKeywords(args, kwds, |sns#O!O!O!O!:FFI, keywords,
  ffiname, version, types, types_len,
  PyTuple_Type, globals,
- struct_unions, enums,
- typenames, consts))
+ PyTuple_Type, struct_unions,
+ PyTuple_Type, enums,
+ PyTuple_Type, typenames))
 return -1;
 
 ffi = (FFIObject *)self;
@@ -153,22 +177,41 @@
 }
 
 if (globals != NULL) {
-/* unpack a tuple of strings, each of which describes one global_s
-   entry with no specified address or size */
+/* unpack a tuple alternating strings and ints, each two together
+   describing one global_s entry with no specified address or size.
+   The int is only used with integer constants. */
 struct _cffi_global_s *nglobs;
-Py_ssize_t i, n = PyTuple_GET_SIZE(globals);
+cdl_intconst_t *nintconsts;
+Py_ssize_t i, n = PyTuple_GET_SIZE(globals) / 2;
 
-i = n * sizeof(struct _cffi_global_s);
+i = n * (sizeof(struct _cffi_global_s) + sizeof(cdl_intconst_t));
 building = PyMem_Malloc(i);
 if (building == NULL)
 goto error;
 memset(building, 0, i);
 nglobs = (struct _cffi_global_s *)building;
+nintconsts = (cdl_intconst_t *)(nglobs + n);
 
 for (i = 0; i  n; i++) {
-char *g = PyString_AS_STRING(PyTuple_GET_ITEM(globals, i));
-nglobs[i].type_op = cdl_opcode(g);
-nglobs[i].name = g + 4;
+char *g = PyString_AS_STRING(PyTuple_GET_ITEM(globals, i * 2));
+nglobs[i].type_op = cdl_opcode(g); g += 4;
+nglobs[i].name = g;
+if (_CFFI_GETOP(nglobs[i].type_op) == _CFFI_OP_CONSTANT_INT ||
+_CFFI_GETOP(nglobs[i].type_op) == _CFFI_OP_ENUM) {
+PyObject *o = PyTuple_GET_ITEM(globals, i * 2 + 1);
+nglobs[i].address = _cdl_realize_global_int;
+if (PyInt_Check(o)) {
+nintconsts[i].neg = PyInt_AS_LONG(o) = 0;
+nintconsts[i].value = (long long)PyInt_AS_LONG(o);
+}
+else {
+nintconsts[i].neg = PyObject_RichCompareBool(o, Py_False,
+ Py_LE);
+nintconsts[i].value = PyLong_AsUnsignedLongLongMask(o);
+if (PyErr_Occurred())
+goto error;
+}
+}
 }

[pypy-commit] cffi cffi-1.0: typenames

2015-05-11 Thread arigo

Author: Armin Rigo ar...@tunes.org
Branch: cffi-1.0
Changeset: r1978:6a5911a15fca
Date: 2015-05-11 19:32 +0200
http://bitbucket.org/cffi/cffi/changeset/6a5911a15fca/

Log:typenames

diff --git a/_cffi1/cdlopen.c b/_cffi1/cdlopen.c
--- a/_cffi1/cdlopen.c
+++ b/_cffi1/cdlopen.c
@@ -305,6 +305,30 @@
 building = NULL;
 }
 
+if (typenames != NULL) {
+/* unpack a tuple of strings, each of which describes one typename_s
+   entry */
+struct _cffi_typename_s *ntypenames;
+Py_ssize_t i, n = PyTuple_GET_SIZE(typenames);
+
+i = n * sizeof(struct _cffi_typename_s);
+building = PyMem_Malloc(i);
+if (building == NULL)
+goto error;
+memset(building, 0, i);
+ntypenames = (struct _cffi_typename_s *)building;
+
+for (i = 0; i  n; i++) {
+char *t = PyString_AS_STRING(PyTuple_GET_ITEM(typenames, i));
+/* 't' is a string describing the typename */
+ntypenames[i].type_index = cdl_4bytes(t); t += 4;
+ntypenames[i].name = t;
+}
+ffi-types_builder.ctx.typenames = ntypenames;
+ffi-types_builder.ctx.num_typenames = n;
+building = NULL;
+}
+
 /* Above, we took directly some char * strings out of the strings,
typically from somewhere inside tuples.  Keep them alive by
incref'ing the whole input arguments. */
diff --git a/_cffi1/manual2.py b/_cffi1/manual2.py
--- a/_cffi1/manual2.py
+++ b/_cffi1/manual2.py
@@ -25,5 +25,6 @@
 print ffi.offsetof(struct point_s, y)
 
 print ffi.cast(enum myenum_e, 2)
+print ffi.cast(myint_t, -2)
 
 del ffi, lib
___
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit

[pypy-commit] cffi cffi-1.0: Start reviewing the docs

[pypy-commit] cffi cffi-1.0: Tweak tweak tweak: use an initialization sequence on CPython that is closer

[pypy-commit] pypy cffi-1.0: Passing test

[pypy-commit] cffi cffi-1.0: Make ffi-types_builder an inlined struct; add a ref from the lib to the ffi

[pypy-commit] cffi cffi-1.0: Passing test

[pypy-commit] cffi cffi-1.0: A case where raising NotImplementedError is more appropriate than just

[pypy-commit] cffi cffi-1.0: Python 3 compat

[pypy-commit] pypy vecopt: added some missing vector x86 instructions to mc

[pypy-commit] pypy vecopt: renamed detect_sse2.py to detect_feature.py

[pypy-commit] pypy vecopt: simplifications, added a x86_64 sse4 cpu

[pypy-commit] pypy cffi-1.0: ffi.addressof(lib, var)

[pypy-commit] cffi cffi-1.0: Reasonably messy to implement: ffi.addressof(lib, var)

[pypy-commit] cffi cffi-1.0: in-progress: ffi.dlopen()

[pypy-commit] cffi cffi-1.0: Just enough to pass this small demo

[pypy-commit] cffi cffi-1.0: in-progress

[pypy-commit] cffi cffi-1.0: merge heads

[pypy-commit] pypy fix-result-types: extract W_Ufunc1.find_specialization()

[pypy-commit] pypy fix-result-types: Use the same logic as cnumpy in W_Ufunc1.find_specialization()

[pypy-commit] pypy fix-result-types: Fix casting table

[pypy-commit] cffi cffi-1.0: struct/unions

[pypy-commit] cffi cffi-1.0: enums, integer constants

[pypy-commit] cffi cffi-1.0: typenames

22 matches

Site Navigation

Mail list logo

Footer information