Re: [pypy-dev] pickle numpy array from pypy to cpython?

[Matti Picus]

So it seems the tests are lacking. Someone should: - go through all the existing calls to dumps in tests and add "assert '_numpypy' not in data" - add tests for scalars - fix so the tests pass Matti On 16/07/16 07:40, David Brochart wrote: To be more precise, PyPy pickling of Numpy arrays works fine, it is when PyPy pickles a Numpy scalar that I get the error. David. On Sat, Jul 16, 2016 at 2:04 PM, David Brochart <david.broch...@gmail.com> wrote: Hi, I verified that this version of PyPy can load a Numpy array that was pickled by CPython (and do stuff with it), but it looks like a Numpy array pickled by PyPy cannot be loaded by CPython, because PyPy still uses  '_numpypy.multiarray' in the pickle string for dumping: ImportError: No module named _numpypy.multiarray David. On Sat, Jul 16, 2016 at 12:07 PM, Matti Picus <matti.pi...@gmail.com> wrote: The issue with '_numpypy.multiarray' in the pickle string rather than 'numpy.core.multiarray' should be fixed on the numpypy_pickle_compat branch (thanks to Eli) A linux 64 build is available http://buildbot.pypy.org/nightly/numpypy_pickle_compat/pypy-c-jit-85727-6d909c810029-linux64.tar.bz2. Eli or David or anyone who uses numpy pickle, could you check that this works as advertised? I am concerned about how compatible our pickling is with upstream numpy, but do not really use that feature of numpy so another pair of eyes would be nice before merging to default. Note this requires that http://bitbucket.org/pypy/numpy be installed since the Unpickler must be able to import numpy.core.multiarray Matti On 15/07/16 10:47, David Brochart wrote: Hi, I'd like to use the (numerical) performances of PyPy as an equivalent to Numba's @jit decorator (https://github.com/davidbrochart/piopio). The only thing preventing that right now is the passing around (pickling) of Numpy arrays, so it would be great to have that compatibility. David. On Mon, Jul 11, 2016 at 6:43 PM, Eli Stevens (Gmail) <wickedg...@gmail.com <mailto:wickedg...@gmail.com>> wrote:     FWVLIW, I think that conforming to upstream numpy makes the most     sense.     I think that the issue would go away if the `_numpypy` module were     renamed to `numpy` (and appropriate things moved into `numpy.core`).     Is there a technical reason to keep the actual implementation in a     separately named module?     Thinking larger picture, would it be possible and sensible to switch     to using the slow cpyext numpy approach for compatability, then     overlay custom implementation of things on top of that when speed is     needed?  I'm imagining a vague inverse of the cpython approach, where     modules are implemented in C when the python performance isn't     acceptable.     Eli     On Wed, Jun 29, 2016 at 10:58 PM, Armin Rigo <ar...@tunes.org     <mailto:ar...@tunes.org>> wrote:     > Hi Eli, hi Matti,     >     > On 29 June 2016 at 21:37, Eli Stevens (Gmail)     <wickedg...@gmail.com <mailto:wickedg...@gmail.com>> wrote:     >> To make sure I'm understanding, are you saying that     upstream/cpython     >> numpy should pick up an alternate way to import multiarray (via     >> _numpypy.multiarray, instead of numpy.core.multiarray)     >     > Hum, in my opinion we should always pickle/unpickle arrays by     > reproducing and expecting the exact same format as CPython's numpy,     > with no warnings.  Any difference (e.g. with complicated dtypes)     is a     > bug that should eventually be fixed.     >     >     > A bientôt,     >     > Armin.     _______________________________________________     pypy-dev mailing list     pypy-dev@python.org <mailto:pypy-dev@python.org>     https://mail.python.org/mailman/listinfo/pypy-dev _______________________________________________ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev

_______________________________________________
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev