I would like to use pyarrow with pyzmq no copy to send dicts of arrays across the network without having to make copies of the arrays stored in the dicts.
I've come up with an example, here: https://gist.github.com/syamajala/51d52f5e326ff719bf6231546091991d However, I'm having trouble with deserializing and am seeing this: Traceback (most recent call last): File "zmq_pyarrow.py", line 99, in <module> col.run_collector() File "zmq_pyarrow.py", line 74, in run_collector msg = pa.deserialize_components(comp) File "pyarrow/serialization.pxi", line 450, in pyarrow.lib.deserialize_components File "pyarrow/serialization.pxi", line 307, in pyarrow.lib.SerializedPyObject.from_components File "pyarrow/error.pxi", line 78, in pyarrow.lib.check_status pyarrow.lib.ArrowInvalid: Could not unwrap Buffer from the passed Python object. Does anyone have any tips?