Rob created ARROW-7961: -------------------------- Summary: pyarrow 0.16.0 cannot deserialize content serialised with < 0.16.0 Key: ARROW-7961 URL: https://issues.apache.org/jira/browse/ARROW-7961 Project: Apache Arrow Issue Type: Bug Components: Python Affects Versions: 0.16.0 Environment: MacOS, python 3.7 Reporter: Rob
Pandas data frame has been serialised into a Redis cache using pyarrow 0.14.x. After upgrading to 0.16.0 deserialise() fails. Have upgraded to 0.15.1 and this works with objects serialised with 0.14.x {{import pyarrow as pa, redis}} {{print(pa.__version__)}} {{c = redis.Redis.from_url("redis://127.0.0.1")}} {{obj = c.get("breakable")}} {{df = pa.deserialize(obj)}} {{print(df.head())}} {{c.set("breakable", pa.serialize(df).to_buffer().to_pybytes())}} When run venv with 0.15.1 installed. No errors. Version number goes to stdout and no stderror. When run in 0.16.0 following error is generated {{'0.16.0',}} {{ 'Traceback (most recent call last):',}} {{ ' File "/tmp/pa.py", line 6, in <module>',}} {{ ' df = pa.deserialize(obj)',}} {{ ' File "pyarrow/serialization.pxi", line 476, in pyarrow.lib.deserialize',}} {{ ' File "pyarrow/serialization.pxi", line 438, in pyarrow.lib.deserialize_from',}} {{ ' File "pyarrow/serialization.pxi", line 414, in pyarrow.lib.read_serialized',}} {{ ' File "pyarrow/error.pxi", line 99, in pyarrow.lib.check_status',}} {{ 'OSError: Expected IPC message of type unknown but got unknown']}} -- This message was sent by Atlassian Jira (v8.3.4#803005)