Re: Saving Binary Arrow memory objects as blobs in Cassandra

2019-11-06 Thread Wes McKinney
I suggest you use the IPC protocol http://arrow.apache.org/docs/python/ipc.html This protocol will be considered stable starting with the 1.0.0 release but I would guess (without making any guarantees) that blobs written with 0.15.1 will be readable in 1.0.0 and beyond. On Wed, Nov 6, 2019 at 12

Saving Binary Arrow memory objects as blobs in Cassandra

2019-11-06 Thread Lee, David
Is there anyway to save Arrow memory as a blob? I tried using Feather and Parquet, but neither one supports writing complex nested structures yet. I tried with the following test file. test.jsonl: {"a": 1, "b": "abc", "c": [1, 2], "d": {"e": true, "f": "1991-02-03"}, "g": [{"h": 1, "i": "a"}, {