Re: Can't deserialize Arrow Schema with empty metadata?

2021-04-05 Thread Jack Chan
every schema can be serialized-deserialized back in > itself. > > I have the same expectation. > > If metadata can't be serialized with serde_json that would seem like a bug > to me and I think the project would appreciate help fixing it. > > Thanks, > Andrew > &g

Re: Can't deserialize Arrow Schema with empty metadata?

2021-04-04 Thread Jack Chan
:59 AM Andrew Lamb wrote: > This looks like a rust program. Perhaps you could try using `serde_json` or > some other format to narrow down where the issue is > > On Fri, Apr 2, 2021 at 10:00 PM Jack Chan wrote: > > > Hi > > > > I tried to serialize and deserializ

Can't deserialize Arrow Schema with empty metadata?

2021-04-02 Thread Jack Chan
Hi I tried to serialize and deserialize an arrow schema, but I got into an error of (Error { message: "missing field `metadata`" }). Is this a bug in apache arrow or in serde_dynamodb? ``` let schema = ArrowSchema::empty(); let object = serde_dynamodb::to_hashmap(&schema)?;

Re: [DISCUSS] [Rust] Donate Ballista to Apache Arrow

2021-03-11 Thread Jack Chan
Hey Andy I want to discuss the areas of Ballista code that you proposed above to move to Arrow. These are: 1. serde code for translating between protobuf and Arrow/DataFusion/Ballista data structures 2. Distributed query planner 3. Scheduler process that coordinates query execution across availabl