Re: [VOTE][SPARK-27396] SPIP: Public APIs for extended Columnar Processing Support

2019-04-20 Thread Bryan Cutler
The Arrow data format is not yet stable, meaning there are no guarantees on backwards/forwards compatibility. Once version 1.0 is released, it will have those guarantees but it's hard to say when that will be. The remaining work to get there can be seen at

[Rust] [DataFusion] Parallel query execution PoC

2019-04-20 Thread Andy Grove
I'm now actively working on a PoC of parallel query execution. This is my first time working with multi-threaded Rust so I have reached out to the Rust community for help via a blog post that I posted to Reddit just now. Here is the link to the blog post. There is a link to the Reddit thread in