Re: [DISCUSS] [Rust] Adding support for Flight protocol

2019-11-17 Thread Neville Dipale
Hi Andy, I've fixed the issue and left a description of the problem on the PR. On Sun, 17 Nov 2019 at 19:23, Andy Grove wrote: > I'm now trying to create a Flight server in Rust and am struggling a bit. > See https://github.com/apache/arrow/pull/5852 for more information. If > anyone is availab

Re: [DISCUSS] [Rust] Adding support for Flight protocol

2019-11-17 Thread Andy Grove
I'm now trying to create a Flight server in Rust and am struggling a bit. See https://github.com/apache/arrow/pull/5852 for more information. If anyone is available to take a look I'd appreciate it. I'm going to reach out to Lucio directly since he isn't currently subscribed to the mailing list II

Re: [DISCUSS] [Rust] Adding support for Flight protocol

2019-10-18 Thread Lucio Franco
Hi all! I am the author of Tonic, I'd love to see the rust flight implementation done with Tonic. David, it looks like what you implemented with tower-grpc should work just fine with tonic as well. I am also interested in helping out. Since, I guess most of my experience up to now is with Tonic

Re: [DISCUSS] [Rust] Adding support for Flight protocol

2019-10-17 Thread Neville Dipale
Thanks Andy, Please see https://github.com/apache/arrow/pull/4167#issuecomment-543381089 for the status of the PR. We have a few missing data types (fixed list, timezone to timestamp, etc.) that are currently stopping me from testing the reading of files. I'm trying out creating a fixed size list

Re: [DISCUSS] [Rust] Adding support for Flight protocol

2019-10-17 Thread Andy Grove
Thanks for all the updates. I'd like to get involved and help out with this effort as well. I don't have any major work planned for DataFusion for 1.0.0 now other than maybe moving to the new parquet ArrowReader, if it is ready in time. I have been chatting with the author of the Rust Flatbuffer p

Re: [DISCUSS] [Rust] Adding support for Flight protocol

2019-10-17 Thread Neville Dipale
Good evening With support for testing against integration files now done, I've resumed work on the IPC reader. If I don't encounter trouble reading the existing files, I expect to be done with this work by the end of the weekend. I had taken the approach of one large PR to include all Rust-support

Re: [DISCUSS] [Rust] Adding support for Flight protocol

2019-10-17 Thread David Li
Just for reference, it's possible once the basic IPC support is merged; I had a proof of concept, though it needs to be updated to use Tonic over tower-grpc, actually implement the zero-copy optimizations, provide a real API, etc. https://github.com/apache/arrow/pull/4167#issuecomment-529695811 O

Re: [DISCUSS] [Rust] Adding support for Flight protocol

2019-10-17 Thread Wes McKinney
I hope to see Flight in all the reference implementations eventually. Having hardened IPC support is a pre-requisite, it would be ideal to have Rust as a participant in the integration tests On Thu, Oct 17, 2019 at 9:41 AM Andy Grove wrote: > > I was approached directly about adding Flight suppo

[DISCUSS] [Rust] Adding support for Flight protocol

2019-10-17 Thread Andy Grove
I was approached directly about adding Flight support to the Rust implementation, and said I would start a discussion here on the mailing list. There is ongoing work with IPC and integration and I believe that it would make sense to start looking at adding Flight support. I'd like to hear what ot