Thanks Andy, Please see https://github.com/apache/arrow/pull/4167#issuecomment-543381089 for the status of the PR. We have a few missing data types (fixed list, timezone to timestamp, etc.) that are currently stopping me from testing the reading of files.
I'm trying out creating a fixed size list, and I'll open a PR for that if my attempt works. On Fri, 18 Oct 2019 at 01:10, Andy Grove <andygrov...@gmail.com> wrote: > Thanks for all the updates. I'd like to get involved and help out with this > effort as well. I don't have any major work planned for DataFusion for > 1.0.0 now other than maybe moving to the new parquet ArrowReader, if it is > ready in time. > > I have been chatting with the author of the Rust Flatbuffer project about > some of the issues and I can take an action to follow up with that. > > I have also been talking with one of the authors of Tonic, and I believe > they might be interested in helping here too. > > Let me know how else I can help out with this effort. > > Andy. > > > > On Thu, Oct 17, 2019 at 2:42 PM Neville Dipale <nevilled...@gmail.com> > wrote: > > > Good evening > > > > With support for testing against integration files now done, I've resumed > > work on the IPC reader. If I don't encounter trouble reading the existing > > files, I expect to be done with this work by the end of the weekend. I > had > > taken the approach of one large PR to include all Rust-supported Arrow > > types. > > > > I'm not sure of how long the writer would take, but I remain committed to > > having this work completed by 1.0. > > > > I have to catch up on null-type roundtrip and the padding alignment work > as > > I haven't been able to keep abreast with development these last few > months. > > Also, the Rust flatbuffer issues that we've had haven't progressed in the > > relevant repo, so it still makes ergonomics not great, but at least our > > users don't have to worry about that. > > > > @Andy I have good experience with gRPC in Rust, and also want to see > Flight > > support landing soon. > > > > On Thu, 17 Oct 2019, 17:06 David Li, <li.david...@gmail.com> wrote: > > > > > Just for reference, it's possible once the basic IPC support is > merged; I > > > had a proof of concept, though it needs to be updated to use Tonic over > > > tower-grpc, actually implement the zero-copy optimizations, provide a > > real > > > API, etc. > > > > > > https://github.com/apache/arrow/pull/4167#issuecomment-529695811 > > > > > > On Thu, Oct 17, 2019, 10:51 Wes McKinney <wesmck...@gmail.com> wrote: > > > > > > > I hope to see Flight in all the reference implementations eventually. > > > > > > > > Having hardened IPC support is a pre-requisite, it would be ideal to > > > > have Rust as a participant in the integration tests > > > > > > > > On Thu, Oct 17, 2019 at 9:41 AM Andy Grove <andygrov...@gmail.com> > > > wrote: > > > > > > > > > > I was approached directly about adding Flight support to the Rust > > > > > implementation, and said I would start a discussion here on the > > mailing > > > > > list. > > > > > > > > > > There is ongoing work with IPC and integration and I believe that > it > > > > would > > > > > make sense to start looking at adding Flight support. > > > > > > > > > > I'd like to hear what others think though. > > > > > > > > > > Thanks, > > > > > > > > > > Andy. > > > > > > > > > >
