FYI, if you want to see an example server you can run with a GRPC generated client, you can run the ExampleFlightServer located at [1]. Very basic 'test' with that class and client is located at [2].
[1] https://github.com/jacques-n/arrow/tree/flight/java/flight/src/main/java/org/apache/arrow/flight/example [2] https://github.com/jacques-n/arrow/blob/flight/java/flight/src/test/java/org/apache/arrow/flight/example/TestExampleServer.java On Thu, May 24, 2018 at 11:51 AM, Jacques Nadeau <jacq...@apache.org> wrote: > Hey All, > > I used my Strata talk today as a forcing function to make additional > progress on a GRPC-based Arrow RPC protocol [1]. I’m calling it “Apache > Arrow Flight”. You can take a look at the work here [2]. I’ll work to clean > up my work and explain my thoughts about the protocol in the coming days. > High-level: use protobuf as a encapsulation format so that any client that > is supported in GRPC will work. However, we can optimize the read/write > path for targeted languages and hand control the > serialization/deserialization and memory handling. (I did that in this Java > patch [3][4][5].) I also looked at starting to use GRPC generated bindings > within Python but it looks like some glue code may be needed in the C++ > layer since Python delegates down frequently. I also am still trying to > understand GRPC back-pressure patterns and whether the protocol > realistically needs to change to cover real-world high performance use > cases. > > I’ll send out some slides about the ideas and update README, etc. soon. > > Thanks, > Jacques > > [1] https://github.com/jacques-n/arrow/blob/flight/java/flight/ > src/main/protobuf/flight.proto > [2] http://github.com/jacques-n/arrow/ > [3] https://github.com/jacques-n/arrow/tree/flight/ > java/flight/src/main/java/org/apache/arrow/flight/grpc > [4] https://github.com/jacques-n/arrow/blob/flight/ > java/flight/src/main/java/org/apache/arrow/flight/ArrowMessage.java#L253 > <https://github.com/jacques-n/arrow/blob/flight/java/flight/src/main/java/org/apache/arrow/flight/ArrowMessage.java#L253> > [5] https://github.com/jacques-n/arrow/blob/flight/ > java/flight/src/main/java/org/apache/arrow/flight/ArrowMessage.java#L185 > >
