Hi Andy, thanks for the VERY quick response and links, I will study them. However, I had in mind an implementation like the example you implemented in the Rust example folder[1], I mean implementing the FlightService trait from the gRPC service. This is why meant to build a FlightData, directly [2]. Would it mean I "just" have to transform my VectorSchemaRoot in a ByteString (set in the dataBody of the FlightData object) that will be streamed over gRPC?
(I wanted to use Scala AkkaStream gRPC to implement my server, because I don't think *org.apache.arrow.flight.FlightServer* will fit my needs.) Or Maybe I didn't understand how a Flight Server should be implemented and I got it wrong? Cheers, Damien [1] https://github.com/andygrove/arrow/blob/master/rust/datafusion/examples/flight_server.rs [2] https://github.com/apache/arrow/blob/master/format/Flight.proto#L300 Le mar. 12 mai 2020 à 21:58, Andy Grove <[email protected]> a écrit : > Hi Damien, > > Here is a brief answer that hopefully at least points you in the right > direction. > > You need to use the VectorSchemaRoot class to build batches of data. There > is some documentation on how to do that [1]. Then, in your FlightProducer > implementation, you need to pass the batches to the > FlightProducer.ServerStreamListener using the "start" and "next" methods > when batches are ready to be sent. There is sample code in the Arrrow repo > [2] and there is a Kotlin example that I wrote here [3]. > > Andy. > > [1] https://arrow.apache.org/docs/java/ipc.html > [2] > https://github.com/apache/arrow/blob/master/java/flight/flight-core/src/main/java/org/apache/arrow/flight/example/ExampleFlightServer.java > [3] > https://github.com/ballista-compute/ballista/blob/master/jvm/executor/src/main/kotlin/BallistaFlightProducer.kt > > On Tue, May 12, 2020 at 6:43 PM Damien Chaillou <[email protected]> > wrote: > >> Hi! >> >> I'm currently playing with Apache Arrow Flight in java and cant get my >> head around how to implement something. >> In the *doGet* method, for example, I'm doing simple JDBC calls that I >> would like to stream over. >> If I understand correctly, the *FlightData*'s body should be a >> ArrowMessage serialised as a ByteString (?) I built from a ResultSet from >> my JDBC call. I though using JdbcToArrow >> <https://github.com/apache/arrow/blob/master/java/adapter/jdbc/src/main/java/org/apache/arrow/adapter/jdbc/JdbcToArrow.java#L149> >> helper >> class to help, but I can't find any example of how to do such thing. >> I came with few questions : >> >> - What is the data we must set in dataBody ? >> - If those are ArrowMessages, how can I map a ResultSet to this type? >> - How do we serialise to ByteString objects typed like ArrowMessage, >> Schema ... ? >> >> >> Could anyone point a piece of code/blog post/anything to me please ? >> >> My toy project would be a generic proxy server in front of any database >> with available JDBC drivers that could stream queries over Arrow Flight >> (gRPC). >> >> Cheers, thanks! >> >> >> Damien >> >
